A dynamic scale-mixture model of motion in natural scenes
- Department of Organismal Biology and Anatomy, Department of Physics, Physics Frontier Center for Living Systems, The University of Chicago, United States
Figures
Figure 1
Automated point tracking reveals a diversity of motion statistics across natural scenes.
(A) Natural movie data analyzed via point tracking yields an ensemble of ∼1-s-long point trajectories. (B–D) Raw data summaries for three example movies, (B) bees8-full, (C) trees14-1, and (D) water3. (i) Joint and marginal distributions for horizontal () and vertical () velocity components. Overlaid isoprobability contours for the joint distributions are for B and C and for D. (ii) Seven example horizontal velocity component time series. (iii) Horizontal and vertical velocity correlation functions.
Figure 2
Velocity distributions are jointly heavy-tailed.
(A–D) Velocity distributions for a single example movie, bees8-full. The marginal distribution for horizontal velocity (A) has much heavier tails than a Gaussian with the same variance and is well fit by a Gaussian scale-mixture model. The joint velocity distribution (B) is roughly radially symmetric, which differs substantially from the shuffled (and thereby, independent) distribution (C) and indicates a nonlinear dependence between the two velocity components. This dependence is alternatively revealed by the conditional distribution of the vertical velocity given the horizontal velocity (D), showing a characteristic bow-tie shape.
Figure 3
Schematic of the two-dimensional Gaussian scale-mixture model.
(A–D) A Gaussian random variable (C) is passed through an exponential nonlinearity to yield a log-normal scale variable (B). The scale multiplies both components of an underlying Gaussian distribution (A) to produce radially symmetric heavy tails (D). For the joint distributions, probabilities less than 10−3 were set to zero to facilitate comparison with empirical histograms. (E) Dependency graph for the variables in the model.
Figure 4
Quantifying heavy tails across scenes and categories (A).
Marginal distributions for vertical and horizontal velocity components, grouped by category. (B) Legend of individual movie names for A and all subsequent plots. (C) Marginal distributions for the combined data across categories. Each velocity component of each movie was normalized by its standard deviation before combining. (D) Estimated standard deviations for the scale generator variable, , varied across movies, corresponding to different amounts of kurtosis. (E) The ratio of estimated standard deviations of the underlying Gaussian variables, and , showing the degree of anisotropy. (F) Akaike information criterion (AIC) values for the two-dimensional, shared scale Gaussian scale-mixture (GSM) model versus the two-dimensional, independent Gaussian model. (G) Coding efficiency as a function of signal-to-noise ratio for different values of .
Figure 5
Temporal correlations in velocity and scale.
(A–B) Joint (A) and conditional (B) histograms for horizontal velocity across two adjacent frames for an example movie (bees8-full). The tilt indicates a strong linear correlation, while the elliptic shape in (A) and bow-tie shape in (B) indicate the coexistence of a nonlinear dependence due to an underlying scale variable. (C) The correlation coefficient between velocity component magnitudes, offset in time by , decays as a function of , indicating that the shared scale variable fluctuates in time. (D) The joint distributions of the two components separated by show a gradual transformation from the original radially symmetric shape (Figure 2B) toward the diamond shape of the shuffled distribution (Figure 2C) (red curves are isoprobability contours at p=0.01).
Figure 6
Schematic of the dynamic Gaussian scale-mixture model.
(A–D) Both (A) and (C) are modeled by high-order autoregressive processes to capture arbitrary correlation functions. (Only AR(1) processes are depicted graphically and used to simulate data.) The scale process (B) is generated by passing through an element-wise exponential nonlinearity. It then multiplies the underlying Gaussian process element-wise to yield the observed process with fluctuating scale (D). Only one component is depicted. In the full model, two independent Gaussian processes share a common scale process. (E) Dependency graph for the variables in the one-dimensional model.
Figure 7
Quantifying velocity and scale correlations.
(A–C) Example traces of the raw velocity (A), scale-normalized velocity (B), and estimated scale variable (C). (D) Temporal correlation functions for the underlying Gaussian processes of each movie, grouped by category. Horizontal and vertical components were averaged before normalizing (equivalently, each component was weighted by its variance). (E) As in D, for the scale-generating Gaussian process, . (F) The Gaussian process correlation functions in D averaged within categories. (G) As in F, for the scale-generating Gaussian process correlation functions in E. (H) Lag time to reach a correlation of 0.5 for the underlying velocity Gaussian processes for each movie (components were weighted by variance as in D). (I) As in H, for the scale-generating Gaussian process. (J) Predictive variance explained for each movie. Variances were averaged across horizontal and vertical components before calculating . (K) Akaike information criterion (AIC) values for different models for each movie. Lower values indicate better model fit.
Figure 8
A dynamic scale-mixture model is necessary for effective normalization.
(A) Kurtosis of the velocity before and after dividing by a point estimate of the scale (bottom) under the time-independent model (top). Kurtosis was computed by pooling the two components after normalizing by each standard deviation, so that differences in the variance across components do not contribute additional kurtosis. A Gaussian distribution has a kurtosis of 3 (dashed lines). (B) As in A, but for a model with autocorrelated Gaussian processes and a constant scale for each trajectory. (C) As in A, but for the fully dynamic model.
Appendix 2—figure 1
Tracking performance.
(A) Fraction of trajectories within a given radius of the center of the moving disks. (B) Tracking signal-to-noise ratio for the subset of trajectories within a given radius. (C) Horizontal velocity histogram for the tracking data and ground truth. (D) Horizontal velocity autocorrelations for the tracking data and ground truth.
Appendix 2—figure 2
Velocity histograms.
Histograms for horizontal velocity with and without restriction to trajectories.
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
A dynamic scale-mixture model of motion in natural scenes
eLife 14:e104054.
https://doi.org/10.7554/eLife.104054
