Visual pursuit behavior in mice maintains the pursued prey on the retinal region with least optic flow

  1. Carl D Holmgren
  2. Paul Stahr
  3. Damian J Wallace
  4. Kay-Michael Voit
  5. Emily J Matheson
  6. Juergen Sawinski
  7. Giacomo Bassetto
  8. Jason ND Kerr  Is a corresponding author
  1. Department of Behavior and Brain Organization, Research center caesar, Germany
  2. Machine Learning in Science, Eberhard Karls University of Tübingen, Germany
7 figures, 6 videos, 5 tables and 1 additional file

Figures

Figure 1 with 1 supplement
Reconstruction of experimental arena and surrounds from the animal’s perspective.

(A) Schematic of experimental arena with olfactory and auditory noise. (B) Schematic of tracking, anatomical and eye camera calibration. Head position and orientation was tracked using seven IR-LEDs (colored circles). Nostrils (red, yellow filled circles), left (blue filled circle), and right (green filled circle) medial canthi were identified and triangulated in calibration images and used to define a common coordinate system (forward, blue arrow, right, green arrow, and up, red arrow), into which the calibrated eye camera location and orientation could also be placed (eye camera vertical, cyan, horizontal, purple, camera optical axis, red). (C) Example left- and right eye camera images with tracked pupil position (white dashed outlines). (D) Rendered digital reconstruction of the laboratory room and (E) experimental arena. (F) Schematic representation of mouse’s left- (blue) and right (green) visual fields, showing also the region of binocular overlap (yellow) and un-seen region (white). (G) Reconstruction of the arena and room from the animal’s left- and right eye perspective, with monocular and binocular regions colored as in (F). (H) Reconstruction of the animal’s view of the prey (cricket - black) in the experiment arena. (I) Representation of left and right eye views of the arena and surrounding objects grayscale-coded by distance from the eye. (J) Rendered animal’s eye views from the left- and right eyes with overlay of arrows representing optic flow during 10 ms of free motion.

Figure 1—figure supplement 1
Generation of mouse eye views during cricket pursuit.

(A) Head pitch (red), roll (blue) and yaw (orange) and associated left (blue) and right (green) horizontal, vertical and torsional eye movements during the 46.2 s, example cricket pursuit sequence shown in C. (Right) Head and eye rotations during the 0.65 s region between i and ii in the cricket pursuit sequence in C. (B) Example (upper rows) head pitch, roll, and horizontal, vertical and torsional, eye rotations during non-pursuit and pursuit sequences (n=1 animal). Lower rows: head and eye rotations during non-pursuit and pursuit sequences from three mice. (C) Mouse (black) and cricket (orange) paths during a 46.2 s segment of a single pursuit sequence for one animal. (D) Photograph of experiment arena and surrounding environment. (E) Digital rendering of the same experiment arena and surrounding environment. (F) Top-down view of the mouse’s left and right monocular and binocular fields of view (mouse’s head would be centered at the intersection point of monocular and binocular fields of view). (G) Cricket (red) position in the rendered left and right eye corneal fields of view of the experiment arena and surrounding environment during the pursuit sequence in C. (H) Trajectory of the projected cricket position in the left and right corneal views, during the pursuit sequence in C.

Figure 2 with 1 supplement
Mice use a focal region of their visual field to track prey.

(A) Mouse (black) and cricket (orange) paths during a single pursuit sequence (left), and for all pursuit sequences in one session for one animal (right). Pursuit start denoted as filled circles and cricket capture as X. (B) Mouse (red and blue) and cricket (orange) paths during an individual pursuit sequence (left) and all pursuit sequences in one session from one animal (right), showing detect (red) and track (blue) epochs of the mouse path. Paths after a cricket escape shown dashed. Pursuit sequence start shown as filled circles, cricket landing point after a jump shown as a filled triangle. (C) Euclidean distance between mouse and cricket during detect (red) and track (blue) epochs (n=65 trajectories, n=3 mice). (D) Mean and SD bearing to cricket (angle between mouse’s forward direction and cricket location) during detect (red), and track (blue) epochs from all animals (detect: 57 epochs; track: 65 epochs, n=3 animals, bin size = 5°). (E) Trajectory of the projected cricket position in the left and right corneal views, during a single pursuit sequence. Color scheme as for D. The inner dashed circle is 45° from the optical axes. Dorsal (D), ventral (V), nasal (N), and temporal (T) directions indicated. (F) Average probability density maps for detect epochs (4628 frames from three animals). Orientation as in E. (G) Average probability density maps for track epochs (13641 frames from three animals). Orientation as in E. (H) Isodensity contours calculated from the average probability density maps for track epochs. (note that 50% means that this region contains 50% of the total density, and likewise for the other contours). Orientation as in E.

Figure 2—figure supplement 1
Individual corneal prey image heatmaps.

(A) Probability density maps for detect (upper row) and track (lower row) epochs for each of the three animals individually. Data from 4 detect and 5 track sequences, 27 detect and 28 track sequences and 17 detect and 19 track sequences for mouse 1, 2, and 3, respectively. (B) Isodensity contours calculated from the average probability density maps for all detect epochs from all three animals. (C) Isodensity contours for all track epochs from all three animals. (D) 50% isodensity contour (defined as in Figure 2H) during track epochs for each of the three mice (m1-m3) individually.

Figure 3 with 1 supplement
Functional foci are not sampled by the highest density retinal ganglion cell region.

(A) Schematic of mouse eye model (left upper) with profile of all refractive indices (RI, left lower). Reconstructions of the optic disc (black), highest (>8000 cells/mm2, beige) and second highest (>7000 cells/mm2, brown) retinal ganglion cell (RGC) density regions redrawn from Dräger and Olsen, 1981, shown in lower right. (B) Position in corneal views of the high RGC density regions (brown and beige filled regions), and isodensity contours from Figure 2H after projection through the eye model. Orientation as in Figure 2E. (C) Horizontal axis histograms for the nasal half of the corneal view of the second highest RGC region (brown) and 50% isodensity contour for left (blue) and right (green) eyes. (D) Top-down view of the coverage regions for the right eye of the 50% isodensity contour (green, N = 7551 frames) and second highest RGC region (brown, N = 51007 frames) for a single animal. Bars represent the probability density function for the respective regions at that azimuth angle. (E) Position in corneal views of Alpha-ON sustained RGC densities (redrawn from Bleckert et al., 2014) after projection through the eye model. Colored regions show the 95% (dark purple), 75% (medium purple), and 50% (light purple) contour regions of the peak Alpha-ON sustained RGC density. Isodensity contours from Figure 2H. (F) Top-down view of the coverage regions for the right eye of the 95% (dark purple), 75% (medium purple), and 50% (light purple) Alpha-ON sustained RGC contour regions (same as in E, N = 51007 frames) and the 50% isodensity contour from D (green) for a single animal. For the Alpha-ON sustained RGC contour regions, 50% means that this region contains all points which are at least 50% of the peak RGC density.

Figure 3—figure supplement 1
Projecting high retinal ganglion cell density region from retina to cornea.

(A) Retinal whole mount redrawn from Dräger and Olsen, 1981 including whole mount outline (black), and outlines of the optic disc (black) and highest (>8000 cells/mm2, beige) and second highest (>7000 cells/mm2 brown) retinal ganglion cell density isodensity lines. (B) Overlay of the redrawn retinal whole mount from A and a representation of the mouse eye equatorial diameter (dashed) from Tkatchenko et al., 2010. The center of the equatorial diameter was overlaid with the center of mass of the outline of the optic disc of the redrawn whole mount (black cross). Color coding as in A. (C) Retinal isodensity lines represented in spherical coordinates. Color coding as in A. (D) Schematic of mouse eye model (from Figure 3A). (E) Regions within the isodensity contours from A and the 50% isodensity contour from the track epochs from Figure 2H projected through the mouse eye model into the corneal view from the left eye (from Figure 3B). (F) Top-down view of the coverage region for the left eye of the 50% isodensity contour (blue) and second highest RGC region (brown). Bars represent the probability density function for the respective regions at that azimuth angle. Mouse’s forward direction directed to 0°, and mouse’s right directed to 90°. (G) Top-down view of the coverage region for the right eye of the 50% isodensity contour (green) and second highest RGC region (brown). Conventions as in F.

Figure 4 with 3 supplements
Functional foci are located within binocular regions in which motion is stabilized.

(A) Schematic of the common head and eye rotational axes. (B) Relationship between head and eye rotations around the common X (left, 154,625 frames from three animals) and Y (right, 165,432 frames from three animals) rotational axes during pursuit and non-pursuit sequences. Plots show mean for left (blue) and right (green) eyes with standard deviation (gray). (C) Relationship between head and left eye rotations around the common X (left) and Y (right) rotational axes with; all eye rotations present (blue), torsional eye rotations frozen (red), vertical eye rotations frozen (black) or horizontal eye rotations frozen (orange). Plots show mean (lines) and standard deviations (colored filled regions). (D) Corneal view showing probability of overlap of left and right visual fields for one example animal (71995 frames), with overlay of isodensity contours (red) from functional foci (see Figure 2—figure supplement 1D) and contour of second highest RGC region (brown) from Figure 3B. (E) Schematic of inter-ocular alignment. (F) Corneal view of alignment reference point in left eye (left) and variability in alignment of the re-projection of that point in the right eye (right) for a 1.6 s data segment. (G) Kinetics (left) and associated distribution (right) of the variability in ocular alignment for left eye point projected to right eye (blue) and right eye point in left eye (green) for one example data segment (shown in G) from one animal. (H) Distributions of ocular alignment from all data segments (159,318 frames, n=3 mice) with the measured eye movements for left into right eye (blue) and right into left eye (green) and alignment with eye movements frozen (left into right eye, black, right into left, orange). (I) Map of average inter-ocular alignment for all data segments (159,318 frames, n=3 animals) with overlay of isodensity contours from Figure 2H. (J) Map of average inter-ocular alignment as in J with left eye movements frozen.

Figure 4—figure supplement 1
VOR relationships between head and eye rotations and abrupt shifts in gaze.

VOR relationships between head and eye rotations and alignment of left and right eyes. (A) Relationship between mouse head pitch and horizontal eye rotations (left eye (171,942 frames), blue; right eye (364,259 frames), green; mean ± SD). (B) Relationship between head roll and vertical eye rotations. Plot conventions as in A. Left eye (164840 frames); right eye (363,713 frames). (C) Relationship between head pitch and torsional eye rotations. Plot conventions as in A. Left eye (171,942 frames); right eye (364,259 frames). Data for A-C, from three animals. (D) Example trace from one animal of abrupt shifts (gray boxes) in gaze (lower) resulting from combined head rotation (upper) and conjugate horizontal eye rotations (middle).

Figure 4—figure supplement 2
Ocular torsion during cricket pursuit.

(A) Distribution of left (blue) and right (green) eye torsional rotations during detect epochs. Data from 57 epochs (4406 frames) from three animals. (B) Distribution of ocular torsion during track epochs. Conventions as in A. Data from 65 epochs (13624 frames) from three animals. (C) Average relationship (mean ± SD) between head pitch and torsional eye rotations during detect epochs for left (blue and right (green) eyes). Data from 57 epochs (4406 frames) from three animals. (D) Average head pitch and torsional eye rotations relationships during track epochs. Conventions as in C. Data from 65 epochs (13,624 frames) from three animals. (E) Mouse (detect epoch, blue; track epoch, red) and cricket (black) trajectories during one example pursuit sequence. (F) Torsional rotations of the left (blue) and right (green) eyes, and head pitch (purple) and roll (orange), during the pursuit sequence in E. Lower panels show example eye images from the indicated time points in the kinetic traces. Red arrows indicate tracked TiO2 spots. (G) Example sequences showing torsional rotation kinetic traces for left (blue) and right (green) eyes during in- (i) and excyclovergence (ii) from one pursuit sequence. Schematics show the ocular rotations in the left and right eyes. (H) Example sequence showing dextrocycloversion in one pursuit sequence. Conventions as in G. (I) Example of the effect of torsional rotations on prey image location. Corneal eye views of the cricket (black ellipse in red dashed circle) and arena (upper) and associated eye images (middle) at the time points indicated in the torsion kinetic traces (lower) for the left (blue) and right (green) eyes. Note cricket trajectories in left and right corneal eye views, which show the trajectory of the cricket in the corneal views between time points 1 and 2. Red arrows in eye images show TiO2 torsion tracking spots. (J) Performance of a model predicting torsion based on head pitch alone for left (blue) and right (green) eyes during detect and (K) track epochs. (L) Average (mean ± SD) relationship between head roll and torsional eye rotations during detect epochs for left (blue) and right (green) eyes. Data from 57 epochs (4406 frames) from three animals. (M) Average head roll and torsional eye rotation relationship during track epochs. Conventions as in L. Data from 65 epochs (13,624 frames) from three animals. (N) Performance of a model predicting torsion based on both head pitch and roll. Conventions as in J. For both J and N, data taken from 57 detect epochs (4406 frames), from three animals. (O) Performance of a model predicting torsion based on both head pitch and roll during tracking phases. For both K and O, data taken from 65 prey tracking epochs (13,624 frames), from three animals.

Figure 4—figure supplement 3
VOR relationships between head and eye rotations and alignment of left and right eyes.

(A) Image of mouse with detachable miniaturized eye cameras and head position tracking system. (B) Example eye images showing horizontal, vertical and torsional eye rotations. Note TiO2 spots on the corneal surface for tracking torsion highlighted in lower panels. (C) Schematic of the common head and eye rotational axes. Relationship between (D) left and right eye X-rotations, (E) Y-rotations and (F) Z-rotations in common rotational axes. (G) Relationship between head X rotations and eye Z rotations for left eye (blue) and right eye (green). Data for D-G are represented as mean ± SD, and are from 168,400 frames from three animals. (H) Corneal view showing probability of overlap of left and right visual fields for two example animals m1 (left, 36,449 frames) and m2 (right, 50,874 frames), with overlay of isodensity contours (m1 -black, m2 - blue) from functional foci (see Figure 2—figure supplement 1D) and contour of second highest RGC region (brown) from Figure 3B. (I) Profile of probability of overlap for left (dotted) and right (solid) eyes as a function of angular distance from optical axis for all three animals. Profile taken from horizontal axis through optical axis as shown in Figure 4D (dotted line in 4D, N = 3 animals, green = 36,449 frames, blue = 50,874 frames, purple = 71,995 frames, respectively). Examples of ocular alignment for the reference spot in the left eye projected into the right eye (J), reference spot in the right eye projected into the left eye (K), and alignment over time for both reference spots (L).

Figure 5 with 1 supplement
Mechanisms used to maintain prey within a focal visual region.

(A) Corneal locations of cricket position color-coded by Euclidean distance to cricket for non-track (upper) and track (lower) epochs (18 data sequences, 15649 non-tracking and 8510 tracking frames, n=3 animals). (B) Mean and SD head pitch with Euclidean distance to cricket (left) and distribution of head pitch angles (right) for non-track (red) and track (blue) epochs (datasets as in A). (C) Mouse trajectories during non-track epochs rotated and overlaid to show deviation from a direct path (13 trajectories from three animals). (D) Mouse trajectories as in C but during track epochs (13 trajectories from three animals). (E) Histogram of lateral deviations for non-track (red) and track (blue) data in C and D calculated 20 cm from the end of the trajectory. (F) Boxplots and individual data points of absolute maximal lateral deviation from a direct path between start and end points for non-track (red) and track (blue) epochs (datasets as in C and D), ** p = 0.0006, Wilcoxon’s Rank Sum Test. (G) Boxplots and individual data points of area under the curve (AUC) of mouse trajectories during non-track (red) and track (blue) epochs (datasets as in C and D), ** p = 0.0029, Wilcoxon’s Rank Sum Test.

Figure 5—figure supplement 1
Eye movements during non-tracking and tracking periods.

(A) Violin plots showing the variability in horizontal eye rotations for left (blue) and right (green) eyes during non-tracking (Non-trk) and track (Track) epochs. (B) Variability in vertical eye rotations during non-tracking and track epochs. Conventions as in A. (C) Variability in torsional eye rotations during non-tracking and track epochs. Conventions as in A. (D) Variability in ocular vergence during non-tracking and track epochs. Conventions as in A. (E) Average relationship (mean ± SD) between head roll and Euclidean distance from mouse to cricket during non-track (red) and track (blue) epochs. Data histogram shown at right. (F) Average relationship (mean ± SD) between vertical eye rotations of left (blue) and right (green) eyes and Euclidean distance between mouse and cricket during non-track epochs. Data histogram shown at right. (G) Average relationship between vertical eye rotations and mouse-cricket Euclidean distance during track epochs. Conventions as in F. (H) Average relationship between torsional eye rotations and mouse-cricket Euclidean distance during non-track epochs. Conventions as in F. (I) Average relationship between torsional eye rotations and mouse-cricket Euclidean distance during non-track epochs. Conventions as in F. (J) Average relationship between ocular vergence and mouse-cricket Euclidean distance during non-tracking and tracking epochs. Conventions as in E. For all panels, data taken from 18 non-track epochs and 18 track epochs, from three animals.

Functional foci are located in the regions of reduced optic flow during forward motion.

(A) Schematic of idealized optic flow (black arrows) as a mouse translates forwards (after Sabbah et al., 2017). Left (blue arrow) and right (green arrow) gaze vectors. (B) Location of optic flow minima in reconstructed mouse eye views of the cricket and experiment arena (from Figure 1H), dashed circle represents 45°. (C) Optic flow map in corneal views, showing flow velocity (color coding) and direction (white arrows) calculated for the idealized spherical environment in 6A with forward motion of 50 cm/s. (D) Optic flow maps in corneal views during track epochs (5269 frames), from one animal. (E) Probability density map of optic flow poles in mouse corneal views during track epochs (data as in Figures 2G, 13,641 frames), with overlay of isodensity contours from Figure 2H.

Methods.

(A) Implanted baseplate with magnetic attachment point and restraining pin holes. (B) Miniaturized eye cameras and head position tracking system. Infrared illumination LEDs were mounted on the camera objective and reflected onto the eye using an IR-reflective mirror. Head position tracking IR-LEDs were mounted on three carbon-fiber struts attached to the head-mount. (C) Cricket capture times in lit or dark conditions in mice without (n=19 pursuit sequences, n=6 mice) or with (n=10 pursuit sequences in lit conditions and n=9 pursuit sequences in the dark, n=3 mice) corneal TiO2 torsion tracking spots, Lit vs Dark with no spot, p = 0.0012, Lit vs Dark TiO2 spot, p = 0.0133, Lit without spot vs Lit with TiO2 spot, p = 0.69, Dark without spot vs Dark with TiO2 spot, p = 1. n.s. = non-significant, *p<0.05, **p<0.01. Paired Wilcoxon’s signed rank tests. For these experiments, pursuits were conducted in a smaller arena (480 x 375 x 210 cm). (D) Images of mouse with eye camera and head position tracking system for anatomical calibration. Head mount and anatomical features marked. Anatomical features: Left (blue filled circles) and right (green filled circle) medial canthi, left (orange filled circles) and right (red filled circle) nostril positions. Head mount features: position tracking LEDs (large colored circles), IR mirror corner positions (small colored filled squares). (E) Sensitivity of the radial elevation on the retina in the mouse eye model to changes in the radii of curvature and thicknesses of the model optical components.

Videos

Video 1
Digitized and rendered view of the experiment arena and surrounding environment.

Laser scanned and digitally reconstructed experiment environmental, providing positional information of objects within the mouse’s environment. When combined with the tracked 3D cricket positions and the tracked mouse head and eye positions and rotations this allowed the generation of a frame-by-frame mouse eye view of the prey and the surroundings.

Video 2
Reconstruction of the mouse’s left and right eye field of view during one example behavioral sequence.

Real speed.

Video 3
Reconstruction of the mouse’s left and right eye field of view during one example behavioral sequence, as shown in Video 2, but slowed to 0.5x real speed.
Video 4
Left and right eye camera images and one overhead camera view showing one complete cricket pursuit, from shortly after release of the cricket into the arena to cricket capture.

Real Speed.

Video 5
The same cricket pursuit as shown in Video 4 but slowed to 0.5x real speed.
Video 6
Reconstructed left and right mouse-eye views for one example pursuit behavioral sequence, showing the trajectory of the cricket position in the eye views during the detect (red) and track (blue) segments of the behavior.

Tables

Table 1
Mouse eye model curvatures.

Radii of curvature of the optical components of the mouse eye model in Figure 3A.

Ocular ComponentRadius of curvature (μm)
Anterior Cornea−1408*
Posterior Cornea−1372*
Anterior Lens1150*
Posterior Lens1134*
Retina1598*
  1. * Values from Barathi et al., 2008.

Table 2
Mouse eye model thicknesses and refractive indices.
Ocular ComponentThickness(μm)Index of refraction
Cornea92*1.402*
Anterior chamber278*1.334*
Lens2004*1.36–1.55
Vitreous chamber609*1.333*
  1. * Values from Barathi et al., 2008.

    † Minimum and maximum values after eye model optimization.

Table 3
Compensation gain of eye rotations for head X or Y-axis rotations.

Effect of digitally freezing torsional, vertical, and horizontal eye rotations on the gain of compensation of X and Y head rotations. Data taken from 168,852 frames, from three animals.

EyeRotation directionRotationAll Rotations
(mean ± SD)
Eye rotation frozen
(mean ± SD)
LeftXTorsion−0.45 ± 0.12−0.24 ± 0.1
Horizontal−0.45 ± 0.12−0.32 ± 0.06
Vertical−0.45 ± 0.12−0.35 ± 0.08
RightXTorsion−0.48 ± 0.06−0.24 ± 0.01
Horizontal−0.48 ± 0.06−0.36 ± 0.08
Vertical−0.48 ± 0.06−0.34 ± 0.03
LeftYTorsion−0.51 ± 0.12−0.35 ± 0.05
Horizontal−0.51 ± 0.12−0.51 ± 0.11
Vertical−0.51 ± 0.12−0.16 ± 0.14
RightYTorsion−0.62 ± 0.05−0.45 ± 0.05
Horizontal−0.62 ± 0.05−0.62 ± 0.02
Vertical−0.62 ± 0.05−0.17 ± 0.03
Table 4
Eye rotations during non-tracking and tracking periods.

Horizontal, vertical, and torsional eye rotations during the non-tracking and tracking periods in Figure 5. Data taken from 18 non-track epochs and 18 track epochs, from three animals.

Ocular RotationNon-Trk
(mean ± SD) (median)
Track
(mean ± SD) (median)
p value (KS)P value (Student
T-test)
Lt Horizontal−1.8 ± 9.9°
(−1.7°)
−1.8 ± 14.9°
(−3.5°)
3.9x10−20.162
Lt Vertical0.8 ± 11.2°
(−0.4°)
4.5 ± 11.1°
(4.9°)
0.4250.616
Lt Torsional2.9 ± 16.1°
(0.0°)
1.3 ± 20.6°
(0.0°)
0.9450.610
Rt Horizontal5.7 ± 10.9°
(5.5°)
1.0 ± 9.9°
(1.7°)
9.82x10−21.08x10−2
Rt Vertical−3.6 ± 13.4°
(−6.3°)
5.6 ± 12.7°
(−7.1°)
0.9450.804
Rt Torsional0.32 ± 13.5°
(0.0°)
0.7 ± 12.3°
(0.0°)
0.4250.366
Key resources table
Reagent type (species) or resourceDesignationSource or referenceIdentifiersAdditional information
Software, algorithmMatlabMathworksMatlab 2019b
Software, algorithmOpenJDKOracleVersion 1.8.0_292
Software, algorithmCudaNvidiaRelease 10.1, V10.1.243
Software, algorithmPythonPython Software FoundationPython 3.8.10
Software, algorithmQtQt ProjectQmake 3.1, Qt 5.9.5
Software, algorithmOpenGLKhronos Group/Nvidia/AMDVersion 4.6.0

Additional files

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Carl D Holmgren
  2. Paul Stahr
  3. Damian J Wallace
  4. Kay-Michael Voit
  5. Emily J Matheson
  6. Juergen Sawinski
  7. Giacomo Bassetto
  8. Jason ND Kerr
(2021)
Visual pursuit behavior in mice maintains the pursued prey on the retinal region with least optic flow
eLife 10:e70838.
https://doi.org/10.7554/eLife.70838