To measure what part of the visual field mice use during prey capture while also considering that mice can use multisensory strategies during prey pursuit (Gire et al., 2016; Langley, 1983; Langley, 1988), we first developed an arena which isolated the visual component of prey pursuit by masking olfactory and auditory spatial cues (Figure 1A, see Materials and methods for details). By removing both olfactory and auditory cues, the average time to capture a cricket approximately doubled compared to removal of auditory cues alone (time to capture, median ± SD, control 24.92 ± 16.77 s, olfactory and auditory cues removed, 43.51 ± 27.82 s, p = 0.0471, Wilcoxon rank sum test, N=13 control and 12 cue removed trials from N = 5 mice). To track mouse head and eye rotations during prey capture, we further developed a lightweight version of our head mounted oculo-videography and camera-based pose and position tracking system (Wallace et al., 2013; Figure 1B and Materials and methods). This approach allowed quantification of head rotations in all three axes of rotation (pitch, roll, and yaw), as well as eye rotations in all three ocular rotation axes (torsion, horizontal, and vertical, Figure 1C, Figure 1—figure supplement 1A and B). The same camera-based system was used to track and triangulate the position of the cricket (see Materials and methods and Figure 1—figure supplement 1C). To quantify the position and motion of the environment and cricket in the mouse's field of view, we also developed a method that enabled a calibrated environment digitization to be projected onto the corneal surface. This approach utilized a combination of laser scanning and photogrammetry, giving a resolution for the reconstruction of the entire experimental room of 2 mm, as well as a detailed measurement of eye and head rotations (Figure 1D–E, and see Materials and methods). Mice, like rats (Wallace et al., 2013), have a large visual field of view which extends to also cover the region over the animal’s head (Figure 1F). To ensure the entire visual fields of the mouse could be captured during behavior, we digitized the entire experimental room and contents (Figure 1E, Figure 1—figure supplement 1D–F, Video 1). The coordinate systems of the environmental digitization and mouse and cricket tracking systems were registered using 16–20 fiducial markers identified in both the overhead camera images and the digitized environment. The average differences in position of fiducial points between the two coordinate systems were less than 1 mm (mean ± SD, x position, 0.18 ± 3.1 mm, y position, 0.07 ± 1.6 mm, z position, 0.66 ± 1.8 mm, N=54 fiducial points from three datasets). The next step was to re-create the view for each eye. First, and for each mouse, the positions of both eyes and nostrils were measured with respect to both the head-rotation tracking LEDs and head-mounted cameras, then calibrated into a common coordinate system (Figure 1B). Together, this enabled a rendered representation of the digitized field of view for each combination of head and eye rotations. This rendered image, from the animal’s point of view, contained all the arena and lab objects (Figure 1G–H, Video 2 and 3, Figure 1—figure supplement 1G). In addition to object position and distance (Figure 1I), the motion of the environment and each object in the field of view could be quantified as the mouse performed prey capture behaviors (Figure 1J, and Figure 1—figure supplement 1H).

Figure 1 with 1 supplement see all

Download asset Open asset

Figure 1—source data 1

Related to Figure 1D.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig1-data1-v1.zip

Download elife-70838-fig1-data1-v1.zip

Figure 1—source data 2

Related to Figure 1G.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig1-data2-v1.zip

Download elife-70838-fig1-data2-v1.zip

Figure 1—source data 3

Related to Figure 1H.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig1-data3-v1.zip

Download elife-70838-fig1-data3-v1.zip

Figure 1—source data 4

Related to Figure 1I.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig1-data4-v1.zip

Download elife-70838-fig1-data4-v1.zip

Figure 1—source data 5

Related to Figure 1J.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig1-data5-v1.zip

Download elife-70838-fig1-data5-v1.zip

Video 1

Download asset

Video 2

Download asset

Video 3

Download asset

Crickets (Acheta domesticus), shown previously to be readily pursued and preyed upon by laboratory mice (Hoy et al., 2016), provided a prey target that could successfully evade capture for extended periods of time (total time for each cricket before capture: 64.4 ± 39.3 s, average time ± SD, N = 21 crickets and three mice, Video 4 and 5). To ensure that only data where the mouse was actively engaged in the detection and tracking of the cricket were used, we identified occasions where the mouse either captured the cricket, or contacted the cricket but the cricket escaped (see Materials and methods for definitions), and then quantified the trajectories of both mouse and cricket leading up to the capture or capture-escape (Figure 2A). Within these chase sequences we defined three behavioral epochs (detect, track, and capture, Figure 2B, see Materials and methods for definition details) based on the behavior of mouse and cricket, and similar to previous studies (Hoy et al., 2016).

Figure 2 with 1 supplement see all

Download asset Open asset

Figure 2—source data 1

Related to Figure 2A,B,C,D,H.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig2-data1-v1.xlsx

Download elife-70838-fig2-data1-v1.xlsx

Figure 2—source data 2

Related to Figure 2E.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig2-data2-v1.zip

Download elife-70838-fig2-data2-v1.zip

Figure 2—source data 3

Related to Figure 2F.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig2-data3-v1.zip

Download elife-70838-fig2-data3-v1.zip

Figure 2—source data 4

Related to Figure 2G.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig2-data4-v1.zip

Download elife-70838-fig2-data4-v1.zip

Video 4

Download asset

Video 5

Download asset

Upon cricket detection, mice oriented and ran towards the cricket, resulting in a significant adjustment to their trajectory (Δ target bearing: 40.2 ± 35.1°, P = 6.20 x 10⁻¹⁰, Δ speed: 10.2 ± 7.4 cm/s, P = 1.91 x 10⁻¹⁰; N=57 detect-track sequences N = 3 mice; Paired Wilcoxon’s signed rank test for both tests), and a rapid reduction in the Euclidean distance to the cricket (Figure 2C). During tracking, the cricket was kept in front of the mouse, resulting in a significant reduction in the spread of target bearings compared to during detect epochs (Figure 2D, Target bearing: detect 6.2 ± 62.1°, track: 2.5 ± 25.6°, mean ± SD, Brown-Forsythe test p = 0, F statistic=7.05x10³, N=4406 detect and 13624 track frames, N=3 mice), consistent with previous findings (Hoy et al., 2016). To avoid the closing phase of the pursuit being associated with whisker strikes (Shang et al., 2019; Zhao et al., 2019), tracking periods were only analyzed when the mouse was more than 3 cm from the cricket, based on whisker length (Ibrahim and Wright, 1975).

Using the detailed digitization of the behavioral arena and surrounding laboratory (Figure 1E, Video 1), an image of the cricket and objects in the environment was calculated for each head and eye position during the predator-prey interaction (Video 2 and 3). Using this approach, we addressed the question of what area of the visual field was the cricket located in during the various behavioral epochs. In the example pursuit sequence in Figure 2E, the cricket was initially located in the peripheral visual field and then transitioned to the lower nasal binocular quadrant of the cornea-view during pursuit and capture (red trace in left eye to blue trace in both eyes). Correspondingly, an average probability density map calculated for all animals during the detect epoch showed a very broad distribution of cricket positions across the visual field (Figure 2F, Figure 2—figure supplement 1A and B). Upon detection the mouse oriented toward the cricket, bringing it toward the lower nasal binocular visual field (Figure 2E, Video 6). When averaged for all pursuit sequences from all animals, projected cricket positions formed a dense cluster on the cornea of both eyes (Figure 2G and H, Figure 2—figure supplements 1A,C–D, 50% contour center for left and right eye respectively, radial displacement from optical axis 64.3 ± 7.5° and 63.3 ± 9.9°, rotational angle 126.2 ± 8.9° and −115.7 ± 6.1°, mean ± SD, N = 3 mice), which was significantly different from the cluster in the detect epoch (average histogram of the location of cricket image during tracking phase vs average histogram of the location of cricket during detect phase: Left eye P = 3.54 x 10⁻⁴⁶, Right eye P = 1.08 x 10⁻⁸¹, differences calculated by taking the Mean Absolute Difference with bootstrapping, N=57 detect-track sequences, N = 3 mice). Thus, during the tracking and pursuit behavior the image of the prey consistently fell on a local and specific retinal area that we refer to from here on as the functional focus. The functional focus fell within the binocular field, while the region of elevated density of RGCs has been found to be located near the optical axis (Dräger and Olsen, 1981), which suggests that the location of the retinal specialization may not overlap with the functional focus.

Video 6

Download asset

To establish the relation between the identified functional focus and the density distribution of RGCs, we made a mouse eye-model (Figure 3A), modified from previous models (Barathi et al., 2008). Using the eye model, retinal spatial locations could be projected through the optics of the mouse eye to the corneal surface. We first reconstructed the isodensity contours quantifying the distribution of all RGCs (Dräger and Olsen, 1981) to define the retinal location with the highest overall ganglion cell density (Figure 3—figure supplement 1A–C, note that these contours are also in agreement with other recently published maps of total RGC density [Bleckert et al., 2014; Zhang et al., 2012]). The lens optical properties were based on a GRIN lens (present in both rats [Hughes, 1979; Philipson, 1969] and mice [Chakraborty et al., 2014]). To determine the optical characteristics of this lens, we developed a method which combined models of the lens surface and refractive index gradient (Figure 3A, Figure 3—figure supplement 1D and Tables 1 and 2, see Materials and methods for details). Using this model, the contours representing the retinal specializations were projected through the eye model onto the corneal surface to determine equivalent corneal locations (Figure 3B, Figure 3—figure supplement 1E). Comparing this location to the functional focus location showed that the region with the highest overall RGC counts and the functional focus (Figure 3B) occupied distinct retinal locations (Figure 3C). Viewed from above the animal’s head, the functional foci were directed at the region in front of the animal’s nose and within the region of stable binocular overlap (azimuth: 1.4 ± 8.8° and −4.4 ± 9.3°, elevation 5.7 ± 2.1° and 4.9 ± 1.4° for left and right eyes respectively, N = 13641 frames, N=3 mice), while the retinal specialization was directed laterally (azimuth: −66.2 ± 6.7° and 70.3 ± 4.7°, elevation: 30.8 ± 12.2° and 41.0 ± 13.5° for left and right eyes respectively, N = 13641 frames N=3 mice. Figure 3D, Figure 3—figure supplement 1F–G). Given that density distributions for different subtypes of RGCs can be spatially heterogeneous with density peaks in distinctly different retinal locations, and that the region of peak density for Alpha-ON sustained RGC’s is spatially located on the dorso-temporal retina (Bleckert et al., 2014), consistent with projecting to the front of the animal, we next quantified whether this region overlapped with the functional focus observed here (Figure 3E).

Figure 3 with 1 supplement see all

Download asset Open asset

Figure 3—source data 1

Related to Figure 3A.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig3-data1-v1.zip

Download elife-70838-fig3-data1-v1.zip

Figure 3—source data 2

Related to Figure 3B.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig3-data2-v1.zip

Download elife-70838-fig3-data2-v1.zip

Figure 3—source data 3

Related to Figure 3C.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig3-data3-v1.xlsx

Download elife-70838-fig3-data3-v1.xlsx

Figure 3—source data 4

Related to Figure 3D.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig3-data4-v1.zip

Download elife-70838-fig3-data4-v1.zip

The average 50% contour of the functional focus was overlapped by the highest density of Alpha-ON sustained RGC's by 35% and 67% for left and right eye respectively (Figure 3E, black, mean ± SD for left and right eye, 35.1 ± 19.8%, 66.7 ± 0.09%, p = 0.095 and 0.019, one-sided Student’s t-test), and the overlap with the second highest density was 83% and 95% (mean ± SD for left and right eye, 82.8 ± 20.1%, 94.8 ± 24.7%, p = 0.042 and 0.003, one-sided Student’s t-test), suggesting a high degree of correspondence between the highest density of Alpha-ON sustained RGC’s and the functional focus during pursuit behavior. Viewed from above the animal’s head the functional foci were directed at the region in front of the animal’s nose azimuth: 1.4 ± 8.8° and −4.4 ± 9.3°, elevation: 5.7 ± 2.1° and 4.9 ± 1.4° for left and right eyes respectively, N = 13641 frames, N=3 mice. The Alpha-ON sustained RGC’s were also directed in front of the animal’s nose (mean ± SD, elevation:16.0 ± 6.9° and 10.8 ± 11.0°, azimuth: −3.6 ± 0.7° and 5.8 ± 7.9° for left and right eyes respectively, N = 168400 frames, N = 3 mice, Figure 3F). Together this suggests that objects falling within the functional foci are processed by Alpha-ON sustained RGC’s.

Eye movements in freely moving mice, like with rats (Wallace et al., 2013), can be large and rapid (Meyer et al., 2020; Payne and Raymond, 2017), and counter head rotations through the VOR, enabling the large field of view around the animals head to be stabilized while the animal is moving. The relationships between head rotations and both the horizontal and vertical eye rotations have recently been quantified, and in addition it has been reported that both during exploration and hunting, mice also have abrupt gaze shifts brought about by the combination of head rotation and conjugate saccade-like horizontal eye rotations (Meyer et al., 2020; Michaiel et al., 2020). We also observed both forms of eye movements in the current study (Figure 4—figure supplement 1). However, how these rotations combine with torsional rotations is not known. If mouse VOR operates similar to that observed in the rat (Wallace et al., 2013), torsional rotations in the mouse will play a significant role in stabilizing the visual field particularly during changes in head pitch. As with the vertical and horizontal rotations (Meyer et al., 2018), torsional rotations in freely moving mice spanned a wide range of rotation angles (Figure 4—figure supplement 2A–D), and were correlated with head pitch (Pearson’s correlation coefficient (r): detect −0.72, 0.58, track: −0.60 and 0.53 for left and right eyes respectively, N=4406 detect and 13624 track frames, N=3 mice, Figure 4—figure supplement 2C–D) as well as head roll (Pearson’s correlation coefficient (r): detect: −0.46, –0.47 track: −0.45 and −0.48 for left and right eyes respectively, N=4406 detect and 13624 track frames, N=3 mice, Figure 4—figure supplement 2L–M), as found previously for freely moving rats (Wallace et al., 2013). As with rats, the rotational relationship between the two eyes was dynamic with different forms of coordination (Figure 4—figure supplement 2E–I), including episodes of in- and excyclovergence (torsional rotation of both eyes toward or away from the nose, respectively) as well as dextro- and levocycloversion (torsional rotation of both eyes to the animal’s right or left, respectively). We next analyzed how effectively rotations of the eye around multiple rotational axes combined to compensate the rotation of the head (Figure 4A, Figure 4—figure supplement 3A–G). We compared movement of the head around its rotational axes and eye movements around the same rotational axes (Figure 4A), effectively defining alternative rotational axes for the eyes to match the axes of the head. Rotation of the eye around these re-defined axes would involve simultaneous rotations in multiple of the eye’s anatomical axes. The gain of this compensation was relatively linear and less than unity for both pitch- and roll-axes, indicating on average under-compensation of the head rotation (slope (gain) of relation for pitch axis, −0.45 ± 0.12 and −0.48 ± 0.06; roll axis −0.51 ± 0.12 and −0.62 ± 0.05 for left and right eye respectively, 168852 frames, N=3 mice). The relatively linear relationships between head and eye rotation around the head pitch and roll axes (Figure 4B) with a transition through the origin suggests that the horizontal, vertical and torsional eye movements are combined to effectively compensate pitch- and roll-related head movements. We next digitally froze each individual eye rotation axis (torsion, vertical, and horizontal) and measured the effect on countering the head rotation (Figure 4C). For rotations around the head x-axis (head pitch changes) the gain of compensation was most affected by freezing torsional rotations (Figure 4C, gain mean ± SD, control: −0.45 ± 0.12and-0.48 ± 0.06; torsion frozen −0.24 ± 0.1 and −0.24 ± 0.01, for left and right eyes respectively, N = 168852 frames, N=3 mice), while freezing vertical or horizontal rotations had more minor effects (Figure 4C, Table 3). The gain of compensation for rotations around the head y-axis (head roll changes) was dramatically affected by freezing vertical rotations (Figure 4C, gain mean ± SD, control: −0.51 ± 0.12 and −0.62 ± 0.05, vertical frozen −0.16 ± 0.14 and −0.17 ± 0.03, for left and right eyes respectively, N = 168852 frames, N=3 mice), with freezing torsion also reducing compensation gain but to a lesser extent (Figure 4C, Table 3). We next quantified the stability and alignment of the animal’s binocular visual field during the pursuit sequences and determined the location of the functional foci within the stabilized region.

Figure 4 with 3 supplements see all

Download asset Open asset

Figure 4—source data 1

Related to Figure 4B,C.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data1-v1.xlsx

Download elife-70838-fig4-data1-v1.xlsx

Figure 4—source data 2

Related to Figure 4D.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data2-v1.zip

Download elife-70838-fig4-data2-v1.zip

Figure 4—source data 3

Related to Figure 4G.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data3-v1.zip

Download elife-70838-fig4-data3-v1.zip

Figure 4—source data 4

Related to Figure 4H.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data4-v1.zip

Download elife-70838-fig4-data4-v1.zip

Figure 4—source data 5

Related to Figure 4I.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data5-v1.zip

Download elife-70838-fig4-data5-v1.zip

Figure 4—source data 6

Related to Figure 4J.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig4-data6-v1.zip

Download elife-70838-fig4-data6-v1.zip

Similar to rats, left and right visual fields overlapped extensively (Dräger and Olsen, 1980; Hughes, 1979), with eye movements creating variability in the extent of the overlap at the edges of the two visual fields, the transition from monocular to binocular (Figure 4D). The functional foci for both eyes were predominately contained within the region of continuous binocular overlap. A horizontal transect through the optical axis for all animals showed a gradual transition from continuous binocular coverage to zero binocular coverage commencing just nasal of the optical axis (Figure 4D, Figure 4—figure supplement 3H and I), indicating that the region of highest overall RGC density spans this transitional region whereas the functional foci are, on average, contained within the binocular region (Figure 4—figure supplement 3H).

We next quantified the variability of alignment of the left and right visual fields within the binocular region, and specifically in the functional focus location (Figure 4E) by using the center of mass (50% isodensity contour center) of the left eye functional focus as an initial reference point and projecting this point to the boundary of a hypothetical sphere surrounding the head. This contact point on the sphere was then re-projected into the right eye to identify the matching location of the left eye (Figure 4E). We then followed the trajectory of the re-projected point in the right eye to get a measurement of alignment variability (Figure 4F, for comparison with the locations in the right eye projected into the left eye see Figure 4—figure supplement 3I–K). While pursuing crickets, alignment precision varied through time (Figure 4G) with the mean alignment of the reference point over all animals and data segments being ~8–9°, which is around the size of V1 cortical neuron receptive fields (~5–15° [Niell and Stryker, 2008], Figure 4H, mean ± SD, left eye projected into right eye 8.8 ± 6.9°; right eye projected into left eye 8.6 ± 6.7°). Repeating this analysis for all points within the region where the probability of binocular overlap was greater than 5% showed that there was a relatively uniform alignment over the entire region (Figure 4I), and that the average alignment error in the functional foci was 8–10°. Coordination of eye movements was important for alignment, as freezing the movements of one eye to its mean position resulted in a significant increase in the alignment error when comparing individual cricket tracking sequences (left all rotations vs. left eye frozen P = 1.78 x 10⁻¹⁰, right eye all rotations vs. right eye frozen P = 7.12 x 10⁻¹¹, N=52 sequences, unpaired Student’s t-test), and a ~54% increase in the mean alignment error over all frames for the reference location (Figure 4I, left eye projected into right eye (left eye frozen) 13.4 ± 8.3°; Right eye projected into left eye (right eye frozen) 13.4 ± 8.3°, mean ± SD, 159318 frames, N=3 mice), which also resulted in a uniform increase in alignment error over the whole overlap region (Figure 4J and Figure 4—figure supplement 3J–L). In summary, during pursuit behavior the functional foci are located in a stable binocular region of the mouse’s visual field. However, in the absence of a mechanism for voluntarily directing its gaze toward a specific target, such as smooth pursuit, tight coupling of VOR-evoked eye movements to head rotations would seem to be restrictive to an animal’s ability to move the target into a specific part of their visual field during pursuit. We therefore next measured what mechanisms mice use to bring the prey into their functional focus.

At detection, mice orient toward their target, aligning their head with the prey and running towards it (Figure 2D), keeping the cricket within a narrow window around its forward direction. This provides a direct way for mice to hold their prey within their binocular visual fields (Figure 4D). We next measured whether additional head or eye movements are used to keep the target within the functional foci. If the mice were actively maintaining the prey within a fixed location of their visual fields, the position of the cricket image should not change as the mouse approaches the cricket. The cricket image location could be maintained by either a head or eye rotation. If they were not actively maintaining the prey in a fixed location, the cricket images should shift downwards in the visual fields as the mouse approaches. To distinguish between these two possibilities, we plotted the cricket positions in the eye views color-coded by the distance between the mouse and cricket (Figure 5A). As the mouse approached the cricket during the track behavioral epoch, the projected cricket positions systematically shifted lower in the visual field (Figure 5A lower). This suggests that the mice did not use additional head or eye movements (Figure 5—figure supplement 1) to bring the cricket into the functional foci, but rather manipulated the cricket’s position in the eye view by orienting and moving towards the target. Consistent with this, head pitch remained stable as the mice approached the crickets (Figure 5B). Furthermore, there was no significant difference in head pitch as a function of distance to the cricket between non-tracking and tracking periods (non-tracking head pitch: −3.7 ± 26.5°, mean ± SD, median = −11.3°, tracking head pitch: −12.9 ± 15.7°, mean ± SD, median = −14.6°, Ks test, P = 0.709, paired Student’s t-test P = 0.197, N = 18 non-tracking and track sequences, N = 3 mice). In addition, and consistent with previous findings (Michaiel et al., 2020), mice did not significantly converge their eyes as they approached the crickets (non-tracking head vergence: 7.6 ± 13.5°, mean ± SD, median = 8.6°, tracking head vergence: 2.5 ± 16.7°, mean ± SD, median = 3.2°. Ks test, P = 0.425, paired Student’s t-test P = 0.225, N=18 non-tracking and track sequences, N = 3 mice, Figure 5—figure supplement 1J and Table 4). These observations suggest that the primary role for the eye movements is stabilizing the visual fields.

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 5—source data 1

Related to Figure 5A–G.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig5-data1-v1.xlsx

Download elife-70838-fig5-data1-v1.xlsx

If mice successfully track and capture prey by retaining the target in front of them then this should be reflected in the trajectories taken by the mice during the tracking epoch compared to the non-tracking behavioral epochs. During cricket tracking periods, mice ran directly toward the cricket, and their trajectories were significantly straighter than during equivalent non-tracking phases (Figure 5C–G). Lateral deviation at the half-way point (Figure 5E, non-tracking 4.3 ± 4.0 cm, tracking 1.4 ± 2.0 cm, p = 0.009), maximum lateral deviation (Figure 5F, non-tracking, 7.7 ± 4.9 cm, tracking 2.8 ± 2.0 cm, p = 0.0006) and the area between the trajectory and minimum distance path to the target (Figure 5G, area under the curve, non-tracking 135.6 ± 102.7 cm², tracking 51.3 ± 45.8 cm², p = 0.0029) were all significantly smaller in the tracking epochs (all comparisons mean ± SD, N=13 tracking and non-tracking sequences, N=3 mice, Wilcoxon’s Rank Sum Test).

Together this suggests that mice do not make compensatory vertical head movements, tracking eye movements or vergence eye movements to keep prey within their functional foci, but instead retain their target within a restricted bearing by running straight towards it. This raised the question of what is unique about the position of the functional focus on the cornea?

Optic flow is the pattern of object motion across the retina that can be self-induced, through eye, head or translational motion, or induced by motion of objects in the environment, or combinations thereof (for review see: Angelaki and Hess, 2005). In a freely moving animal in a still environment, translational motion results in a pattern of optic flow that consists of a radial flow-fields emanating from a point of zero-optic flow (Figure 6A). While optic flow is used by many species for both navigation and the estimation of the motion properties of moving objects, motion induced blur degrades image formation on the retina and decreases resolution depending on the animal’s direction of travel (Land, 1999). Optic flow is minimized in the direction of travel directly in front of the animal (Sabbah et al., 2017), with flow fields directed away from the travel direction and forming a second minimum directly behind the animal’s head (Figure 6A, see also Angelaki and Hess, 2005). To measure the characteristics of optic flow in in both eyes of freely moving mice and to relate this flow pattern to the location of the functional foci, we next calculated average optic flow from freely moving data during pursuit behavior using the digitized environment and eye-views (Figure 6B). First, we calculated optic flow in the idealized case of forward translation motion when all surrounding surfaces were equidistant (Figure 6C). As mice have laterally facing eyes (optical axis = 59.9 ± 19.8° and −62.3 ± 14.7° lateral of frontal for the left and right eyes respectively, N=3 mice), idealized forward motion resulted in the region of minimal optic flow in each eye being located off optical axis in the ventro-medial corneal region representing the animal’s forward direction (radial displacement from optical axis 36.64 ± 0.92° and −41.11 ± 2.27°, rotational angle 122.95 ± 17.05° and −107.94 ± 9.96°, for the left and right eyes respectively, mean ± SD, N=2 mice, Figure 6C). During free movement both the distance from the eyes to objects in the environment, as well as head and eye-rotations had a strong influence on the optic flow fields. We visualized the average flow fields during free motion by calculating the optic flow on the cornea during multiple pursuit trials (N=20 prey chases containing 52 tracking sequences, initial Euclidean distance mouse-cricket >20 cm). The resulting optic flow density maps were complex with a wide range of average speeds (133.44 ± 221.42 °/s, mean ± 1SD, median 28.64 °/s, interquartile range 4.57–137.18 °/s, N=2 mice, Figure 6D). The area of lowest optic flow extended from the nasal part of the field of view to overhead (Figure 6D) but, unlike the simulated case (Figure 6C), optic flow was not symmetric around the regions of minimal optic flow. Optic flow in the 30x30° region surrounding the ventro-medial point of minimal optic flow was significantly lower than that in an equivalent region in the ventro-temporal region during free movement, but not in the simulated case (free movement: nasal 46.3 ± 9.8 °/s, temporal: 199.4 ± 29.0 °/s, p = 0.0014, simulated: nasal 163.6 ± 82.2 °/s, temporal: 833.0 ± 416.5 °/s, p = 0.0662, mean ± SD, two-sided t-test, unequal variance, N=2 mice). Optic flow was higher in the lower visual field and considerably lower in the upper visual field (lower left eye visual field: 262.44 ± 106.50 °/s, upper left eye visual field: 44.87 ± 24.31 °/s, p = 1.78x10⁻²⁰, lower right eye visual field: 361.91 ± 168.80 °/s, upper right eye visual field: 40.59 ± 22.79 °/s, p = 6.68x10⁻¹⁹, Two-sided t-test, unequal variance, N=2 mice), due to the greater distance between ceiling and mouse (distance to floor 2 ± 1 cm, distance to ceiling 308 ± 107 cm, 9873 frames, N=3 mice). Given the advantage of low optic flow to mammalian vision, we next quantified the position of least optic flow during prey tracking. We calculated the location of the translational optic flow minimum in each frame for each eye, and created a probability map of this location over the visual field (Figure 6E). The region of highest likelihood for the presence of the optic flow minimum overlapped considerably with the functional foci in both eyes during the tracking epochs of the pursuit behavior (overlap of optic flow 95% minima and functional foci 50% regions: 100% and 99 ± 1%, overlap of optic flow 50% minima and functional foci 50% regions: 61 ± 14% and 72 ± 4% in left and right eyes respectively, N=3 mice, Figure 6E). Together this shows that the behavioral strategy employed by mice during hunting, consisting of orienting themselves to directly face the prey and following a straight and direct course to it, results in the image of their prey coinciding with the region of reduced optic flow during pursuit, where the retinal image of their prey is least distorted due to motion induced image blur.

Figure 6

Download asset Open asset

Figure 6—source data 1

Related to Figure 6C.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig6-data1-v1.zip

Download elife-70838-fig6-data1-v1.zip

Figure 6—source data 2

Related to Figure 6D.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig6-data2-v1.zip

Download elife-70838-fig6-data2-v1.zip

Figure 6—source data 3

Related to Figure 6E.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig6-data3-v1.zip

Download elife-70838-fig6-data3-v1.zip

Experiments were carried out in accordance with protocols approved by the local animal welfare authorities (Landesamt für Natur Umwelt und Verbraucherschutz, Nordrhein-Westfalen, Germany, protocol number 84–02.04.2017.A260). Experiments were carried out using male C57Bl/6J mice acquired from Charles River Laboratories. At the time of the cricket hunting experiments, mice (n=9) were between 2 and 8 months old, and weighed between 21 and 29 g. Mice were maintained on a 12 hr light/dark cycle. Crickets (Acheta domesticus, Bugs-International, Germany) were housed in 480x375x210 cm cages with ad lib water and food (powdered mouse chow). During experiments in which head and eye rotations were recorded cricket body sizes ranged from 1 cm to 2 cm (1.8 ± 0.3 cm, mean ± SD, n=25).

Animals were anesthetized using fentanyl, medetomidine, and midazolam (50 µg/kg, 5 mg/kg, and 0.5 mg/kg, delivered i.p., respectively), and analgesia was provided with carprofen (7 mg/kg delivered s.c.). Body temperature was maintained using a thermostatically regulated heating pad. Respiration rate and depth of anesthesia was monitored throughout the procedure. Following opening of the skin and removal of connective tissue overlying the sagittal suture and parietal bones, the skull was cleaned with H₂O₂ (3%). A custom-made implant, consisting of a flat circular attachment surface for attachment to the skull, and implant body with three anti-rotation pins and a magnet (Figure 7A–B), was fixed to the dried skull using a UV-curing dental adhesive (Optibond FL, Kerr Corporation, Orange, California, USA) and a UV-curing dental composite (Charisma, Kulzer GmbH, Hanau, Germany). The implant attachment surface and body were made from light-weight, bio-compatible dental resin (Dental SG, Formlabs, Germany). Skin margins were closed with 5/0 Vicryl sutures (Ethicon Inc, Somerville, NJ, USA) and a cyanoacrylate adhesive (Histoacryl, B.Braun, Melsungen, Germany). The injectable anesthetic combination was antagonized with naloxone, atipamezole and flumazenil (respectively 1.2 mg/kg, 0.5 mg/kg and 0.75 mg/kg, delivered i.p.), and the animal was allowed to recover.

Figure 7

Download asset Open asset

Figure 7—source data 1

Related to Figure 7C.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig7-data1-v1.xlsx

Download elife-70838-fig7-data1-v1.xlsx

Figure 7—source data 2

Related to Figure 7E.

https://cdn.elifesciences.org/articles/70838/elife-70838-fig7-data2-v1.zip

Download elife-70838-fig7-data2-v1.zip

The eye cameras for oculo-videography were mounted on mounting arms which were attached to a baseplate with complementary holes to the anti-rotation pins on the implant and fitted with a magnet of complementary polarity. During positioning of the head-camera, mice were anesthetized with isoflurane (induction: 3–5% isoflurane, maintenance: 2.0% isoflurane in air). Anesthetic depth and body temperature were monitored as above. The cameras were positioned to have a sharp image of the entire eye, with the mounting arms adjusted such that the cameras and mounting system caused minimal disruption to the mouse’s lateral and frontal field of view. Mounting arms were secured with cyanoacrylate adhesive glue (Histoacryl, B.Braun, Melsungen, Germany). The eye-camera system was then removed and the animal allowed to recover.

Mice were acclimated to cricket capture in their home cage. Individual crickets were placed in the mouse’s home cage overnight, in addition to their standard ad lib mouse food. Mice were handled and habituated to the experimenter, the head cameras, and the head tracking mounts. The ability of each mouse to visually track the crickets was assessed using the protocol of Hoy et al., 2016. Briefly, the ability of the mice to track and capture crickets in a white walled, 480 x 375 x 210 cm arena was assessed in lit and dark conditions (Figure 7C). Mice were given 2 min in which to capture the crickets. Prior to the assessment mice were food deprived overnight before the trial.

Crenellations along the iridial-pupil border were less distinct in mice than those previous described in rats (Wallace et al., 2013). Ocular torsion changes were therefore measured by tracking the rotations of small spots of titanium dioxide (TiO₂) paste dots (~300 μm) applied to ventral and/or temporal locations on the cornea as described in van Alphen et al., 2010. The TiO₂ paste consisted of TiO₂ powder (Kronos Titan GmBH, Leverkusen, Germany) mixed with a small quantity of sterile artificial cerebrospinal fluid solution with the following composition (in mM): 135 NaCl, 5.4 KCl, 1.8 CaCl2, 1 MgCl2, 5 HEPES, pH balanced to 7.2 (300 mOsm/l). Application of the TiO₂ spots was performed with the animal anesthetized with isoflurane (induction: 5% isoflurane, maintenance: 0.5–1.0% isoflurane in air, total time anesthetized 5-10mins). Anesthetic depth and body temperature was monitored as above. Following application of TiO₂ spots, mice were allowed to recover for >45 min prior to a cricket hunt. The presence of the TiO₂ marks did not significantly change the animal’s cricket hunting performance as assessed by the average time taken to capture crickets (Figure 7C).

Initially, mice were allowed to explore the experimental arena (1x1x0.26 m) without head camera mounts. During subsequent training sessions, mice were acclimated to cricket hunting, with the head cameras on, in the experiment arena. Auditory white noise (60–65 dB, NCH-Tone generator v 3.26, NCH Software, Inc Greenwood Village, USA) was provided through four speakers (Visaton, Germany), one on each wall of the arena. Olfactory noise was provided by ventilating the arena (TD-1000/200 Silent fan, S and P, Barcelona, Spain) through a perforated floor (5 cm perforation spacing) with air blown through a cage containing live crickets (cricket cage dimensions 480x375x210cm). During experiments the arena was lit by a single lamp (4000 K, 9W, Osram, Munich, Germany) positioned ~1 m above the arena. During each experiment, the mouse was given 5 min to explore the arena without head cameras. After this period, the mouse was removed from the arena and the head cameras were mounted. At the commencement of each hunt, the cricket was released at a variable location into the central region of the arena.

Head and eye tracking was performed as described in Wallace et al., 2013, with modifications as described below. The eye camera mount and implant were re-designed to be smaller, lighter and stronger (Figure 7A–B). The camera system body, camera holders and mounting arms were produced using a Formlabs Form2 SLA 3D printer (Formlabs Inc, USA), with Dental SG Resin (Formlabs Inc, USA) as the primary construction material. The cable used for position tracking LEDs power inputs and for data transfer and camera were custom cables (Axon Kabel GmbH, Leonberg, Germany) combined with custom-designed flexible flat cables (IBR Ringler, Bad Rappenau, Germany) for the cameras, to reduce stiffness over the last 30 cm. Eye movements were recorded at 60 Hz (camera resolution 752x480 pixels), with illumination provided by a ring of three IR-LEDs (λ=850 nm, OSRAM SFH4050 or SFH4053 @ 70mA, RS Components, Germany) surrounding the camera lens. The mouse’s head position and head rotations were tracked using seven IR-LEDs (λ = 950 nm, OSRAM SFH4043 @ 70mA, RS Components, Germany) mounted on three struts of carbon fiber that projected from the body of the camera system. The resultant total system weight was ~3 g, including effective cable weight.

The positions of the cricket within the arena were recorded using four cameras (488 x 648 px, recorded at 200 Hz, piA640-210gm, Basler cameras, Basler Ahrensburg, Germany) fitted with NIR-blocking filters (Calflex X, Qioptiq, Germany). Cameras were located ~1.5 m above the arena and were positioned so that the arena was covered at all points by two or more cameras from differing vantage points. Mouse IR-head tracking LEDs were recorded at 200 Hz using four cameras (piA640-210gm, Basler cameras, Basler Ahrensburg, Germany). Image acquisition, synchronization, and mouse head rotation calculations were performed as described previously (Wallace et al., 2013).

Head mount features and mouse anatomical features (medial canthi and nostril positions) were recorded at 50 Hz using four synchronized cameras (acA2040-90um, Basler cameras, Basler Ahrensburg, Germany) fitted with 25 mm focal length objectives (CCTV lens, Kowa Optical Products Co. Ltd, Japan) calibrated as described below. Cameras were positioned to provide images of the animal and headset from different angles to allow triangulation of the anatomical features (Figure 7D). During acquisition of the calibration images, the animal was illuminated with 12 IR-LED modules, (λ = 850 nm, Oslon Black PowerStar IR-LED module, ILH-IO01-85ML-SC201_WIR200, i-led.co.uk, Berkshire, UK) run at 1 A. Position tracking LEDs, medial canthi, nares, mirror corner and camera chip corner positions were marked in two or more camera views, in multiple synchronized frames. Based on the triangulated positions of anatomical features, head cameras and position tracking LEDs the mouse’s eye position could be placed a common coordinate system.

To establish the animal’s horizontal plane from the head tracking LEDs, a position for the animal’s nose was first defined by averaging to 3D positions of the marked nostrils. A pre-forward vector was calculated using the direction between mean of eyes and nose and a pre-up vector as vector orthogonal to the pre-forward and vector between the eyes. Next, the left vector was defined as orthogonal to pre-forward and pre-up. Finally, the system was rotated by 40° around the left vector such that forward vector was elevated. This established a head-fixed forward-left-up coordinate system that was based on the bregma-lambda sagittal plane by tilting the eyes-nose plane by an angle of 40°.

Head tracking frame rates were 200 Hz, while eye tracking cameras recorded at 60 Hz. Eye positions were consequently interpolated as follows: Let

be two rotations that transform the vector ${\left(\mathrm{0,0},-1\right)}^t$ into the gaze vectors $v_{t1}$, $v_{t2}$ in head fixed coordinates at times ${\displaystyle t_1}$, ${\displaystyle t_2}$. Then for a time ${\displaystyle t^{\prime }}$ with

the corresponding rotation $R_{t\text{'}}$ is interpolated such that $v_{t\text{'}}$ is placed on the geodesic defined by $v_{t1}$, $v_{t2}$ with an angle of $s*\angle (v_{t1},v_{t2})$ to $v_{t1}$, and the rotation of a vector perpendicular to ${(\mathrm{0,0},-1)}^t$ is continuous and uniform between ${\displaystyle t_1}$ and ${\displaystyle t_2}$.

Overhead cameras for animal position and cricket tracking, were calibrated as previously described for the overhead cameras (Wallace et al., 2013), with the addition of automated detection of corresponding points in the calibration images using openCV and the eye camera calibration performed as described in Wallace et al., 2013.

Pupil boundary tracking, compensation for eye image displacement, and gaze vector calculation was performed as described previously in Wallace et al., 2013. Where contrast between pupil and iris was insufficient to allow automated pupil position tracking, pupil positions were manually tracked.

The TiO₂ spots for tracking ocular torsion were tracked manually in each image frame. Torsional rotations were determined based on the tracked TiO₂ spot positions as follows. Total rotation of the eye was defined as previously described in Wallace et al., 2013, as:

where $\varphi$ = vertical, $\theta$ = horizontal and $\psi$ = torsional rotations. The mouse’s gaze vector has the coordinates ${\left[\begin{array}{ccc}0& 0& -1\end{array}\right]}^T$ for the reference position of the eye, and in each frame:

With the eye in its reference position, we assume that the marked TiO₂ spot is located in the x-y plane of the eye camera (Wallace et al., 2013). The anatomical location of this marked spot can then be described by two unknown parameters $r$ (where r>1 is the 3D distance of the eyeball surface to the eyeball center, and a distance of 1 describes the rotation radius of the pupil) and $\alpha$ is the fixed angle between the TiO₂ mark and the gaze vector. After eye rotation the 3D location of the TiO₂ is:

and the predicted pixel coordinates of the spot in the image are:

where ${\displaystyle a_{\text{EC}}}$ and ${\displaystyle b_{\text{EC}}}$ are the location in the image of the center of the eye ball and $\frac{f}{z_0}$ a scaling factor, both of which are determined in the calibration procedure for pupil boundary tracking, described in full in Wallace et al., 2013.

When $r$ and $\alpha$ are known the value $\psi$ can be determined. Using the Matlab function fminbnd the squared 2D distance

between the predicted and marked locations of the TiO₂ mark is minimized.

This method is used to determine the ocular torsion based on the TiO₂ spot location, both during and after calibration. Calibration was performed as follows:

For a given $r$ and $\alpha$ choice, $\psi$ can be calculated as above. The sum of square errors in pixel locations is then calculated over all frames. We optimized over $r$ and $\alpha$ using the Matlab function fminsearch. To initialize $r$, we make use of the fact that the pupil model, ${\displaystyle p^{\text{mark}}}$ and $r$ together determine the 3D location of the mark ${\displaystyle v^{\text{mark}}}$ in each image. For each frame, we first calculated:

Using this method, ${\displaystyle {\alpha }_{\text{init}}}$ is estimated separately for each frame, and if the choice of $r$ is correct then these values should agree. We can use fminbind to minimize the following with respect to $r$:

After $r$ is initialized, ${\displaystyle {\alpha }_{\text{init}}}$ is calculated, with $\alpha$ initialized using the mean over frames.

Torsional rotations were normalized by calculating a mean torsion value for the 0.01% of frames that were closest to both median pitch and roll of the head. Torsional values in other tracked frames were then normalized to this mean torsion value.

Cricket body positions were automatically tracked using the method and algorithm described for tracking eye corners, as described in the section ‘Compensation for lateral eyeball displacement – tracking of anatomical landmarks around the eye’ in Wallace et al., 2013. To increase the contrast between the region around the cricket in the image and the cricket, ~100 background image frames (in which neither mouse nor cricket was present) were averaged and subtracted from frames in which the cricket was present. In frames where automated cricket position tracking was not possible, frames were tracked manually. As the cameras used for cricket tracking had been calibrated along with the animal position tracking cameras (see above), the three-dimensional location of the cricket could be triangulated in a common coordinate system with the animal’s position.

To decrease the effects of tracking noise and rapid head rotations, mouse velocity, target bearing and inter-animal Euclidean distances were first filtered using a 50 ms sliding window Gaussian filter.

The criteria used to classify the different hunt phases were based on those described in Hoy et al., 2016. In an initial step, behavioral end points (t_end) for capture periods were identified by manual inspection of the tracking videos. Further identification of the behavioral start points (t_start) and t_end points for the different hunt sequence epochs were then identified as described below.

The t_end points were defined as:

1. The t_end point for a detect period was defined as the last frame before (1) Mouse head velocity in the direction of the cricket was >= 20 cm/s, (2) The mouse’s bearing towards the cricket was constantly below 90° and (3) the Euclidean distance between the mouse and cricket was continuously decreasing.

2. The t_end point for a tracking period was identified by locating local minima in the mouse-cricket Euclidean distance time plots, where local minima were defined as points at which the mouse came within a contact distance of 6 cm (measured from the tracked point on the mouse’s head, giving a > 3 cm separation between the mouse’s nose and the cricket). These were followed either by a capture period (see below) or were followed by a ⩾ 5 cm increase in inter-animal Euclidean distance, which were defined as cricket escapes. In cases where the absolute value of the target bearing was > 90° before the mouse turned towards the prey, the start of the tracking period was taken as the first frame in which the bearing to the target was <90°. Only tracking periods, in which the initial Euclidean distance between the mouse and cricket was >20 cm were analyzed.

3. The t_end point for the capture period was taken to be the point 6 cm away from the cricket, following which a cricket captured and consumed.

The start points of the hunt epochs were defined as follows:

1. The t_start for the detect period was the frame 500 ms prior to the detect t_end point.

2. The t_start for the tracking period was the first frame after the t_end detect frame.

3. The t_start for the capture period was either; (1) the first period in which the mouse approached the cricket and directly caught it, or (2) the first frame in which the mouse approached the cricket and all subsequent cricket escapes (prior to the final cricket capture) were less than 5 cm outside the contact zone (11 cm inter-animal Euclidean distance).

Cases in which the eye cameras were dislodged by the animal during the chase (n=4 hunt sequences) were included in the dataset up until the point where the cameras were dislodged.

Target bearing was defined as the angle between the cricket position and the mouse’s forward head direction in the horizontal plane.

For the digital reconstruction, the company 3dScanlab (Cologne, Germany) was engaged to create a complete scan, photo series and 3D mesh model of the arena and room, which they performed using an RTC 360 3D laser scanner (Leica, Germany). The 3D point cloud produced by the laser scanner was converted to a 3D mesh model, to which textures of the experiment arena obtained from photographs (Nikon D810, 36 Mpx) were baked.

The camera tracking coordinate system, in which the mouse and cricket positions were tracked, and the scanned coordinate system of the 3D mesh model were aligned based on 16 fiducial points which could be clearly identified in both tracking camera images and the scan. Crickets were modeled as 2 cm diameter, 1 cm thick disks centered on their tracked position with the disk's axis oriented parallel to gravity.

Each eye was modeled as a hemisphere with a 180° field of view whose equator was perpendicular to the animal’s gaze vector. For the projection of the environment onto the cornea, frame-wise animal’s eye views for both eyes were created with custom written software in C++ (g++ 7.5.0, QMake 3.1, Qt 5.9.5, libopenexr 2.2.0, libpng 1.6.34 and OpenGL-core-profile 4.6.0) on a GeForce RTX 2070 (NVidia driver 450.66), using first cube mapping followed by a transformation into a spherical coordinate system. To do this, individual frame-wise coordinate transformations were made using the eye locations and orientations determined as described above to transform the mesh model of the arena and cricket to a static eye coordinate system using custom written vertex shaders to perform the coordinate transformation and the fragment shaders to texture the mesh. A cube-map (1024 x 1024 pixels per face) was created by performing such coordinate transformations for a 90 degree view in the direction of the optical axis of the eye and four mutually orthogonal directions. Custom written code was then used to transform the cube-map into a spherical coordinate system, with a 180 degree opening angle, using fragment shaders, resulting in a 1024 x 1024 pixel frame exported as png and OpenEXR files. In addition to the color map, maps of depth (pixel-wise object intersection distance), object identification, optic flow and 3D position of the object intersection point in the contralateral eye’s coordinate system were also generated.

For generation of the prey image probability density maps, animal’s eye views were rendered that contained the cricket only (i.e. without inclusion of arena and room). Density maps from multiple detect-track sequences, and multiple animals, were made by averaging.

Ocular alignment was defined as the consistency of the projection of a given point in the eye view of one eye into the other in an infinitely distant environment. This is equivalent to a projection in an idealized finite-distant spherical environment while assuming a distance between the animal’s eyes of 0. For calculation, the radius of the sphere can then be set to 1 (without loss of generality). A point, located at the center of mass of the functional focus in each eye, was chosen from which to calculate the degree of inter-ocular alignment. This point was projected from one eye to the sphere surface and into the contralateral eye. The degree of alignment between the two eyes was calculated as follows:

Let

be the affine transformations for the left and right eye, and let

be the idealized environment. For a given direction $u\in S$² we calculate the projection into the right eye $p_i\in \mathbb{R}$³ by:

The average alignment is then calculated using the formula:

where ${\displaystyle \overline{\sum }{p}_i}$ denotes mean and ${\displaystyle ⟨⟩}$ denotes normalization.

Visual field overlap was analyzed in the idealized finite-distant spherical environment described above for ocular alignment. Visual overlap was calculated from the frame-wise maps of 3D object intersection points in the contralateral eye (see above section ‘Generation of animal’s eye view’) generated for the ocular alignment analysis: pixels whose 3D object intersection points had an angle of less than 90° to the optical axis were considered part of the overlapping field of view. Probability maps of overlap were calculated by averaging.

For analyses of the effect of freezing eye movements, eye rotations (horizontal, vertical, and torsional) were set to the mean rotation in one eye, and the effect quantified in the other eye view.

To calculate the optic flow in a given pixel for a given eye, we consider the difference vector between the 3D positions in the static eye coordinate system of the object intersection point for this pixel one frame before and after the frame of interest, divided by $2\bullet dt$ and mapped to unit distance by dividing by the distance between eye and interception point. This yields a 3D motion vector which is independent of influences of the frame rate and rendering resolution. The spherical projection used in the rendering process described above is a non-conformal, locally non-isometric map, meaning that angles between lines and distances between points are not preserved. This makes it necessary to evaluate the flow in each point in a local, orthonormal 3D coordinate system defined by the direction vector between the eye position and the object intersection point and derivative vectors along the angular coordinates $v\theta$ and $v\phi$ at that point. Thus, we define the 2D flow at a given point as the orthogonal projection of the 3D flow vector onto the local plane spanned by $v\theta$ and $v\phi$. In this study, we only use the first two components of the vector, while the third component contains the motion in radial direction to the eye.

In Figure 5C, optic flow was calculated for the animal in the idealized spherical environment described above, meaning the animal’s head was equidistant to the surrounding at all points. This simplified scene was characterized as follows. Let

be the coordinate of the center of the mouse’s head, then the scene around it was defined as

with r = 50 cm. For optic flow calculations, the sphere is considered fixed in global coordinates, and the flow is evaluated at the point where the mouse is in the center of the sphere translating forward at a speed of 1 cm/s.

In Figure 5E, optic flow was calculated with the animal in the digitally reconstructed environment (see above).

The points in the scatter plot of optic flow poles in mouse corneal views were color-coded for the density of neighboring points using a two-dimensional Gaussian smoother with standard deviation

For a given point, the density was calculated as:

where $F$ is the set of all considered frame indices, and

where ${\partial }_h{\left[p\right]}_i$ is the discrete central difference quotient of the mouse’s eye trajectory p in frame i, in the coordinate system of the respective eye, evaluated over h=4 frames.

When constructing the eye model, we took experimentally determined values from Barathi et al., 2008 (see Table 1). While we recognize that this study employed a different strain of mice to the one used here, the methodology used provides estimates of physical and optical parameters measured under conditions closest to those relevant for the current study. Further, variation of these parameters was not found to change the model to an extent that would influence the conclusions drawn from analyses involving the eye model (see below). These values distinctly define the spatial shapes and positions of the refractive components of the model eye (Figure 3A), as well as refractive indices for all but the lens, ${\displaystyle n_{\text{lens}}}$. We further assume a pupil radius of 594 µm, which is the mean of constricted and dilated mouse pupil sizes from Pennesi et al., 1998. We define the focal point of a bundle of rays as the point with minimal least squares distance to the rays. To optimize the missing refractive index ${\displaystyle n_{\text{lens}}:\mathrm{\Omega }\to {\mathbb{R}}^{+}}$ inside the lens body ${\displaystyle \mathrm{\Omega }\subset {\mathbb{R}}^3}$, we first calculated two lens models and optimized them such that the focal point of 10000 rays emitted from an object at 10 cm distance on the optical axis lay on the retina. The first model, for optimization of the lens surface, was derived with optimal constant refractive index ${\displaystyle n_{\text{c}}\in {\mathbb{R}}^{+}}$ over the volume. The second model, for lens gradient optimization, was derived with a smooth transition of refractive index to the anterior and posterior lens boundary, that is, ${\displaystyle n_{\text{b}}=1.333}$ on $\partial \mathrm{\Omega }$. We then used Poisson’s equation ${\displaystyle \delta n_{\text{g}}=c}$, and optimized the strength of the gradient ${\displaystyle c\in {\mathbb{R}}^{+}}$. We assumed the final lens model as a linear combination of these two models:

with $\alpha \in \left[\mathrm{0,1}\right]$, where we optimized $\alpha$ as described for the above models, but from a point 10 cm away and 45° off optical axis. The derived refractive indices (Table 2) were within the range measured in Cheng et al., 2019.

To test the sensitivity of the model to changes in assumed physical parameters, we systematically changed the radius of curvatures listed in Table 1, and the thickness listed in Table 2 by 10, 50, and 100 µm (several different values were used, to check the linearity of the dependence). We calculated the propagation of uncertainty through the eye model by analyzing the variation of radial elevation on the retina of the 45 rays (above), taking the numerical differentiation of each input variable that was used in the model. Lens optimization was performed for each newly generated eye model (as described above). The maximum deviations were 0.4, 1.38 and 2.76 degrees for the 10, 50, and 100 µm changes, respectively (Figure 7E), and overall, none of the observed effects on the model would influence the conclusions drawn from the analyses performed using the eye model.

The refractive elements in the rodent eye do not behave like ideally corrected optical elements, with the result that there is a distribution of incident rays with slightly varying angles of incidence on the cornea which converge on any given point on the retina. Projection from retina to cornea therefore requires an estimate of the distribution of outside world angles of incidence for any point of interest on the retina. To do this, we used a Monte-Carlo simulation to back-trace through the optics a set of randomly chosen rays emerging from the point of interest on the retina. Since the intensity of light on a surface with an incoming angle of θ is proportional to $\cos \left(\theta \right)$, this function was also chosen for the probability density distribution of ray exit angles. The rays were then traced until they either hit any opaque surface, resulting in the affected ray being discarded, or passed through the anterior cornea, in which case the ray was accepted and its angle added to the distribution of passing exit angles for the respective point on the retina.

Refraction on boundary layers between different indices of refraction was performed analytically according to Snell’s law. In volumes with a continuous variable refractive index (i.e. gradient-index (GRIN) optics), we used a finite-elements model. We first discretized the lens as a 40x40x40 lattice of side length 2.4 mm. We then started from initial conditions where s(0) is the point of incidence and v(0) is the vector of incidence multiplied by the speed of light c. The subsequent discrete trajectory and direction of propagation is then calculated step-wise according to

The gradient is calculated in the lens lattice as the three-dimensional difference quotient, and then trilinearly interpolated to the exact position $s\left(t_i\right)$ of the ray.

To determine the corneal location corresponding to the histologically identified retinal specialization in the mouse, isodensity lines were redrawn from Dräger and Olsen, 1980 in Illustrator and digitized using Matlab. Isodensity lines enclosing regions containing the highest and second highest density of retinal ganglion cells, as well as the optic disc and outline of the retinal whole mount, were redrawn directly from Figure 3A in Dräger and Olsen, 1981, with horizontal being taken as horizontal (nasal-temporal) in the figure. The isodensity lines were scaled to match the eye diameter used for model eye, then placed into the model eye such that the center of mass of the optic disc reconstructed with the retinal ganglion cell contours was coincident with the intersection of the optic axis and retina in the eye model (Figure 2—figure supplement 1A-C). As the eye model was rotationally symmetrical, no further alignment between the histology and eye model was necessary. The high retinal ganglion cell density regions were then back-projected from retina to cornea as described above (Figure 2—figure supplement 1D-E).

To quantify the effect of head rotations on VOR evoked eye movements in a common coordinate system, head rotations were normalized such that the average pitch and roll were 0. Axes were labeled X and Y respectively and eye rotations were represented using this horizon-aligned X-Y coordinate system. Positive head X values indicate head pitched up, while negative head X values indicate head pitched down. Negative head Y values indicate roll left, while positive Y values indicate roll right. Comparisons of the relationship between head and eye rotations were carried out using differential rotations between frame and average pose, defined in the following way:

are the affine transformations between Cartesian global coordinate system $\mathrm{G}$, head-fixed coordinate system $\mathrm{H}$ and left/right-eye coordiante systems $\mathrm{L}$/$\mathrm{R}$.

The transformations from $\mathrm{L}$/$\mathrm{R}$ respectively to $\mathrm{}\mathrm{H}$ are:

We calculate the left and right eye differential rotations as:

where $\overline{\mathrm{l}}$ and $\overline{\mathrm{r}}$ denote the average transformations over all frames (chordal L2 mean, implementation from SciPy 1.4.1).

Within one experimental trial, the experimentally measured variables of interest are highly correlated with each other. This fact prevents us from using standard statistical tests on the whole time-trace to establish if any difference we observed in the data across different experimental conditions are significant or not, as one requirement of these kind on tests is that the samples from the populations being compared are independent of each other. However, we realized that trial-to-trial variability is the dominant source of variability in the data, whereas within-trial variability explains a smaller fraction of the total variance observed (a more detailed report is found in Table 4). For this reason, we decided to represent each temporal trace by its median value. We used the median and not the mean, because the former is more resistant to the presence of outliers and it is better suited to represent the ‘average’ value of a variable in this context. This operation reduced the size of the dataset to one data points per trial, which we can reasonably assume to be independent of each other.

No external funding was received for this work.

We thank Michael Bräuer, Rolf Honnef, Michael Straussfeld and Bernd Scheiding from the mechanical workshop for fabrication of the setup components. Arne Monsees and Florian Franzen for providing a version of the OpenCV based camera calibration software and Arne Monsees for helping with some aspects of the camera calibration. We also thank Samuel Akorli, Nada Eiadeh, Dennis Franzke, Julia Mauz, Kristina Mendeliene, Anastasia Nychyporchuk and Iftekha Hasan Siraje for assistance with pupil, head and cricket position tracking.

Animal experimentation: Experiments were carried out in accordance with protocols approved by the local animal welfare authorities (Landesamt für Natur Umwelt und Verbraucherschutz, Nordrhein-Westfalen, Germany, protocol number 84-02.04.2017.A260). Experiments were carried out using male C57Bl/6JCrl mice acquired from Charles River Laboratories.

© 2021, Holmgren et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.