Introduction

Assessing visual abilities was already important in historic times¹, and the precise measurement of visual acuity, our ability to resolve fine spatial detail by eye, has great importance for many real life scenarios and is up to this day the primary diagnostic tool to determine visual function in a clinical and optometric setting. Quite surprisingly, the widely-believed assumption that the packing density and arrangement of retinal photoreceptors at the foveal center set the limit to this ability has never been experimentally confirmed.

Fovealization, the morphological and functional specialization of the cellular architecture of the light sensitive retina optimizes the human eye for high-acuity daytime vision^2,3. Within the central one-degree diameter of the fovea, termed foveola, postreceptoral neurons are displaced centrifugally and the area is free of potentially shadowing blood vessels and glia cells^4,5. The outer segments of foveolar cone photoreceptors are maximally thinned and densely packed for peak spatial sampling^6–8, which at the same time makes these cells the most difficult to study ex vivo⁹ as well as in vivo¹⁰. Each foveolar cone synapses to one ON- and one OFF-midget bipolar cell, which in turn synapse exclusively upon single ON- and OFF-midget ganglion cells, a circuit that is adult like before birth¹¹. This establishes an undisturbed private line from individual foveal receptors to central processing stages.

Based on indirect comparisons between histological and psychophysical data, the hypothesis that cone spacing imposes the fundamental limit for visual resolution has been put forward^8,12. It is well established that cone spacing, especially in the central fovea, is highly variable between individuals^12–15, making general comparisons between acuity measurements and foveolar density estimated from histological samples susceptible to error. One of the main reasons why the hypothesis lacks direct experimental proof is that because, under natural viewing conditions, both visual resolution and experimental access to foveal photoreceptors is blurred by the imperfect optics of the human eye^16,17. Here, we have overcome the optical barrier of the human eye by employing adaptive optics cell-resolved in vivo retinal imaging in conjunction with micro-psychophysics to study directly whether the individual’s mosaic of foveolar cones determines visual performance in a high-acuity resolution task. Our findings may also resolve another, so far only indirectly tested, hypothesis if a potential visual resolution advantage arises in myopic eyes. Myopes, despite retinal stretching, generally have a higher angular sampling density in and around the fovea compared to emmetropes¹³.

While acuity is assumed to be mainly limited by the resolving capacity of the eye’s optics and retinal mosaic, it is well established that, for different visual tasks, performance thresholds can be substantially lower than the sampling grain of photoreceptors. This phenomenon has been termed hyperacuity¹⁸ and depends on the neural visual system’s ability to extract subtle differences within the spatial patterns of the optical image on the retina¹⁹. Thus, the visual system already incorporates mechanisms to detect relative spatial offsets an order of magnitude smaller than the spatial granularity of the retina. To make use of those fine distinctions in a resolution task, the neuronal system needs to go beyond purely spatial coding of incoming signals.

Unlike a camera, the visual system depends on temporal transients arising in the receptor’s cellular signals. Neurons in the retina, thalamus and later stages of the visual pathways respond strongly to temporal changes^20,21. Thusly, the fovealized retinal architecture in humans is accompanied by a dynamic sampling behavior that, by quick and precise movements of the eye, brings retinal images of objects of interest to land in the foveola^22,23. Even during steady fixation, for example of a distant face or a single letter of this text, incessant fixational eye movements slide tens to hundreds of foveolar photoreceptors across the retinal image, thereby introducing temporal modulations that translate spatial activation patterns into the temporal domain²⁴. Small and rapid gaze shifts known as microsaccades relocate gaze within the foveola during periods of fixation²², and between microsaccades, the eyes perform a more continuous, seemingly random motion termed fixational drift^25,26.

Computational work suggested that fixational eye motion would introduce noise and thus impair visual acuity^27,28. Contrarily, recent studies on human psychophysics demonstrated fixational eye motion to be beneficial for fine spatial vision^26,29,30. Especially drift motion has been increasingly argued to not just be randomly refreshing neural activity, but rather structuring it^26,31,32 and being under central control³³.

The incessant motion of the eye conveys fine spatiotemporal detail that requires deciphering of continuously changing photoreceptor signals, which are linked by the geometry of the photoreceptor array and by how the eye moves. For instance, luminance modulation in individual cones will scale with drift amplitude. Larger luminance variations on single receptors also yields more neuronal activity within the range of temporal frequencies parvocellular ganglion cells are sensitive to. Selective spatial frequencies can thus be amplified by varying drift amplitude²⁶. While the neuronal mechanisms that generate fixational drift are still not fully understood³⁴, its consequence to visual perception has been demonstrated. Drift was shown to improve visual performance in resolution tasks^26,29,35, and a recent model of early retinal signals suggests that if drift amplitude is tuned to object size, visual acuity would be enhanced³⁶. Indeed, considerable differences in ocular drift between individuals exist^32,37, and subjects exhibiting less drift were shown to have better acuity³². If such differences are a consequence of an active, adaptive mechanism, however, and how drift behavior is related to the photoreceptors that sample the retinal image is unknown.

The direct experimental access to the foveolar center, when other limiting factors like image blur or retinal motion are taken out of the equation or can be precisely measured, will allow to confirm or reject the long-standing hypothesis about the individual limits of vision. This will help to understand the fundamental physiological limitations of the visual system and will have important implications for clinical studies of retinal health.

Results

Resolution is finer than single cone sampling limits

We investigated the limitations of the photoreceptor packing density on individual visual resolution acuity by overcoming the optical aberrations of the eye with adaptive optics scanning laser ophthalmoscopy (AOSLO), while simultaneously performing psychophysical measurements and recording the fixational retinal motion (Fig. 1a, b and c). In a four-alternative forced-choice task, 16 healthy participants indicated the orientation of an E-optotype while inspecting the stimulus with their individually preferred fraction of foveolar photoreceptors. These cone photoreceptors were simultaneously imaged and it was later identified which cells contributed to resolving the stimulus (Fig. 1c, f). A psychometric fit to the data expressed as percentage correct from 100 trials was used to compute visual acuity thresholds (see online Methods and Fig. 1d,e). In this near diffraction-limited testing condition, participants reached visual acuity thresholds between 20.6 and 28.5 arcsec (mean ± SD: 24.1 ± 2.4 arcsec), which compares to 20/8 vision (logMAR = −0.4). All participants reached thresholds better than 20/10 vision (logMAR = −0.3), the last line of a typical clinical Snellen chart or projectors of acuity optotypes that are used in clinical as well as optometric daily routine.

Fig. 1:

Cone densities at the CDC ranged between 10,692 and 16,997 cones/deg², with an average density of 13,640 cones/deg² (PCD mean: 13,944 cones/deg², range: 10,823 to 17,309 cones/deg²), comparable to previous reports^{13,14,38–41}. The median sampling cone density ranged between 10,297 and 16,104 cones/deg² (mean: 13,149 cones/deg²). Two experimental runs of the eyes with highest and lowest sampling density are exemplarily shown in Supplementary Movie 1 and 2. The two foveolar cone mosaic images were also visualized and overlayed with a Snellen E stimulus at average threshold size (Fig. 2a). The theoretical prediction, given by the Nyquist sampling limit, would assume the high-density retina where each single cone diameter is smaller than the Snellen E’s gap or bar is able to resolve the stimulus, whereas the low-density retina fails in identifying the correct orientation (schematic representation in Fig. 2b). However, for our 788 nm testing condition, all participants reached individual resolution thresholds below their Nyquist limit predicted by the spacing between rows of cones (Fig. 2c, d). On average, visual acuity thresholds exceeded the theoretical prediction by 20 % and 16 % in dominant and non-dominant eyes, respectively. When participants performed the same resolution task with a longer infrared wavelength (840 nm) imaging background, the absolute thresholds were slightly higher and thus closer to the Nyquist limit. Visual acuity thresholds were on average 7 % below and 2 % above the Nyquist limit for dominant and non-dominant eyes, respectively. These absolute visual acuity thresholds were the only case where noteworthy differences arose between the 788 nm and 840 nm experimental condition. For all other analyses, we found qualitatively similar results for either wavelength and therefore only report the 788 nm results throughout the manuscript.

Fig. 2:

For the first time, we could measure the direct relation between the individual foveolar cone photoreceptor sampling density and participants visual resolution thresholds. We found the diffraction limited visual acuity thresholds to be strongly correlated to the foveolar sampling density in dominant as well as fellow eyes (Fig. 2d). The higher the cone density, the smaller the visual stimulus that could be resolved. The degree of correlation slightly differed for dominant (r² = 0.45, p = 0.005) and non-dominant eyes (r² = 0.28, p = 0.036), suggesting that up to 45 % of the variance in inter-subject visual acuity can be explained by the individual cone sampling densities. Overall, participants reached significantly lower thresholds with their dominant eyes (average: 1.5 arcsec, SD ± 1.1; paired t-test, p < 0.001). Nevertheless, visual acuity thresholds were strongly correlated between dominant and non-dominant eyes (r² = 0.80, p < 0.001, Fig. 2e, see supplementary discussion). To test whether the effect of different absolute thresholds might be explained by underlying differences in the sampling cone density, fellow eyes densities were compared to each other. Sampling densities had a very strong correlation between fellow eyes (r² = 0.85, p < 0.001, Fig. 2e), but did not differ between right and left eyes (p = 0.38) nor when grouping them according to ocular dominance (p = 0.88). This compares well to previous studies that also showed strong correlations between fellow eyes regarding both anatomical¹⁴ as well as functional¹⁵ characteristics. Dominant eyes had a median of 78 cones/deg² higher densities compared to their fellow eyes. To account for the 1.5 arcsec difference in acuity thresholds, a much higher density difference of about 1,500 cones/deg² would have been needed. Next to the spatial cell arrangement, that only partially predicted the achievable resolution acuity, ocular motion and its associated temporal modulations also highly influence visual resolution.

Ocular drift is an active sampling mechanism

As the eye drifts, a visual stimulus projected onto the retina is processed as a spatiotemporal luminance flow. The stimulus itself as well as the extent of drift motion determine the characteristics of modulation. In our experiments, analyzing the exact retinal locations sampling the stimulus, we revealed that participants kept coming back to the same few hundreds of cone photoreceptors (Fig. 3a). To focus on the characteristics and implications of drift eye motion, trials containing microsaccades during stimulus presentation were excluded from the analyses. During the short stimulus duration however, microsaccades rarely occurred, as participants tend to suppress their microsaccades, likely because they can be detrimental to fine-scale discrimination^42,43. Drift motion patterns varied greatly across, but also within participants. Examples of drift motion paths for the eyes that performed the smallest and largest drift motion, on average, show a great variability in shapes as well as extent of motion (Fig. 3b). In our analyses, we chose the sum of piecewise drift amplitude as the prime metric to describe the ocular drift motion, because the randomness underlying alternative metrics of drift eye movements becomes increasingly questionable (see also Discussion). Across all participants and experimental trials, drift amplitudes ranged between 2.5 and 17.2 arcmin, with a median amplitude of 6.5 arcmin (which corresponds to a velocity of 5 to 34.5 arcmin/sec, median: 13 arcmin/sec, Fig. 3c).

Fig. 3:

The drift amplitudes are slightly smaller than in previous non-AO studies, which is attributable to the viewing situation. The participants were looking at a very small imaging field within a completely dark periphery without distracting structures or stimuli. The smallest drift movement performed was similar among eyes (range: 2.5 – 5.4 arcmin), whereas the largest individual drifts differed more than three times as much (range: 7.7 – 17.2 arcmin). Therefore, the individual drift span was rather driven by the larger drift amplitudes of an eye and there was a strong correlation between median drift amplitude and drift range (dominant eyes: r² = 0.55, p = 0.002, non-dominant eyes: r² = 0.34, p = 0.02, Fig. 3d).

In fellow eyes, which were measured consecutively, drift amplitudes had a very strong correlation (r² = 0.86, p < 0.001, Fig. 3e) with no significant difference between eyes (paired t-test, p = 0.2). The median drift amplitudes of all eyes varied between 4.8 and 8.5 arcmin (mean ± SD: 6.6 ± 1.1 arcmin). Individual visual acuity thresholds were significantly correlated with drift amplitudes (dominant: r² = 0.25, p = 0.04; non-dominant: r² = 0.29, p = 0.03, Fig. 3g), with a trend towards better visual acuity for small ocular drift motion. On a photoreceptor resolved scale, this confirms recent findings which showed individual acuity thresholds to be correlated with the drift motion during a non-AO acuity task, closely related to the drift measured in a sustained fixation task ³².

Considering the previously shown correlation between visual acuity and sampling cone density, one could assume those two aspects to go along with an increase of ocular drift for lower cone densities, whereas higher densities potentially need less drift to translate the stimulus over the same number of cones. However, we don’t find the eye motion to be tuned in a way to always let the stimulus slip across a similar number of cones. There was no significant correlation between cone densities and drift amplitude (dominant: r² = 0.07, p = 0.3; non-dominant: r² = 0.06, p = 0.4). Other aspects are likely to also influence the trial-wise motion, e.g. drift motion might be particularly tuned for stimulus sizes close to the threshold and less crucial for larger gap sizes as suggested in a recent model of early visual signal processing³⁶.

Drift is adaptive and directed

Ocular drift has long been assumed to be a persistent jittery motion that follows random trajectories. Recent work showed that the amount of drift can vary and may be adapted to the task that has to be performed^26,32. We here investigated if beyond this, humans are able to actively tune their ocular drift direction to exploit their prime spatial retinal processing properties. We therefore registered the individual drift motion trajectories with the photoreceptor mosaic, tracked them from the retinal location where the stimulus turned on (onset) to where it turned off after 500 ms (offset), and related these trajectories to foveolar landmarks (Fig. 4a, b). Because of the individual retinal locations used for fixation before stimulus onset, we registered that, across all eyes, drift motion occurred towards all directions during stimulus inspection, and no general trend in drift eye movements towards a particular cardinal direction across participants occurred (Fig. 4c). Individual eyes, however, showed different drift behavior mostly directed towards one or two of the four quadrants. All four cardinal directions were represented. Participant P8_right, for example, drifted towards the nasal or superior fovea in 90 % of all trials. P14_right, on the other hand, drifted towards the temporal fovea in 75 % of all trials. When the frame of reference was rotated in each trial to register the motion from the onset location relative to the CDC, we found a clear directional bias in which the drift was likely to move the stimulus closer to the CDC. The drift directionality was evaluated by measuring the relative angle between drift onset to drift offset and drift onset to CDC. We observed a strong trend of drift directionality; 49 % of all drift episodes moved the stimulus towards the CDC ± 45° (Fig. 4d). Among eyes, the individual fractions ranged between 16 and 80 % of trials. Only two eyes drifted towards the CDC less frequently than given by chance (Fig. S2). We computed the directionality tuning as the ratio of relative drift towards the CDC ± 45° (purple quadrant in Fig. 4d) and the mean relative drift towards the 3 other quadrants. A ratio of 1 indicated the same relative frequency of drift towards all cardinal directions, whereas for a tuning ratio of 2 the retina moved the CDC towards the stimulus twice as often compared to each of the other 3 cardinal directions. The directionality tuning ratios ranged between 0.6 and 11.8 with a median value of 3.

Fig. 4:

Directionality tuning ratios had a significant effect on how much the resolution threshold exceeded the Nyquist limit. Participants with highly tuned drift reached larger differences between the Nyquist limit and their visual acuity threshold (dominant eyes: r² = 0.45, p = 0.01; non-dominant eyes: r² = 0.27, p = 0.04, Fig. 4e). Drift directionality was mostly similar between eyes, and if intra-ocular differences occurred, they were not related to ocular dominance. Also, we did not observe an effect of training on drift directionality: one of the two trained observers had a very strong drift directionality (7 and 11.8 in the dominant and non-dominant eye, respectively) while the other one exhibited a tuning ratio below average (2.1 and 2.3 in the dominant and non-dominant eye, respectively).

Next to the CDC, two other foveolar landmarks are often reported as anchor locations describing the center of the fovea. When relating the drift trajectories to the preferred retinal locus of fixation (PRL) or the location of peak cone density (PCD), we found a weaker approximation towards both. The retinae moved the stimulus towards the PRL or PCD location in 42 % or 35 % of all trials, respectively (Fig. 4f). Therefore, the observed directionality was strongest towards the CDC. In a considerable number of trials, the stimulus onset was further displaced from all of the 3 retinal locations and therefore a directed drift motion resulted in approximation towards CDC as well as PRL and PCD. Also, in some eyes 2 or all of these retinal locations lay very close together, which results in very similar effects. Nevertheless, in some eyes with particularly stable fixation that had at least a few arcmin distance between their PRL and CDC we repeatedly observed a stimulus onset close to PRL followed by a directional drift towards CDC with a resulting stimulus offset closer to the CDC (see also supplementary discussion). Across participants this also resulted in a significant reduction of the isoline contour area (ISOA) size between stimulus onset and offset (p = 0.02, Fig. 4b, Fig. S1 and Supplementary Movie 3). The median ISOA for stimulus onset locations was 92.5 arcmin² which was reduced to 68.2 arcmin² for stimulus offset locations. This decrease in size of the area of all retinal landing points supports the view of a certain retinal cone or very small area of a few arcmin² to be the target region of the drift eye motion in a resolution task.

When we looked at how much the individual drift trajectory decreased the distance from either location, the median distance convergence (onset/offset distance) towards CDC, PRL and PCD was about 12 %, 7 % and 3 %, respectively. While no participant had an average convergence of more than 30 % towards PRL or PCD, the maximum convergence ratio towards CDC was about 50 %. An adaptive drift behavior was also found in the relative drift amplitudes exhibited in each stimulus presentation. Although the individual drift amplitudes could vary substantially from trial to trial, we found that, across all participants and experimental trials, eyes exhibited significantly larger drift amplitudes when the stimulus onset location was further away from the CDC (ρ² = 0.06, p < 0.001, Fig. 4g). The onset distance was not correlated with drift directionality (Fig. 4g). Across all trials, the average sampling cone density increased between stimulus onset and offset for most of the participants. This sampling gain was computed as the ratio between the maximum sampling density during the trial and the sampling density at the stimulus onset location. The sampling gain was significantly correlated with the potential retinal sampling gain of individuals in dominant (r² = 0.35, p = 0.02) as well as non-dominant eyes (r² = 0.50, p = 0.002, Fig. 4h).

Observers exploited on average 30 % of their potential sampling gain in both fellow eyes. Interestingly, one observer combined all the previously described sampling features particularly strong in his dominant eye (P08_R). It had a steep cone density gradient, exhibited strong directional tuning towards the CDC and had large drift amplitudes for stimulus onsets far from the CDC. This eye was excluded from the sampling gain analysis because fixation behavior differed by more than 4 standard deviations from the group average.

Discussion

By using synchronous adaptive optics imaging and visual stimulation of the foveola, we find that the human visual system is capable of resolving spatial features smaller than a single photoreceptor diameter and uncover a fixational eye motor behavior that optimizes retinal sampling in accordance with the individual photoreceptor mosaic.

Spatial vision and, in particular, visual acuity is the most tested and used performance metric with a close relation to everyday vision. It provides the main behavioral outcome for clinical studies of vision. Measured in daily routine or clinical studies, best corrected visual acuity of young and healthy adults is usually between 20/20 and 20/12.5 (60 and 37.5 arcsec)^44,45. Even if lower order aberrations are corrected by e.g. glasses or contact lenses, higher order aberrations inherently blur the retinal image, depending on their magnitude⁴⁵. Adaptive optics induce a close-to-diffraction limited optical correction, where the optical improvement is significantly correlated with an increase in visual acuity thresholds¹⁷. By correcting aberrations with AOSLO, we measured Snellen-E thresholds that were up to half the size (between 20/10 and 20/6.9; 30 to 20.6 arcsec) compared to the natural viewing condition.

This is slightly lower than previously presented data⁴⁴, very likely because of different wavelengths used for experimentation (Fig. 2c). It might be surprising to learn that the neural machinery of human vision is able to resolve such tiny stimuli, because natural viewing is blurred by the eye’s optics. Even though observers are, to some degree, adapted to their own aberrations⁴⁶, best subjective image quality is seen when on average 88 % of the aberrations are corrected⁴⁷.

In how far are those resolution thresholds linked to or limited by the optimized but at the same time individual morphology of the human foveola? While in the periphery, midget retinal ganglion cell sampling dominates resolution, resolution of the foveal center was estimated to be governed by the cone sampling limit^8,48. By first-time direct experimental validation in the same participants, we here confirm the hypothesis that the individual spacing of cones can predict the resolution capacity of our foveola when optical influences are bypassed (Fig. 2). We found that the individual spatial arrangement of cones was highly correlated to the visual acuity of participants and explains up to 45 % of its variance (Fig. 2d). Eyes with higher foveolar sampling capacity reached lower thresholds than eyes with less densely packed cone photoreceptors. Moreover, all participants reached resolution thresholds that exceeded the Nyquist sampling limit when tested with near infrared, 788 nm light. Natural vision is comprised of multiwavelength stimuli, thus, using 788 nm in isolation is at the top end of our retinal sensitivity. In a first part of our study, participants also performed experiments with 840 nm light. Thresholds were rather approximating the Nyquist limit with this longer near-infrared wavelength (Fig. 2c). The L- and M-cone photopigment absorbance for 840 nm is about 1.4 log unit lower than for 788 nm⁴⁹. The decreased cone sensitivity combined with a larger Airy-Disk size of about 7 % are likely to be detrimental for the longer, 840nm, wavelength. We would expect a potential for even lower thresholds for shorter wavelengths.

Otherwise, a potential for lower thresholds is only expected in eyes with higher angular cone densities. Perhaps contrary at first sight, this could potentially be the case for observers with higher myopia. Most likely, the myopic eye growth lies between models of global expansion and an equatorial stretching, which results in increased cone densities on an angular scale and decreased cone densities on a linear scale for increased axial length¹³. The only two myopic participants in our population had mild myopia (spherical equivalent between −1 and - 2 D) and axial lengths between 24.5 and 25 mm. Those two myopes had rather average acuity thresholds, which is, in conjunction with the parallel finding of an overall below Nyquist resolution, explainable by their average angular cone densities. Therefore, we would expect acuity thresholds to be lower for myopic participants, in the case that (a) also the angular cone density is increased like previously suggested and (b) the AO correction and display resolution are still sufficient to completely resolve the foveolar cone mosaic. Psychophysical data for more participants with higher myopia and longer axial lengths would be needed to verify this assumption.

Theoretical predictions of the Nyquist resolution limit are implying stationary sampling. In reality, however, our eyes are never at rest, even when we attempt to maintain steady fixation. Fixational eye movements continuously modulate the luminance flow on individual cones and postreceptoral neuronal activity. Drift motion has long been presumed as a random jitter, a result of limited precision of the oculomotor system^50,51. More recent work revealed that drift motion is neither random nor detrimental due to the introduction of noise^27,28, but rather a fine tuned motion, beneficial for psychophysical measures of visual acuity in the parafovea³⁵ as well as foveola^26,36. Neurons in the visual system are strongly selective not just for spatial patterns, but also for temporally changing stimuli, a finding that is also supported by computational modeling, suggesting that the visual system may utilize principles comparable to those used in computational imaging for achieving super-resolution via camera motion⁵². Within the past decades, the interdisciplinary term “geometrical super-resolution” which is devoted to the filtering properties of sensor systems has become common⁵³. These resolution advantages may be achieved in the visual system by incorporating mechanisms that allow for the recognition of positional differences smaller than a single cell. That such mechanism exist is exemplified in a phenomenon known as hyperacuity. Fine localization discriminations of only a few seconds of arc are performed by identification of the centroid of the retinal light distributions⁵⁴ of the involved pattern components. In a diffraction limited resolution task, the visual system seems to be able to translate the temporal luminance modulation in individual photoreceptors by ocular drift to additional spatial information about the stimulus position and shape. Contrary, the indirect suppression of natural fixational eye motion by retinal stabilization techniques impairs visual acuity outside the foveolar center^26,29. For prolonged static stimulus presentations, retinal spiking decays over time, while drift motion keeps the luminance change active, continuously refreshes the receptive field input and sustains neuronal activity²⁴.

We found a significant correlation of drift motion and visual acuity thresholds between individuals, indicating that drift motion may be one of the key elements in reaching sub-cone resolution thresholds. Interestingly, acuity improved for smaller fixational drift and decreased in participants who exhibited larger drift motion, on average. The fact that less drift is beneficial to reach the lowest possible acuity thresholds reflects the characteristics of spatiotemporal luminance changes introduced by smaller or larger drift motion. Smaller drifts induce luminance changes with higher spatial frequencies and models of retinal ganglion cell activity suggest a higher contrast sensitivity for high spatial frequency motion and less for low spatial frequencies compared to a static retina^24,55. This is supported by other recent work which also showed that visual acuity thresholds can even be predicted from drift magnitudes measured in a sustained fixation task³².

There is evidence that fixational eye motion might have systematic components in primates. A previous study in macaque monkeys revealed a systematic directional drift response only a few dozens of milliseconds after various visual transients⁵⁶. In our study, we reveal that a certain drift directionality can not only be triggered by particular visual transients, but that human observers are capable to adapt their drift direction to enact an oculomotor strategy that takes advantage of the maximum resolution capacity provided within the retina. Our participants precisely moved their eye to have the stimulus slip across the most densely packed cone cells within their foveola. We hereby shed light on a mechanism that is potentially particularly active during fine discrimination tasks. Thus, drift is not the long-assumed random walk process between corrective saccades or microsaccades. And yet, the underlying mechanism to drift motion remains not fully understood. Recent work suggested, based on brainstem recordings in rhesus monkeys, that the origin can be found mostly upstream of the ocular motoneurons. It can likely be explained as diffusion in the oculomotor integrator which is mainly driven by noise, but additionally affected by mechanisms within the visual motor pathway (e.g. feedback mechanisms)³⁴. An incorporation of a visual feedback loop to that model was shown to modulate the statistics of eye motion, given a time lag of about 100 ms (mainly due to synaptic processing delays, of order 60-80 ms⁵⁶). This fits our results well. Our presentation time of 500 ms sufficed for a modulation of the fixational drift motion towards retinal areas of higher cone sampling (also see supplementary discussion of PRL displacement). This supports the view that the statistics of motion, but not the superdiffusive nature of fixational drift can be influenced by the visual task^26,34,56,57. The superior colliculus seems to play a major role in modulating drift motion in a feedback loop to visual inputs³¹. It’s not only involved in controlling large eye motions⁵⁸ and microsaccades⁵⁹, but also reflects neural responses to fixational drift that are likely a result of sensory input⁶⁰.

So, even though the CDC is displaced from the PRL in a way to be beneficial for natural binocular vision¹⁵, constant visual feedback allows to adapt the drift direction and therefore also the task related PRL. Commonly, the term PRL is used for describing the retinal location that is preferably used in fixational tasks. It is still a matter of debate what factors drive the development of this very reproducible¹⁵ retinal location and in how far it might provide enhanced visual function. Sensitivity to small light spots in the foveola seems to be rather plateau like and not particularly pronounced at the PRL⁶¹. As recently shown, the PRL slightly differs between different tasks but has a larger interindividual variability⁴³. The here shown results indicate that also when measuring visual resolution, the PRL is not necessarily the center of the sampling drift motion. The directional drift motion leads to a shift of the preferred retinal location for a resolution task towards the CDC (Fig. S3 and Supplementary Movie 3). Previous work that compared active versus passive fixation did not show a systematic offset in a similar experimental setup. However, 5 out of 8 participants also shifted their PRL in a Snellen E task closer to the CDC compared to the PRL for fixating a static disk stimulus⁴³, the conditions that are best comparable to our study. The main difference to our visual acuity experiments was that automatically paced random time intervals between presentations (0.5 – 1.5 sec) were applied to not allow the participants to anticipate the next trial whereas in our study participants self-paced the stimulus output to be able to prepare and focus for the next trial. It might be that this extremely fine-tuned usage of the visual feedback loop can only be kept active for rather short time intervals. By shifting the stimulus towards the CDC in 50 % of cases the potential sampling gain within individual eyes was exploited by 30%, on average, which goes along with a cone density increase of 3 % or 285 cones/deg². Even though this increase in cone density alone would not account for the difference between acuity thresholds and Nyquist limit, this and the simultaneous spatiotemporal luminance modulation contribute to achieving sub cone visual acuity thresholds.

Between fellow eyes we found very strong correlations for all the measured parameters. While drift amplitudes and directionality as well as cone densities are very symmetric between dominant and non-dominant eyes (Fig. 2e and 3e), significantly lower acuity thresholds of 1.5 arcsec, on average, were observed in the dominant eyes of participants (Fig. 2e). The dominant eyes visual input has a tendency to be preferred during binocular viewing, but has not been shown to exhibit relevant differences in visual function in healthy eyes with low refractive errors^62,63. Partially this may be due to limited accuracy in the mainly used clinical methods (e.g. Snellen Chart or projection have ∼ 10 arcsec steps between optotype rows). This very fine binocular difference between eyes emphasizes that some remaining factors which especially comprise the neural postprocessing steps, also play an important role and may facilitate the slight functional advantage of dominant eyes.

For clinical studies of retinal health and in new therapeutical approaches, photoreceptor health and visual acuity can be related to other more standard clinical measures as OCT-derived measures of outer segment length or retinal thickness which have been shown to serve for estimates of cone density⁶⁴. Therefore, building a larger dataset on photoreceptor resolved foveolar maps and associated visual function measures may help to, on the one hand, better understand the interplay between structural and functional changes to draw conclusions about disease progression, intervention efficiency or the interpretation of retinal imaging data in studies aimed at vision restoration. On the other hand, detailed examination of psychophysical measures with knowledge about the exact neural sampling characteristics offers a great potential to answer further questions about e.g. resolution limits in myopia, the effect of image stabilization in the very center of the foveola or implications for binocular viewing that could previously only be hypothesized. The awareness of the oculomotor system being able to finely adjust the drift motion behavior for a particular task may guide future interpretation of fixational eye motion.

Material and Methods

Participants

A total of 38 participants underwent a preliminary screening where ocular biometry, ophthalmologic status, fixational eye motion and adaptive optics correction as well as foveolar image quality were tested. From those, 20 participants with normal ophthalmologic status, resolvable foveolar cones and ocular anatomy that allowed for a 7 mm pupil aperture during experimentation were chosen for subsequent examination. All 6 male and 14 female observers (17 adults [age: 18 – 42], 3 children [age: 10, 12 and 14]) had no or only mild refractive errors (SE: ± 2.5 diopters). The children and 15 adults were naïve participants and two adults were experienced observers. More detailed cone topography and eye motion characteristics of the here studied population have been shown previously¹⁵. The experiments were conducted under two different light conditions (16 participants 788 nm, 12 participants 840 nm). Eight participants took part in both experimental conditions. We mainly report the data acquired for the 788 nm condition in this manuscript and show 840 nm data for comparison where noteworthy differences arise.

Written informed consent was obtained from each participant and all experimental procedures adhered to the tenets of the Declaration of Helsinki, in accordance with the guidelines of the independent ethics committee of the medical faculty at the Rheinische Friedrich-Wilhelms-Universität of Bonn.

Ocular dominance

Ocular dominance was determined by a Miles Test prior to pupil dilation and visual acuity testing. The experimenter stood in a distance of 6 m in front of the participant and asked them to form a small opening between thumbs and forefingers with both hands. The participant was then asked to extend their arms in front of them to look through the formed hole at the experimenter’s face with both eyes open. This procedure was conducted 3-5 times to determine the dominant (= uncovered) eye in a 3/3 or at least 3/5 condition.

AOSLO retinal imaging

In vivo images of the complete foveolar cone mosaic were recorded using a custom-built adaptive optics scanning laser ophthalmoscope (AOSLO). The general setup of the AOSLO has been described previously⁶⁵ and pertinent differences as well as the method of determination of the preferred retinal locus of fixation (PRL) have been described in a recent publication¹⁵.

In brief, the front-end of the Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) was equipped with three f = 500mm focal telescopes. These telescopes were specifically designed for point-scanning an adaptive optics-corrected focal light spot across the retina, ensuring diffraction-limited resolution in both incident and reflected beams. The system incorporated a magnetic actuator-driven deformable mirror (DM97-07, 7.2mm pupil diameter, ALPAO, Montbonnot-Saint-Martin, France) positioned in a retinal conjugate plane. The deformable mirror was controlled by the wavefront error signals from a 25×25 lenslet Shack Hartmann sensor (SHSCam AR-S-150-GE, Optocraft GmbH, Erlangen, Germany) in closed-loop. Imaging and wavefront correction utilized wavelengths of either 788 nm (±12 nm) or 840 nm (±12 nm) light, achieved through serial dichroic and bandpass filtering of a supercontinuum source (SuperK Extreme EXR-15, NKT Photonics, Birkerød, Denmark). The imaging field of view was 0.85 x 0.85 degrees of visual angle. The digital lateral resolution was about 0.1 arcmin, the size of one pixel in the recorded videos and images. Light reflected from the retina was detected by a photomultiplier tube (PMT, H7422-50, Hamamatsu Photonics, Hamamatsu, Japan), positioned behind a confocal pinhole (Pinhole diameter = 20 mm, equivalent to 0.47 (840nm) and 0.5 (788nm) Airy disk diameters).

Continuous sampling of the PMT signal was carried out using a field programmable gate array (FPGA), resulting in a 512 x 512-pixel video at 30 Hz (600 pixels per degree of visual angle). Through rapid acousto-optic intensity modulation of the imaging lights, the square AOSLO imaging field was used as retinal display, where each pixel could be individually controlled to produce the visual stimuli.

Cone map generation and computation of sampling characteristics

The best PRL videos acquired were selected to create spatially registered, high signal-to-noise ratio images of the foveal center, which served as master retinal images for cone labeling as well as referencing of stimulus motion trajectories. This study includes only participants for whom the master retinal image was of sufficient quality to label all cones across the image. Cone centers were identified and labeled semi manually, as previously described^15,66. Cone density was computed in two different ways. First, for deriving landmark metrics of the foveolar cone map, we then computed Voronoi tessellation, estimating a patch with certain area for each individual cone and summed the nearest 150 cone patches around each image pixel. The number of cells was divided by the resulting area to derive a pixel-resolved map of cone densities. Based on this map, the peak cone density (PCD) is defined as the highest cone density value of the map with it’s according retinal location. The cone density centroid (CDC) is computed as the weighted centroid of the 20^th percentile of highest cone densities within the map. We refer to the CDC as the anatomical center and the anchor for further spatial analyses in this study. The CDC has been shown to be a more robust and reproducible metric to describe the anatomical center than the more routinely reported peak cone density (PCD)^15,67.

Second, for analyzing the relation between individual sampling limits and resolution acuity, cone density was computed based on the cone cells contributing to the sampling process. To identify the cones interacting in stimulus sampling, a simple model of cone light capture was employed. Each cone was described by an associated light acceptance aperture with its diameter estimated as 48 % of the average spacing between the cone and all of its neighbors. The efficiency of the aperture along its diameter was approximated as Gaussian profiles. Also, a model of the stimulus retinal image was computed by convolving the eye’s point spread function (diffraction limited at 788 nm for a 7 mm pupil) with the stimulus bitmap. The complete two-dimensional model of cone apertures was then multiplied by models of the presented stimuli to arrive at the cone-level light distribution based on the different stimulus positions, sizes and orientations. The light distribution within each cone was integrated across the entire cone aperture. This value was then normalized to the degree to which the aperture was filled. Cone stimulation was considered to be maximal if the entire aperture was filled. Using this method, a cone activation pattern could be generated for each point in time (e.g. Fig. 1f). To arrive at a task-related cone density estimate for each frame (sampling cone density), the number of cones identified to interact with the stimulus was divided by their summed cone area. In the presented analyses, the median sampling density of all trials is analyzed and standard deviations are shown as grey lines (Fig. 2d, e). This stimulus related cone density was chosen to closely represent the sampling process; however, the results do not qualitatively differ from using the cone density map based on the 150 nearest cones.

We assumed a perfect hexagonal cell mosaic to estimate the average inter-cone-distance (ICD) between neighboring cells and to compute the theoretical Nyquist sampling limit, which is based on the spacing between rows of cones, and given by .

Experimental procedures

For psychophysical acuity testing, participants reported the orientation of a Snellen-E stimulus in a four-alternative forced-choice (4 AFC) task under unrestricted eye motion. Psychophysical experiments were performed monocularly in both eyes. The non-dominant eye was tested first and the dominant eye after a 15-30 minutes break. This protocol was chosen because in pilot experiments in 7 participants (which were performed with a random order) less time was needed and hence less fatigue was reported by the participants when the second eye was the dominant one. In these pilot experiments, the same qualitative difference of acuity thresholds between non-dominant and dominant were found.

Mydriasis and cycloplegia were established by two drops of 1% tropicamide, instilled into the eyelid about 25 and 20 minutes prior to experiments. If experimentation took longer than 40 minutes, another drop of tropicamide was instilled. A customized dental impression mold (bite bar) was used to immobilize and adjust the head position and thus to align the participants eye in front of the imaging system to ensure optimal adaptive optics correction and image quality. The participants were encouraged to take breaks at any time. We found that proper resting is one of the most crucial factors during the rather complex AOSLO experimentation. Frequent breaks ensure constant, high-level compliance and excellent image quality as the basis for artefact-free and reproducible results.

Before recording experimental runs, each participant performed 3 test runs to get used to the test procedure and the appearance of the stimuli. The stimuli were displayed as “off-stimuli” on the infrared background by switching the displayed intensity via an acousto-optic modulator ⁶⁸ (AOM, TEM-250-50-10-840-2FP, Brimrose, Sparks Glencoe, MD, USA)(Fig. 1a). Because of ocular diffraction, the stimulus contrast varied between 0.61 and 0.80 for an 18 arcsec versus 36 arcsec gap sized stimulus (3 and 6 pixel of the scanning raster, respectively). The visual acuity testing followed the Bayesian adaptive procedure QUEST^69–71. Stimulus progression was self-paced by the participant. The stimuli were presented for 500 ms to avoid limitations by insufficient temporal summation⁷². Around each trial, a one second AOSLO video was recorded, with the stimulation onset at around 300ms after video onset. Visual acuity thresholds were estimated by pooling results from 5 consecutively run staircases, with each containing 20 trials. A psychometric function was fitted using psignifit4⁷³ to derive threshold estimates for further analysis. The expected threshold variance is described and visualized by the 95 % confidence interval (Fig. 1d, e and 2d).

Video processing and eye motion analysis

The AOSLO used a raster scanning technique where each frame was acquired over time. The recorded videos were stabilized after psychophysical testing using custom settings within the MATLAB based stabilization software from Stevenson et al.⁷⁴. To acquire eye traces at higher temporal resolution than the 30 Hz frame rate, each frame of the AOSLO movie is broken into 32 horizontal strips of 16 pixels height and cross-correlated against a reference frame. The reference frame was generally chosen automatically and exchanged by a manually chosen frame in cases were stabilization failed despite good overall image quality. This method allowed extraction of eye motion traces at temporal frequencies up to 960 Hz.

The frame-wise (30 Hz) stimulus position was encoded as a white cross marker in each video. As single strip alignments can have small errors due to noise in the strip or retinal torsion (particularly affecting the horizontal motion estimate)⁷⁵, we compute the average offsets from the cross-containing strip and 2 previous/subsequent strips. These steps yielded more accurate trajectories in retinal coordinates for every trial. All individual trial AOSLO frames and the corresponding trajectories are then referenced to the single master retinal image used for cone map generation.

To quantify the retinal motion across the stimulus, drift amplitude was defined as the concatenated vector sum of all frame-wise motion vectors within the 500 ms stimulus duration (see also Fig. 3c). Trials that contained microsaccades or blinks during stimulus presentation were excluded from further analyses. Microsaccade occurrence varied highly between participants (mean ± SD: 14 ± 10 % of trials, range: 2 % - 41 %). If not stated differently, we here report the median drift amplitude of all trials for individual eyes (e.g. of all traces shown in Fig. 3a). To quantify drift direction, the angle between each trajectory’s starting coordinate (coordinate center in Fig. 4c) and end coordinate was computed. To check for potential motion bias, the drift angles were first analyzed in retinal coordinates (Fig. 4c), and then as the relative angle, θ_CDC, formed between the drift vector and the line connecting the retinal onset location and the CDC (Fig. 4d). To compare directionality towards other locations of interest, the same was done for PRL and PCD locations (Fig. 4f).

Statistical information

All statistical analyses were conducted using custom written MATLAB code and significance levels were set at 0.05. To assess the normal distribution of the dataset, a two-sided Shapiro–Wilk test was employed. This test is recognized to be appropriate for small sample sizes. The paired samples t-test was utilized to assess whether there were significant differences between the means of normally distributed paired observations. For non-parametric data, the Wilcoxon Signed-Rank test was employed. Linear correlations were computed to examine the relationships between variables. For variables demonstrating normal distribution, Pearson’s correlation coefficient was employed, while for non-normally distributed data, Spearman’s rank correlation coefficient was utilized. Pearson’s correlation is sensitive to linear relationships, assuming bivariate normality, whereas Spearman’s correlation is a non-parametric measure suitable for monotonic relationships and is robust against outliers and non-normal distributions.

Author contributions

J.L.W and W.M.H conceived the research idea and developed the data analysis pipeline. J.L.W performed the experiments and data analyses. J.L.W, V. L. and W.M.H discussed the results and wrote the manuscript.

Competing interests

The authors declare no competing interests.