Inferential eye movement control while following dynamic gaze

Eye movements are involved in almost every daily human activity, from searching for your apartment key, identifying a friend, reading, and preparing a sandwich. People make about three ballistic eye movements (saccades) per second orienting the central part of the vision (the fovea) to regions of interest in the world and acquiring high-acuity visual information (Bahill et al., 1975). A foveated visual system allocates more retinal cells (Curcio et al., 1987), thalamic, cortical (Azzopardi and Cowey, 1993), and superior colliculus (Chen et al., 2019) neurons to the central part of the visual field and allows for computational and metabolic savings. But it requires that eye movements are programmed intelligently to overcome the deficits of peripheral processing. People execute eye movements effortlessly, rapidly, and automatically giving the impression that these might seem fairly random. However, there are sophisticated computations in the brain that control eye movements involved in fine spatial judgments (Intoy and Rucci, 2020; Rucci et al., 2007), search (Araujo et al., 2001; Eckstein et al., 2015; Hoppe and Rothkopf, 2019; Najemnik and Geisler, 2005), object identification (Renninger et al., 2007), face recognition (Or et al., 2015; Peterson and Eckstein, 2012), reading (Legge et al., 1997; Legge et al., 2002), and motor actions (Ballard et al., 1995; Hayhoe and Ballard, 2005).

Following the gaze of others (gaze-following) with eye movements is critical to infer others’ intentions, current interests, and future actions (Capozzi and Ristic, 2018; Dalmaso et al., 2020; Emery, 2000; Kleinke, 1986; McKay et al., 2021). Gaze-following behavior can be observed as early as 8–10 months in infants and is widely found in nonhumans such as macaques, dogs, and goats (Brooks and Meltzoff, 2005; Kaminski et al., 2005; Senju and Csibra, 2008; Shepherd, 2010; Wallis et al., 2015). The ability to perceive others’ gaze direction accurately and plan eye movements is essential for infants to engage in social interactions to learn objects and languages (Carpenter et al., 1998; Morales et al., 1998; Morales et al., 2000; Woodward, 2003). People are extremely sensitive to others’ direction of gaze (Kleinke, 1986; Langton and Bruce, 1999). When people observe someone’s gaze, they orient covert attention and eye movements toward the gazed location, which improves the detection of a target appearing in the gaze direction (Driver et al., 1999; Egeth and Yantis, 1997; Friesen et al., 2004; Han et al., 2021b; Jonides, 1981; Kingstone et al., 2003; Mulckhuyse and Theeuwes, 2010). Impairments in gaze cueing have also been proposed as an important correlate of autism spectrum disorder (Baron-Cohen, 2001; Leekam et al., 1998; Nation and Penny, 2008) and important in child development (Brooks and Meltzoff, 2005).

The majority of studies investigating gaze-following (but see Gregory, 2022; Han et al., 2021b; Lachat et al., 2012; Macdonald and Tatler, 2013; Sun et al., 2017; Wang et al., 2014) use static images of the eyes or the face in isolation, which are far from the more ecological real-world behaviors of individuals moving their heads and eyes when orienting attention. That gaze cueing triggers eye movements is well known, but the dynamics of eye movements when observing gaze behaviors with naturalistic dynamic stimuli are not known. Studies have investigated how the brain integrates temporal information to program saccades and how it integrates foveal and peripheral information (Stewart et al., 2020; Wolf et al., 2022; Wolf and Schütz, 2015) but have relied on artificial or simplified stimuli.

Little is known about what features across the visual field influence eye movements during gaze-following, their temporal dynamics, and their functionality. How does the brain rely on the features of the gazer’s head and peripheral visual information about likely gaze goals to program eye movements? Do observers wait for the gazer’s head movement to end before initiating the first gaze-following saccades? Do visual properties of the gazer’s head influence the programming of eye movements? Answering these questions has been difficult because they require a well-controlled real-world data set, moment-to-moment characterization of the gazer’s features, and experimental manipulations that alter peripheral information while maintaining the gazer’s information unaltered.

Here, we created a collection of in-house videos of dynamic gaze behaviors in real-world settings by instructing actors to direct their gaze to specific people on the filming set (Figure 1a). We then asked experiment participants that watched the videos to follow the gaze shifts in the videos and decide whether a specific target person was present or absent (Figure 1a). The viewing angles subtended by the people in the videos corresponded to distances for which the eyes provided little information. Thus, our work focuses on the gazer’s head direction.

Figure 1 with 1 supplement see all

Download asset Open asset

Our first goal is to assess the impact of peripheral gaze-goal information on the saccade error and timing. Second, we aimed to elucidate how the brain temporally processes visual information to influence saccade programming during gaze-following. To extract features of the videos that we hypothesized would influence saccade planning we used a state-of-the-art artificial intelligence (AI) model (Chong et al., 2020) to make moment-to-moment estimates of the gazer’s head direction in the videos. We assessed how observers’ saccade direction, timing, and errors related to the extracted features to gain insight into the brain computations during saccade planning.

We investigated eye movement control while following the gaze of others. Although human eye movements are fast and might seem idiosyncratic, our findings show that the human brain uses moment-to-moment information about the gazer’s head dynamics and peripheral information about likely gaze goals to rationally plan the timing and endpoint of saccadic eye movements.

First, we found that the oculomotor system integrates information about the foveally presented gazer’s head and peripheral information about potential gaze goals. When a person was present at the gaze goal, observers executed faster and more accurate saccades. One might have expected the presence of gaze goals to reduce the saccade endpoint Euclidean error but to a lesser degree reduce saccade angular error. Arguably, if observers waited for the gazer’s head to stop, they could rely on the head direction to accurately plan their gaze-following saccade direction without needing to rely on a peripheral gaze goal. Our results showed otherwise. Deleting the gaze goal also increased the saccade angular error. This suggests that the eye movement direction also heavily relies on the peripheral processing of likely gaze goals. This might seem counterintuitive but if one considers that the saccades are programmed before the gazer’s head comes to a stop and points to the gaze goal, then the gazer’s head direction might not provide sufficient information for observers to program saccades to the correct direction.

The evidence for the integration of foveal and peripheral information is consistent with a series of studies showing observers’ ability to simultaneously process foveal and peripheral information for simpler dual tasks with simple stimuli (Ludwig et al., 2014; Stewart et al., 2020) and their joint influence on fixation duration during scene viewing (Laubrock et al., 2013) and subsequent eye movements (Wolf et al., 2022).

Importantly, we found the first saccades are anticipatorily initiated before the gazer’s head movement comes to a stop. And they contain information about the direction of the gaze goal that is more accurate than the direction information provided by the gazer’s head at the time of saccade initiation. This suggests that the brain is using peripheral information to make an active prediction about likely gaze goals. We found that on average the first saccades were anticipating the head direction by 340–370ms and 160–220ms for target present and absent conditions. Furthermore, previous studies have shown that a saccade is typically based on visual information presented ~100ms before saccade execution (Becker and Jürgens, 1979; Caspi et al., 2004; Hooge and Erkelens, 1999; Ludwig et al., 2005). Thus, the first saccade might only have access to the gazer’s head direction up to ~100ms before saccade execution, which means the observes’ first saccades might be anticipating the gazer’s head direction by 440–470ms and 260–320ms. Similarly, the gazer’s head direction at the time of saccade programming (rather than execution) was, on average, pointing 44.1 degrees (for target/distractor present trials) and 39.7 degrees (for target/distractor absent trials) away from the gaze goal.

The evidence for an inferential process that influences saccade programming when a person is present as the gaze goal in the scene might be expected. But, the inferential process still prevailed when a target/distractor was absent. It is likely that even when no person is present at the gaze goal location, the brain uses information about the scene including the ground, the objects, and the sky to make estimates of likely gaze goals. Prior knowledge about the maximum angular rotation of the gazer’s head also constrains the possible gazer goals. This information is used by observers to program anticipatory inferential eye movements even in the absence of an unambiguous visible gazer goal (e.g., a person) in the observer’s visual periphery.

Second, we found that early first saccades are executed when the gazer’s head velocity diminishes to values comparable to the velocity that is typical during the 200ms time interval before the head stops. This is consistent with the idea that observers use the gazer’s head velocity to dynamically make inferences about the likelihood that the head will stop and then execute a saccade towards the likely gaze goals. However, our data also suggest that other cues are used to infer that the gazer’s head will stop. For example, for some trials with longer first saccade latencies (no reverse saccade trials), the head velocity before the observers’ saccade execution is almost double the typical head velocities during the 200ms time interval before a head stops (Figure 4e). Thus, the observers must rely on other cues. In these long latency trials (Figure 4e) there is a reduction of the head velocity in the 200ms before the saccade execution suggesting that observers use the head’s deceleration to infer that the gazer’s head will come to a stop and then execute the first gaze-following saccade. It is also likely that for trials with a gaze goal, observers use an estimated error between the implied gaze direction and the gaze goal to plan saccades. Small estimated angular errors might be used to trigger saccades. Thus, we suggest that the observers’ oculomotor system might use multiple cues (head velocity, head deceleration, estimated gaze errors, etc) to infer that the gazer’s head might stop and trigger gaze-following saccades.

Third, surprisingly, we found that observers often executed reverse saccades in a significant proportion of trials (>20%). These reverse saccades are directed to re-fixate the gazer’s head and can be referred to as return saccades. The reverse saccades were not an artifact of our instruction to the observers to follow the gaze of the person in the video. A follow-up experiment where observers were instructed to decide whether a target person was present with no instruction to participants about their eye movements also resulted in a comparable proportion of reverse saccades. Why might observers make such saccades? Our analysis showed that these reverse saccades do not appear randomly across trials. Reverse saccades occur on trials in which the gazer’s head velocity is slow but starts accelerating about 200ms before the first saccade is executed and observers infer that the gazer’s head will not come to a halt. Why don’t observers simply cancel the forward saccade? Studies have shown that there is a 50–100ms delay between the programming of a saccade and its execution (Becker and Jürgens, 1979; Caspi et al., 2004; Ludwig et al., 2005). The gazer’s head acceleration occurring immediately before the execution of the forward saccade is not used to cancel the impending planned eye movement.

Our findings with the experiment that freezes the gazer after the first forward saccade suggest that the reverse saccade is programmed before the execution of the first forward saccade. This concurrent programming of saccades has been documented for simplified lab experiments (Becker and Jürgens, 1979; Caspi et al., 2004) but not in the context of real-world stimuli and tasks. One alternative explanation we did not explore is that reverse saccades are simply triggered after the first forward saccades that do not land on the target/distractor. In this perspective, a forward saccade is executed and when foveal processing determines that the saccade endpoint was far from a likely gaze goal then a reverse saccade is programmed and executed (regardless of the velocity of the gazer’s head). Data analysis does not favor this interpretation. In a small percentage of trials (15% of reverse saccade trials) first saccades landed within 0.5° visual angle of the target but these were still followed by reverse saccades. This observation suggests that foveating close to the gaze goal was not sufficient to interrupt a reverse saccade programmed before the execution of the first forward saccade.

Our results also show that the reverse saccades had functional importance. Forward saccades, after re-fixating the gazer with a reverse saccade, were more accurate at landing close to the gaze goal. The benefit of re-fixating the gazer was more reliable when there was a person present at the gaze goal. When a gaze goal person was absent we found a less reliable re-fixation benefit (not statistically significant in experiment 1 and marginally significant in experiment 2) suggesting that not having a peripheral likely gaze goal can be a bottleneck to the accuracy of saccade endpoints.

The existence of reverse saccades to re-fixate the gazer’s head might seem puzzling. Why does the oculomotor programming system not wait longer until the gazer’s head comes to a full stop, then executes the gaze-following saccade and avoids programming reverse saccades altogether? Executing anticipatory eye movements that predict future grasping actions (Mennie et al., 2007; Pelz and Canosa, 2001), the location or motion of a stimulus (Fooken and Spering, 2020; Kowler, 1989; Kowler, 2011; Kowler et al., 2019) is common for the oculomotor system. Thus, while following the gaze of others the observers’ oculomotor system plans anticipatory saccades that predict the gaze goal before the completion of the gazer’s head movements. Occasionally, these predictive saccades are premature and the brain rapidly programs a reverse saccade to re-fixate the gazer and collect further information about the potential gazer goal.

Are the reverse saccades unique to gaze following? No, humans make reverse saccades in other visual tasks that require maintaining information in working memory, such as copying a color block pattern across two locations (Hayhoe and Ballard, 2005; Hayhoe, 2017; Meghanathan et al., 2019). Most notably during reading humans make frequent reverse saccades (“called regressive”). Although one might draw a parallel between reading and gaze-following, our findings highlight important distinctions. Regressive saccades during reading are related to inaccurate eye movements that missed critical words or fixations that are too short to deeply process a word’s meaning (Inhoff et al., 2019; Rayner, 1998). The reverse saccades while following dynamic gaze are related to moment-to-moment changes in the visual information in the world (i.e. the gazer’s head velocity) and the oculomotor systems’ rapid response to optimize gaze-following.

A remaining question is whether the gaze-following behaviors documented in the current study reflect real-world eye movement strategies. Do the reported inferential anticipatory and reverse saccades generalize to the real-word or are they a consequence of the timing of our study or the particular tasks in the study? The videos were presented for 1.2 s which should not represent an excessive time pressure to observers when compared to other eye movement studies on faces (Peterson and Eckstein, 2012) or search (Ackermann and Landy, 2013; Eckstein et al., 2015). We evaluated a person search task with explicit instructions to follow the gazer and a second task with no eye movement instructions and observed reverse and anticipatory saccades for both tasks. There is also evidence that for natural tasks such as looking at faces or searching, many of the findings with images in the laboratory do generalize to real word settings (Mack and Eckstein, 2011; Peterson et al., 2016). Still, the generalization to the real world for gaze-following eye movement behaviors needs to be assessed with mobile eye trackers (Land and Hayhoe, 2001).

What might be the brain areas involved in the oculomotor programs for gaze following? There is a large literature relating gaze position to neuronal response properties in the superior temporal sulcus (Oram and Perrett, 1994) and dorsal prefrontal cortex (Lanzilotto et al., 2017). These areas relay information to the attention and gaze network in the parietal and frontal cortex which are responsible for covert attention and eye movements (Pierrot-Deseilligny et al., 2004). Finally, the concurrent programming of saccades has been related to neurons in the Frontal Eye Fields (FEF, Basu and Murthy, 2020). Identifying brain areas that integrate peripheral information to generate predictions of likely gaze goals is an important future goal of research.

There are various limitations of the study. In our study, the target/distractor person always appeared at the gaze goal. Thus, the peripheral presence of a person provided some certainty to observers that such a person would be the gazer’s goal. If we had shown videos in which gazers looked at one of two people or other objects even when a person was present, the larger uncertainty about the gazer goals might delay the execution of observer saccades. Thus, cognitive expectations which have been shown to play an important role in oculomotor control (Kowler, 1989) will influence our findings. Our studies used people as gaze goals which are quite visible in the visual periphery. Gaze goals less visible in the observers' periphery will influence the accuracy and timing of the saccades (van Beers, 2007; Han et al., 2021a). In addition, observers might use prior expectations of what types of targets are often looked at (people vs. the floor) to bias their inferences. There is also literature comparing how the human attention system varies when the gaze goals are objects vs. social entities (Mares et al., 2016). Thus, the social nature of the gaze goal, as well as social variables about the gazer, might also influence the oculomotor dynamics investigated (Dalmaso et al., 2020; Macdonald and Tatler, 2013).

Finally, our study focused on the head movement while a large literature focuses on the influences of the gazer’s eyes (Driver et al., 1999; Friesen et al., 2004; Langton et al., 2000; Mansfield et al., 2003; Ristic et al., 2002). Our study was relevant to gazers situated at a distance from the observers. The mean angle subtended by the heads in our videos (1.47°, std = 0.32°) would match the angle subtended by a real-sized head viewed at a distance of 9.3 m (std = 2.0 m) in real life. At that distance, the eye subtends a mean angle of 0.147° (vertically) providing a poor source of information to infer the gaze goals compared to the head orientation. Future studies should investigate gazers at smaller distances from the observer and assess how dynamic gazer eye and head movements are integrated and their interactions (for static images see Balsdon and Clifford, 2018; Cline, 1967; Langton, 2000; Langton et al., 2004; Otsuka et al., 2014). Similarly, we did not analyze lower body movements. Recent studies have shown the diminished influence of the lower body on the orienting of attention (Han et al., 2021b; Pi et al., 2020).

To conclude, our findings reveal the fine-grained dynamics of eye movements while following gaze and the inferential processes the brain uses to predict gaze goals and rapidly program saccades that anticipate the information provided by the gazer’s head direction. Given that attending to the gaze of others is an integral part of a normal functioning social attention system, our findings might provide new granular analyses of eye movement control to assess groups with social attention deficits for which simpler gaze-following analyses have shown disparate results (Chawarska et al., 2003; Nation and Penny, 2008; Ristic et al., 2005).

All data generated or analyzed during this study are deposited at https://osf.io/g9bzt/.

1. 1. Han NX

   (2022) Open Science Framework

   ID g9bzt. Eye Movement Planning During Dynamic Gaze Following.

   https://osf.io/g9bzt/

1. 1. Ackermann JF
   2. Landy MS

   (2013) Choice of saccade endpoint under risk

   Journal of Vision 13:27.

   https://doi.org/10.1167/13.3.27

   • PubMed
   • Google Scholar
2. 1. Araujo C
   2. Kowler E
   3. Pavel M

   (2001) Eye movements during visual search: the costs of choosing the optimal path

   Vision Research 41:3613–3625.

   https://doi.org/10.1016/s0042-6989(01)00196-1

   • PubMed
   • Google Scholar
3. 1. Azzopardi P
   2. Cowey A

   (1993) Preferential representation of the fovea in the primary visual cortex

   Nature 361:719–721.

   https://doi.org/10.1038/361719a0

   • PubMed
   • Google Scholar
4. 1. Bahill AT
   2. Clark MR
   3. Stark L

   (1975) The main sequence, a tool for studying human eye movements

   Mathematical Biosciences 24:191–204.

   https://doi.org/10.1016/0025-5564(75)90075-9

   • Google Scholar
5. 1. Ballard DH
   2. Hayhoe MM
   3. Pelz JB

   (1995) Memory representations in natural tasks

   Journal of Cognitive Neuroscience 7:66–80.

   https://doi.org/10.1162/jocn.1995.7.1.66

   • PubMed
   • Google Scholar
6. 1. Balsdon T
   2. Clifford CWG

   (2018) Task dependent effects of head orientation on perceived gaze direction

   Frontiers in Psychology 9:2491.

   https://doi.org/10.3389/fpsyg.2018.02491

   • PubMed
   • Google Scholar
7. Book

   1. Baron-Cohen S

   (2001) Mindblindness: An Essay on Autism and Theory of Mind

   MIT Press.

   https://doi.org/10.7551/mitpress/4635.001.0001

   • Google Scholar
8. 1. Basu D
   2. Murthy A

   (2020) Parallel programming of saccades in the macaque frontal eye field: are sequential motor plans coactivated?

   Journal of Neurophysiology 123:107–119.

   https://doi.org/10.1152/jn.00545.2018

   • PubMed
   • Google Scholar
9. 1. Becker W
   2. Jürgens R

   (1979) An analysis of the saccadic system by means of double step stimuli

   Vision Research 19:967–983.

   https://doi.org/10.1016/0042-6989(79)90222-0

   • PubMed
   • Google Scholar
10. 1. Bekerman I
    2. Gottlieb P
    3. Vaiman M

    (2014) Variations in eyeball diameters of the healthy adults

    Journal of Ophthalmology 2014:503645.

    https://doi.org/10.1155/2014/503645

    • PubMed
    • Google Scholar
11. 1. Brooks R
    2. Meltzoff AN

    (2005) The development of gaze following and its relation to language

    Developmental Science 8:535–543.

    https://doi.org/10.1111/j.1467-7687.2005.00445.x

    • PubMed
    • Google Scholar
12. 1. Capozzi F
    2. Ristic J

    (2018) How attention gates social interactions

    Annals of the New York Academy of Sciences.

    https://doi.org/10.1111/nyas.13854

    • PubMed
    • Google Scholar
13. 1. Carpenter M
    2. Nagell K
    3. Tomasello M

    (1998)

    Social cognition, joint attention, and communicative competence from 9 to 15 months of age

    Monographs of the Society for Research in Child Development 63:1–143.

    • PubMed
    • Google Scholar
14. 1. Caspi A
    2. Beutter BR
    3. Eckstein MP

    (2004) The time course of visual information accrual guiding eye movement decisions

    PNAS 101:13086–13090.

    https://doi.org/10.1073/pnas.0305329101

    • PubMed
    • Google Scholar
15. 1. Chawarska K
    2. Klin A
    3. Volkmar F

    (2003) Automatic attention cueing through eye movement in 2-year-old children with autism

    Child Development 74:1108–1122.

    https://doi.org/10.1111/1467-8624.00595

    • PubMed
    • Google Scholar
16. 1. Chen CY
    2. Hoffmann KP
    3. Distler C
    4. Hafed ZM

    (2019) The foveal visual representation of the primate superior colliculus

    Current Biology 29:2109–2119.

    https://doi.org/10.1016/j.cub.2019.05.040

    • PubMed
    • Google Scholar
17. Conference

    1. Chong E
    2. Wang Y
    3. Ruiz N
    4. Rehg JM

    (2020) Detecting Attended Visual Targets in Video

    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5395–5405.

    https://doi.org/10.1109/CVPR42600.2020.00544

    • Google Scholar
18. 1. Cline MG

    (1967)

    The perception of where a person is looking

    The American Journal of Psychology 80:41–50.

    • PubMed
    • Google Scholar
19. 1. Curcio CA
    2. Sloan KR
    3. Packer O
    4. Hendrickson AE
    5. Kalina RE

    (1987) Distribution of cones in human and monkey retina: individual variability and radial asymmetry

    Science 236:579–582.

    https://doi.org/10.1126/science.3576186

    • PubMed
    • Google Scholar
20. 1. Dalmaso M
    2. Castelli L
    3. Galfano G

    (2020) Social modulators of gaze-mediated orienting of attention: A review

    Psychonomic Bulletin & Review 27:833–855.

    https://doi.org/10.3758/s13423-020-01730-x

    • PubMed
    • Google Scholar
21. 1. Driver J
    2. Davis G
    3. Ricciardelli P
    4. Kidd P
    5. Maxwell E
    6. Baron-Cohen S

    (1999) Gaze perception triggers reflexive visuospatial orienting

    Visual Cognition 6:509–540.

    https://doi.org/10.1080/135062899394920

    • Google Scholar
22. 1. Eckstein MP
    2. Schoonveld W
    3. Zhang S
    4. Mack SC
    5. Akbas E

    (2015) Optimal and human eye movements to clustered low value cues to increase decision rewards during search

    Vision Research 113:137–154.

    https://doi.org/10.1016/j.visres.2015.05.016

    • PubMed
    • Google Scholar
23. 1. Egeth HE
    2. Yantis S

    (1997) Visual attention: control, representation, and time course

    Annual Review of Psychology 48:269–297.

    https://doi.org/10.1146/annurev.psych.48.1.269

    • PubMed
    • Google Scholar
24. 1. Emery NJ

    (2000) The eyes have it: the neuroethology, function and evolution of social gaze

    Neuroscience and Biobehavioral Reviews 24:581–604.

    https://doi.org/10.1016/s0149-7634(00)00025-7

    • PubMed
    • Google Scholar
25. 1. Fooken J
    2. Spering M

    (2020) Eye movements as a readout of sensorimotor decision processes

    Journal of Neurophysiology 123:1439–1447.

    https://doi.org/10.1152/jn.00622.2019

    • PubMed
    • Google Scholar
26. 1. Friesen CK
    2. Ristic J
    3. Kingstone A

    (2004) Attentional effects of counterpredictive gaze and arrow cues

    Journal of Experimental Psychology. Human Perception and Performance 30:319–329.

    https://doi.org/10.1037/0096-1523.30.2.319

    • PubMed
    • Google Scholar
27. 1. Gramfort A
    2. Luessi M
    3. Larson E
    4. Engemann DA
    5. Strohmeier D
    6. Brodbeck C
    7. Goj R
    8. Jas M
    9. Brooks T
    10. Parkkonen L
    11. Hämäläinen M

    (2013) MEG and EEG data analysis with MNE-Python

    Frontiers in Neuroscience 7:267.

    https://doi.org/10.3389/fnins.2013.00267

    • PubMed
    • Google Scholar
28. 1. Gregory SEA

    (2022) Investigating facilitatory versus inhibitory effects of dynamic social and non-social cues on attention in a realistic space

    Psychological Research 86:1578–1590.

    https://doi.org/10.1007/s00426-021-01574-7

    • PubMed
    • Google Scholar
29. 1. Han NX
    2. Chakravarthula PN
    3. Eckstein MP

    (2021a) Peripheral facial features guiding eye movements and reducing fixational variability

    Journal of Vision 21:7.

    https://doi.org/10.1167/jov.21.8.7

    • PubMed
    • Google Scholar
30. Preprint

    1. Han NX
    2. Wang WY
    3. Eckstein MP

    (2021b) Gaze Perception in Humans and CNN-Based Model

    arXiv.

    https://arxiv.org/abs/2104.08447

    • Google Scholar
31. 1. Han NX
    2. Eckstein MP

    (2022) Gaze-cued shifts of attention and microsaccades are sustained for whole bodies but are transient for body parts

    Psychonomic Bulletin & Review 29:1854–1878.

    https://doi.org/10.3758/s13423-022-02087-z

    • PubMed
    • Google Scholar
32. 1. Hayhoe M
    2. Ballard D

    (2005) Eye movements in natural behavior

    Trends in Cognitive Sciences 9:188–194.

    https://doi.org/10.1016/j.tics.2005.02.009

    • PubMed
    • Google Scholar
33. 1. Hayhoe MM

    (2017) Vision and Action

    Annual Review of Vision Science 3:389–413.

    https://doi.org/10.1146/annurev-vision-102016-061437

    • PubMed
    • Google Scholar
34. 1. Hooge IT
    2. Erkelens CJ

    (1999) Peripheral vision and oculomotor control during visual search

    Vision Research 39:1567–1575.

    https://doi.org/10.1016/s0042-6989(98)00213-2

    • PubMed
    • Google Scholar
35. 1. Hoppe D
    2. Rothkopf CA

    (2019) Multi-step planning of eye movements in visual search

    Scientific Reports 9:144.

    https://doi.org/10.1038/s41598-018-37536-0

    • PubMed
    • Google Scholar
36. 1. Inhoff AW
    2. Kim A
    3. Radach R

    (2019) Regressions during Reading

    Vision 3:35.

    https://doi.org/10.3390/vision3030035

    • PubMed
    • Google Scholar
37. 1. Intoy J
    2. Rucci M

    (2020) Finely tuned eye movements enhance visual acuity

    Nature Communications 11:795.

    https://doi.org/10.1038/s41467-020-14616-2

    • PubMed
    • Google Scholar
38. 1. Jonides J

    (1981)

    Voluntary versus automatic control over the mind’s eye’s movement

    Attention and Performance.

    • Google Scholar
39. 1. Kaminski J
    2. Riedel J
    3. Call J
    4. Tomasello M

    (2005) Domestic goats, Capra hircus, follow gaze direction and use social cues in an object choice task

    Animal Behaviour 69:11–18.

    https://doi.org/10.1016/j.anbehav.2004.05.008

    • Google Scholar
40. 1. Kingstone A
    2. Smilek D
    3. Ristic J
    4. Kelland Friesen C
    5. Eastwood JD

    (2003) Attention, researchers! It is time to take a look at the real world

    Current Directions in Psychological Science 12:176–180.

    https://doi.org/10.1111/1467-8721.01255

    • Google Scholar
41. 1. Kleinke CL

    (1986)

    Gaze and eye contact: a research review

    Psychological Bulletin 100:78–100.

    • PubMed
    • Google Scholar
42. 1. Kowler E

    (1989) Cognitive expectations, not habits, control anticipatory smooth oculomotor pursuit

    Vision Research 29:1049–1057.

    https://doi.org/10.1016/0042-6989(89)90052-7

    • PubMed
    • Google Scholar
43. 1. Kowler E

    (2011) Eye movements: the past 25 years

    Vision Research 51:1457–1483.

    https://doi.org/10.1016/j.visres.2010.12.014

    • PubMed
    • Google Scholar
44. 1. Kowler E
    2. Rubinstein JF
    3. Santos EM
    4. Wang J

    (2019) Predictive smooth pursuit eye movements

    Annual Review of Vision Science 5:223–246.

    https://doi.org/10.1146/annurev-vision-091718-014901

    • PubMed
    • Google Scholar
45. 1. Lachat F
    2. Conty L
    3. Hugueville L
    4. George N

    (2012) Gaze cueing effect in a face-to-face situation

    Journal of Nonverbal Behavior 36:177–190.

    https://doi.org/10.1007/s10919-012-0133-x

    • Google Scholar
46. 1. Land MF
    2. Hayhoe M

    (2001) In what ways do eye movements contribute to everyday activities?

    Vision Research 41:3559–3565.

    https://doi.org/10.1016/s0042-6989(01)00102-x

    • PubMed
    • Google Scholar
47. 1. Langton SRH
    2. Bruce V

    (1999) Reflexive visual orienting in response to the social attention of others

    Visual Cognition 6:541–567.

    https://doi.org/10.1080/135062899394939

    • Google Scholar
48. 1. Langton SRH

    (2000) The mutual influence of gaze and head orientation in the analysis of social attention direction

    The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology 53:825–845.

    https://doi.org/10.1080/713755908

    • PubMed
    • Google Scholar
49. 1. Langton SR
    2. Watt RJ
    3. Bruce I

    (2000) Do the eyes have it? Cues to the direction of social attention

    Trends in Cognitive Sciences 4:50–59.

    https://doi.org/10.1016/s1364-6613(99)01436-9

    • PubMed
    • Google Scholar
50. 1. Langton SRH
    2. Honeyman H
    3. Tessler E

    (2004) The influence of head contour and nose angle on the perception of eye-gaze direction

    Perception & Psychophysics 66:752–771.

    https://doi.org/10.3758/bf03194970

    • PubMed
    • Google Scholar
51. 1. Lanzilotto M
    2. Gerbella M
    3. Perciavalle V
    4. Lucchetti C

    (2017) Neuronal encoding of self and others’ head rotation in the macaque dorsal prefrontal cortex

    Scientific Reports 7:8571.

    https://doi.org/10.1038/s41598-017-08936-5

    • PubMed
    • Google Scholar
52. 1. Laubrock J
    2. Cajar A
    3. Engbert R

    (2013) Control of fixation duration during scene viewing by interaction of foveal and peripheral processing

    Journal of Vision 13:11.

    https://doi.org/10.1167/13.12.11

    • PubMed
    • Google Scholar
53. 1. Lee JH
    2. Hwang Shin SJ
    3. Istook CL

    (2006)

    Analysis of human head shapes in the united states

    International Journal of Human Ecology 7:77–83.

    • Google Scholar
54. 1. Leekam SR
    2. Hunnisett E
    3. Moore C

    (1998)

    Targets and cues: gaze-following in children with autism

    Journal of Child Psychology and Psychiatry, and Allied Disciplines 39:951–962.

    • PubMed
    • Google Scholar
55. 1. Legge GE
    2. Klitz TS
    3. Tjan BS

    (1997) Mr. Chips: an ideal-observer model of reading

    Psychological Review 104:524–553.

    https://doi.org/10.1037/0033-295x.104.3.524

    • PubMed
    • Google Scholar
56. 1. Legge GE
    2. Hooven TA
    3. Klitz TS
    4. Stephen Mansfield JS
    5. Tjan BS

    (2002) Mr. Chips 2002: new insights from an ideal-observer model of reading

    Vision Research 42:2219–2234.

    https://doi.org/10.1016/s0042-6989(02)00131-1

    • PubMed
    • Google Scholar
57. 1. Ludwig CJH
    2. Gilchrist ID
    3. McSorley E
    4. Baddeley RJ

    (2005) The temporal impulse response underlying saccadic decisions

    The Journal of Neuroscience 25:9907–9912.

    https://doi.org/10.1523/JNEUROSCI.2197-05.2005

    • PubMed
    • Google Scholar
58. 1. Ludwig CJH
    2. Davies JR
    3. Eckstein MP

    (2014) Foveal analysis and peripheral selection during active visual sampling

    PNAS 111:E291–E299.

    https://doi.org/10.1073/pnas.1313553111

    • PubMed
    • Google Scholar
59. 1. Macdonald RG
    2. Tatler BW

    (2013) Do as eye say: gaze cueing and language in a real-world social interaction

    Journal of Vision 13:6.

    https://doi.org/10.1167/13.4.6

    • PubMed
    • Google Scholar
60. 1. Mack SC
    2. Eckstein MP

    (2011) Object co-occurrence serves as a contextual cue to guide and facilitate visual search in a natural viewing environment

    Journal of Vision 11:1–16.

    https://doi.org/10.1167/11.9.9

    • PubMed
    • Google Scholar
61. 1. Mansfield E
    2. Farroni T
    3. Johnson M

    (2003) Does gaze perception facilitate overt orienting?

    Visual Cognition 10:7–14.

    https://doi.org/10.1080/713756671

    • Google Scholar
62. 1. Mares I
    2. Smith ML
    3. Johnson MH
    4. Senju A

    (2016) Direct gaze facilitates rapid orienting to faces: Evidence from express saccades and saccadic potentials

    Biological Psychology 121:84–90.

    https://doi.org/10.1016/j.biopsycho.2016.10.003

    • PubMed
    • Google Scholar
63. 1. McKay KT
    2. Grainger SA
    3. Coundouris SP
    4. Skorich DP
    5. Phillips LH
    6. Henry JD

    (2021) Visual attentional orienting by eye gaze: A meta-analytic review of the gaze-cueing effect

    Psychological Bulletin 147:1269–1289.

    https://doi.org/10.1037/bul0000353

    • PubMed
    • Google Scholar
64. 1. Meghanathan RN
    2. Nikolaev AR
    3. van Leeuwen C

    (2019) Refixation patterns reveal memory-encoding strategies in free viewing

    Attention, Perception & Psychophysics 81:2499–2516.

    https://doi.org/10.3758/s13414-019-01735-2

    • PubMed
    • Google Scholar
65. 1. Mennie N
    2. Hayhoe M
    3. Sullivan B

    (2007) Look-ahead fixations: anticipatory eye movements in natural tasks

    Experimental Brain Research 179:427–442.

    https://doi.org/10.1007/s00221-006-0804-0

    • PubMed
    • Google Scholar
66. 1. Morales M
    2. Mundy P
    3. Rojas J

    (1998) Following the direction of gaze and language development in 6-month-olds

    Infant Behavior and Development 21:373–377.

    https://doi.org/10.1016/S0163-6383(98)90014-5

    • Google Scholar
67. 1. Morales M
    2. Mundy P
    3. Delgado CEF
    4. Yale M
    5. Messinger D
    6. Neal R
    7. Schwartz HK

    (2000) Responding to joint attention across the 6- through 24-month age period and early language acquisition

    Journal of Applied Developmental Psychology 21:283–298.

    https://doi.org/10.1016/S0193-3973(99)00040-4

    • Google Scholar
68. 1. Mulckhuyse M
    2. Theeuwes J

    (2010) Unconscious attentional orienting to exogenous cues: A review of the literature

    Acta Psychologica 134:299–309.

    https://doi.org/10.1016/j.actpsy.2010.03.002

    • PubMed
    • Google Scholar
69. 1. Najemnik J
    2. Geisler WS

    (2005) Optimal eye movement strategies in visual search

    Nature 434:387–391.

    https://doi.org/10.1038/nature03390

    • PubMed
    • Google Scholar
70. 1. Nation K
    2. Penny S

    (2008) Sensitivity to eye gaze in autism: Is it normal? Is it automatic? Is it social?

    Development and Psychopathology 20:79–97.

    https://doi.org/10.1017/S0954579408000047

    • PubMed
    • Google Scholar
71. 1. Or CCF
    2. Peterson MF
    3. Eckstein MP

    (2015) Initial eye movements during face identification are optimal and similar across cultures

    Journal of Vision 15:12.

    https://doi.org/10.1167/15.13.12

    • PubMed
    • Google Scholar
72. 1. Oram MW
    2. Perrett DI

    (1994) Responses of Anterior Superior Temporal Polysensory (STPa) Neurons to “Biological Motion” Stimuli

    Journal of Cognitive Neuroscience 6:99–116.

    https://doi.org/10.1162/jocn.1994.6.2.99

    • PubMed
    • Google Scholar
73. 1. Otsuka Y
    2. Mareschal I
    3. Calder AJ
    4. Clifford CWG

    (2014) Dual-route model of the effect of head orientation on perceived gaze direction

    Journal of Experimental Psychology. Human Perception and Performance 40:1425–1439.

    https://doi.org/10.1037/a0036151

    • PubMed
    • Google Scholar
74. 1. Pedregosa F
    2. Varoquaux G
    3. Gramfort A
    4. Michel V
    5. Thirion B
    6. Grisel O
    7. Blondel M
    8. Prettenhofer P
    9. Weiss R
    10. Dubourg V
    11. Vanderplas J
    12. Passos A
    13. Cournapeau D
    14. Brucher M
    15. Perrot M
    16. Duchesnay É

    (2011)

    Scikit-learn: Machine learning in python

    The Journal of Machine Learning Research 12:2825–2830.

    • Google Scholar
75. 1. Pelz JB
    2. Canosa R

    (2001) Oculomotor behavior and perceptual strategies in complex tasks

    Vision Research 41:3587–3596.

    https://doi.org/10.1016/s0042-6989(01)00245-0

    • PubMed
    • Google Scholar
76. 1. Peterson MF
    2. Eckstein MP

    (2012) Looking just below the eyes is optimal across face recognition tasks

    PNAS 109:E3314–E3323.

    https://doi.org/10.1073/pnas.1214269109

    • PubMed
    • Google Scholar
77. 1. Peterson MF
    2. Lin J
    3. Zaun I
    4. Kanwisher N

    (2016) Individual differences in face-looking behavior generalize from the lab to the world

    Journal of Vision 16:12.

    https://doi.org/10.1167/16.7.12

    • PubMed
    • Google Scholar
78. 1. Pi Z
    2. Xu K
    3. Liu C
    4. Yang J

    (2020) Instructor presence in video lectures: Eye gaze matters, but not body orientation

    Computers & Education 144:103713.

    https://doi.org/10.1016/j.compedu.2019.103713

    • Google Scholar
79. 1. Pierrot-Deseilligny C
    2. Milea D
    3. Müri RM

    (2004) Eye movement control by the cerebral cortex

    Current Opinion in Neurology 17:17–25.

    https://doi.org/10.1097/00019052-200402000-00005

    • PubMed
    • Google Scholar
80. 1. Rayner K

    (1998) Eye movements in reading and information processing: 20 years of research

    Psychological Bulletin 124:372–422.

    https://doi.org/10.1037/0033-2909.124.3.372

    • PubMed
    • Google Scholar
81. 1. Renninger LW
    2. Verghese P
    3. Coughlan J

    (2007) Where to look next? Eye movements reduce local uncertainty

    Journal of Vision 7:6.

    https://doi.org/10.1167/7.3.6

    • PubMed
    • Google Scholar
82. 1. Ristic J
    2. Friesen CK
    3. Kingstone A

    (2002) Are eyes special? It depends on how you look at it

    Psychonomic Bulletin & Review 9:507–513.

    https://doi.org/10.3758/bf03196306

    • PubMed
    • Google Scholar
83. 1. Ristic J
    2. Mottron L
    3. Friesen CK
    4. Iarocci G
    5. Burack JA
    6. Kingstone A

    (2005) Eyes are special but not for everyone: the case of autism

    Brain Research. Cognitive Brain Research 24:715–718.

    https://doi.org/10.1016/j.cogbrainres.2005.02.007

    • PubMed
    • Google Scholar
84. 1. Rucci M
    2. Iovin R
    3. Poletti M
    4. Santini F

    (2007) Miniature eye movements enhance fine spatial detail

    Nature 447:851–854.

    https://doi.org/10.1038/nature05866

    • PubMed
    • Google Scholar
85. 1. Senju A
    2. Csibra G

    (2008) Gaze following in human infants depends on communicative signals

    Current Biology 18:668–671.

    https://doi.org/10.1016/j.cub.2008.03.059

    • PubMed
    • Google Scholar
86. 1. Shepherd SV

    (2010) Following gaze: gaze-following behavior as a window into social cognition

    Frontiers in Integrative Neuroscience 4:5.

    https://doi.org/10.3389/fnint.2010.00005

    • PubMed
    • Google Scholar
87. 1. Stewart EEM
    2. Valsecchi M
    3. Schütz AC

    (2020) A review of interactions between peripheral and foveal vision

    Journal of Vision 20:2.

    https://doi.org/10.1167/jov.20.12.2

    • PubMed
    • Google Scholar
88. 1. Sun Z
    2. Yu W
    3. Zhou J
    4. Shen M

    (2017) Perceiving crowd attention: Gaze following in human crowds with conflicting cues

    Attention, Perception & Psychophysics 79:1039–1049.

    https://doi.org/10.3758/s13414-017-1303-z

    • PubMed
    • Google Scholar
89. 1. Vallat R

    (2018) Pingouin: statistics in Python

    Journal of Open Source Software 3:1026.

    https://doi.org/10.21105/joss.01026

    • Google Scholar
90. 1. van Beers RJ

    (2007) The sources of variability in saccadic eye movements

    The Journal of Neuroscience 27:8757–8770.

    https://doi.org/10.1523/JNEUROSCI.2311-07.2007

    • PubMed
    • Google Scholar
91. 1. Wallis LJ
    2. Range F
    3. Müller CA
    4. Serisier S
    5. Huber L
    6. Virányi Z

    (2015) Training for eye contact modulates gaze following in dogs

    Animal Behaviour 106:27–35.

    https://doi.org/10.1016/j.anbehav.2015.04.020

    • PubMed
    • Google Scholar
92. 1. Wang L
    2. Yang X
    3. Shi J
    4. Jiang Y

    (2014) The feet have it: local biological motion cues trigger reflexive attentional orienting in the brain

    NeuroImage 84:217–224.

    https://doi.org/10.1016/j.neuroimage.2013.08.041

    • PubMed
    • Google Scholar
93. 1. Wolf C
    2. Schütz AC

    (2015) Trans-saccadic integration of peripheral and foveal feature information is close to optimal

    Journal of Vision 15:1.

    https://doi.org/10.1167/15.16.1

    • PubMed
    • Google Scholar
94. 1. Wolf C
    2. Belopolsky AV
    3. Lappe M

    (2022) Current foveal inspection and previous peripheral preview influence subsequent eye movement decisions

    iScience 25:104922.

    https://doi.org/10.1016/j.isci.2022.104922

    • PubMed
    • Google Scholar
95. 1. Woodward AL

    (2003) Infants’ developing understanding of the link between looker and object

    Developmental Science 6:297–311.

    https://doi.org/10.1111/1467-7687.00286

    • Google Scholar

Author details

1. Nicole Xiao Han

   Department of Psychological and Brain Sciences, Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Project administration, Writing – review and editing

   For correspondence

   xhan01@ucsb.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2860-2743

2. Miguel Patricio Eckstein

   Department of Psychological and Brain Sciences, Department of Electrical and Computer Engineering, Department of Computer Science, Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, United States

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Validation, Methodology, Writing – original draft, Project administration, Writing – review and editing

   Competing interests

   No competing interests declared

• 578

  views

• 81

  downloads

• 1

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.