Dynamic updating of spatial working memory across eye movements: a computational investigation of transsaccadic integration

Reviewer #1 (Public review):

Summary:

This study employed a saccade-shifting sequential working memory paradigm, manipulating whether a saccade occurred after each memory array to directly compare retinotopic and transsaccadic working memory for both spatial location and color. Across four participant groups (young and older healthy adults, and patients with Parkinson's disease and Alzheimer's disease), the authors found a consistent saccade-related cost specifically for spatial memory - but not for color - regardless of differences in memory precision. Using computational modeling, they demonstrate that data from healthy participants are best explained by a complex saccade-based updating model that incorporates distractor interference. Applying this model to the patient groups further elucidates the sources of spatial memory deficits in PD and AD. The authors then extend the model to explain copying deficits in these patient groups, providing evidence for the ecological validity of the proposed saccade-updating retinotopic mechanism.

Strengths:

Overall, the manuscript is well written, and the experimental design is both novel and appropriate for addressing the authors' key research questions. I found the study to be particularly comprehensive: it first characterizes saccade-related costs in healthy young adults, then replicates these findings in healthy older adults, demonstrating how this "remapping" cost in spatial working memory is age-independent. After establishing and validating the best-fitting model using data from both healthy groups, the authors apply this model to clinical populations to identify potential mechanisms underlying their spatial memory impairments. The computational modeling results offer a clearer framework for interpreting ambiguities between allocentric and retinotopic spatial representations, providing valuable insight into how the brain represents and updates visual information across saccades. Moreover, the findings from the older adult and patient groups highlight factors that may contribute to spatial working memory deficits in aging and neurological disease, underscoring the broader translational significance of this work.

Weaknesses:

Several concerns should be addressed to enhance the clarity of the manuscript:

(1) Relevance of the figure-copy results (pp. 13-15).

Is it necessary to include the figure-copy task results within the main text? The manuscript already presents a clear and coherent narrative without this section. The figure-copy task represents a substantial shift from the LOCUS paradigm to an entirely different task that does not measure the same construct. Moreover, the ROCF findings are not fully consistent with the LOCUS results, which introduces confusion and weakens the manuscript's coherence. While I understand the authors' intention to assess the ecological validity of their model, this section does not effectively strengthen the manuscript and may be better removed or placed in the Supplementary Materials.

(2) Model fitting across age groups (p. 9).

It is unclear whether it is appropriate to fit healthy young and healthy elderly participants' data to the same model simultaneously. If the goal of the model fitting is to account for behavioral performance across all conditions, combining these groups may be problematic, as the groups differ significantly in overall performance despite showing similar remapping costs. This suggests that model performance might differ meaningfully between age groups. For example, in Figure 4A, participants 22-42 (presumably the elderly group) show the best fit for the Dual (Saccade) model, implying that the Interference component may contribute less to explaining elderly performance.

Furthermore, although the most complex model emerges as the best-fitting model, the manuscript should explain how model complexity is penalized or balanced in the model comparison procedure. Additionally, are Fixation Decay and Saccade Update necessarily alternative mechanisms? Could both contribute simultaneously to spatial memory representation? A model that includes both mechanisms-e.g., Dual (Fixation) + Dual (Saccade) + Interference-could be tested to determine whether it outperforms Model 7 to rule out the sole contribution of complexity.

Minor point: On p. 9, line 336, Figure 4A does not appear to include the red dashed vertical line that is mentioned as separating the age groups.

(3) Clarification of conceptual terminology.

Some conceptual distinctions are unclear. For example, the relationship between "retinal memory" and "transsaccadic memory," as well as between "allocentric map" and "retinotopic representation," is not fully explained. Are these constructs related or distinct? Additionally, the manuscript uses terms such as "allocentric map," "retinotopic representation," and "reference frame" interchangeably, which creates ambiguity. It would be helpful for the authors to clarify the relationships among these terms and apply them consistently.

(4) Rationale for the selective disruption hypothesis (p. 4, lines 153-154).

The authors hypothesize that "saccades would selectively disrupt location memory while leaving colour memory intact." Providing theoretical or empirical justification for this prediction would strengthen the argument.

(5) Relationship between saccade cost and individual memory performance (p. 4, last paragraph).

The authors report that larger saccades were associated with greater spatial memory disruption. It would be informative to examine whether individual differences in the magnitude of saccade cost correlate with participants' overall/baseline memory performance (e.g. their memory precision in the no-saccade condition). Such analyses might offer insights into how memory capacity/ability relates to resilience against saccade-induced updating.

(6) Model fitting for the healthy elderly group to reveal memory-deficit factors (pp. 11-12).

The manuscript discusses model-based insights into components that contribute to spatial memory deficits in AD and PD, but does not discuss components that contribute to spatial memory deficits in the healthy elderly group. Given that the EC group also shows impairments in certain parameters, explaining and discussing these outcomes of the EC group could provide additional insights into age-related memory decline, which would strengthen the study's broader conclusions.

(7) Presentation of saccade conditions in Figure 5 (p. 11).

In Figure 5, it may be clearer to group the four saccade conditions together within each patient group. Since the main point is that saccadic interference on spatial memory remains robust across patient groups, grouping conditions by patient type rather than intermixing conditions would emphasize this interpretation.

Reviewer #2 (Public review):

Summary:

Zhao et al investigate how object location and colour are degraded across saccadic eye movements. They employ an eye-tracking task that requires participants to remember two sequentially presented items and subsequently report the colour and position of either one of these. Through counterbalancing of the presence or absence of saccades across items, the authors endeavour to dissect the impact of saccades independently on item location or colour. These behavioural findings form the basis of generative models designed to test competing, nested accounts of how stored information is stored and updated across saccades.

Strengths:

The combination of eye-tracking and generative modelling is a strength of the paper, which opens new perspectives into the impact of Alzheimer's and Parkinson's disease on the performance of visuospatial cognitive tests. The finding that the model parameters covary with clinical performance on the ROCF test is a nice example of a "computational assay" of disease.

Weaknesses:

I have a number of substantial and minor concerns for the authors to consider in a revision:

Reviewer #3 (Public review):

Summary:

The manuscript introduces a visual paradigm aimed at studying trans-saccadic memory.

The authors observe how memory of object location is selectively impaired across eye movements, whereas object colour memory is relatively immune to intervening eye movements.
Results are reported for young and elderly healthy controls, as well as PD and AD participants.

A computational model is introduced to account for these results, indicating how early differences in memory encoding and decay (but not trans-saccadic updating per se) can account for the observed differences between healthy controls and clinical groups.

Strengths:

The data presented encompasses healthy and elderly controls, as well as clinical groups.

The authors introduce an interesting modelling strategy, aimed at isolating and identifying the main components behind the observed pattern of results.

Weaknesses:

The models tested differ in terms of the number of parameters. In general, a larger number of parameters leads to a better goodness of fit. It is not clear how the difference in the number of parameters between the models was taken into account.

It is not clear whether the modelling results could be influenced by overfitting (it is not clear how well the model can generalize to new observations).

Results specificity: it is not clear how specific the modelling results are with respect to constructional ability (measured via the Rey-Osterrieth Complex Figure test). As with any cognitive test, performance can also be influenced by general, non-specific abilities that contribute broadly to test success.

Author response:

(1) About ROCF figure-copy results

Reviewer #1 queried the necessity of including the Rey-Osterrieth Complex Figure (ROCF) results in the main text. We appreciate the reviewer’s perspective on the narrative flow and the transition between the LOCUS paradigm and the ROCF results. However, we remain keen to retain these findings in the main tex, as they provide critical ecological and clinical validation for the computational mechanisms identified in our study.

We argue that the following points support the retention of these results:

(1) The ROCF we used is a standard neuropsychological tool for identifying constructional apraxia. Our results bridge the gap between basic cognitive neuroscience and clinical application by demonstrating that specific remapping parameters—rather than general memory precision—predict real-world deficits in patients.

(2) The finding that our winning model explains approximately 62% of the variance in ROCF copy scores across all diagnostic groups further indicates that these parameters from the LOCUS task represent core computational phenotypes that underpin complex, real-life visuospatial construction (copying drawings).

(3) Previous research has often observed only a weak or indirect link between drawing ability and traditional working memory measures, such as digit span (Senese et al., 2020). This was previously attributed to “deictic” strategies—like frequent eye movements—that minimise the need to hold large amounts of information in memory (Ballard et al., 1995; Cohen, 2005; Draschkow et al., 2021). While our study was not exclusively designed to catalogue all cognitive contributions to drawing, our findings provide significant and novel evidence indicating that transsaccadic integration is a critical driver of constructional (copying drawing) ability. By demonstrating this link, we offer a new direction for future research, shifting the focus from general memory capacity toward the precision of spatial updating across eye movements.

By including the ROCF results in the main text, we provide evidence for a functional role for spatial remapping that extends beyond perceptual stability into the domain of complex visuomotor control. We will expand on these points in the Discussion in our revised manuscript.

(2) Model complexity and overfitting

We would like to clarify that the Bayesian model selection (BMS) procedure utilised in this manuscript inherently balances model fit with parsimony. Unlike maximum likelihood inference, where overfitting is a primary concern often requiring cross-validation via out-of-sample prediction, our approach depends upon the comparison of marginal likelihoods. This method directly penalises model complexity — a principle often described as the “Bayesian Occam’s Razor” (Rasmussen and Ghahramani, 2000). This means that a model is only favoured if the improvement in fit justifies the additional parameter space. If a parameter were redundant, it would lower the model's evidence by “diluting” the probability mass over the parameter space. The emergence of the “Dual (Saccade) + Interference” model as the winning candidate suggests it offers the most plausible generative account of the data while maintaining necessary parsimony. We would be happy to point toward literature that discusses how these marginal likelihood approximations provide a more robust guard against overfitting than standard metrics like BIC or AIC (MacKay, 2003; Murray and Ghahramani, 2005; Penny, 2012).

(3) On model fitting across age groups

This approach is primarily supported by our empirical findings: there was no significant interaction between age group and saccade condition for either location or colour memory. While older adults demonstrated lower baseline precision, the specific disruptive effect of saccades (the “saccade cost”) was remarkably consistent across cohorts. This justifies the use of a common generative model to assess quantitative differences in parameter estimates.

This approach does implicitly assume that participants perform the task in a qualitatively similar way. However, as this assumption is mitigated by the fact that our winning model nests simpler models as special cases, it supports the assessment of group differences in parameters that play consistent mechanistic roles. This flexibility allows the model to naturally accommodate groups where certain components—such as interference—may play a reduced role, while remaining sensitive to the specific mechanistic failures that differentiate healthy aging from neurodegeneration.

(4) Conceptual terminology and patient group descriptions

We will clarify our conceptual terminology, explicitly defining the relationships between retinotopic (eye-centred), transsaccadic (across-saccade), and spatiotopic (world-centred) representations.

Regarding the demographics of the clinical cohorts, we apologise for any lack of clarity in our initial presentation. The patient demographics for both the Parkinson’s disease (PD) and Alzheimer’s disease (AD) groups—including age, gender, education, and ACE-III scores—are currently detailed alongside the healthy control data (two groups: Young Healthy Controls and Elderly Healthy Controls) in the table within the Participants section of the Materials and Methods. In our revision. We will ensure that this table is correctly labelled as Table 2 and will provide more comprehensive recruitment and characterisation details for both patient groups within the main text. Finally, we will include a detailed discussion in the Supplementary Materials regarding eye-tracking data quality across all cohorts, specifically comparing calibration accuracy, trace stability, and trial rejection rates to demonstrate that our findings are not confounded by differences in recording quality between healthy and clinical populations.

References

Ballard DH, Hayhoe MM, Pelz JB. 1995. Memory Representations in Natural Tasks. Journal of Cognitive Neuroscience 7:66–80. DOI: https://doi.org/10.1162/jocn.1995.7.1.66

Cohen DJ. 2005. Look little, look often: The influence of gaze frequency on drawing accuracy. Perception & Psychophysics 67:997–1009. DOI: https://doi.org/10.3758/BF03193626

Draschkow D, Kallmayer M, Nobre AC. 2021. When Natural Behavior Engages Working Memory. Current Biology 31:869-874.e5. DOI: https://doi.org/10.1016/j.cub.2020.11.013, PMID: 33278355

MacKay DJC. 2003. Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Murray I, Ghahramani Z. 2005. A note on the evidence and Bayesian Occam’s razor (Technical report No. GCNU TR 2005-003). Gatsby Unit.

Penny WD. 2012. Comparing Dynamic Causal Models using AIC, BIC and Free Energy. Neuroimage 59:319–330. DOI: https://doi.org/10.1016/j.neuroimage.2011.07.039, PMID: 21864690

Rasmussen C, Ghahramani Z. 2000. Occam’ s Razor. Advances in Neural Information Processing Systems. MIT Press.

Senese VP, Zappullo I, Baiano C, Zoccolotti P, Monaco M, Conson M. 2020. Identifying neuropsychological predictors of drawing skills in elementary school children. Child Neuropsychology 26:345–361. DOI: https://doi.org/10.1080/09297049.2019.1651834, PMID: 31390949