Schema-based predictive eye movements support sequential memory encoding

  1. Jiawen Huang  Is a corresponding author
  2. Isabel Velarde
  3. Wei Ji Ma
  4. Christopher Baldassano
  1. Columbia University, United States
  2. New York University, United States

Abstract

When forming a memory of an experience that is unfolding over time, we can use our schematic knowledge about the world (constructed based on many prior episodes) to predict what will transpire. We developed a novel paradigm to study how the development of a complex schema influences predictive processes during perception and impacts sequential memory. Participants learned to play a novel board game ('4-in-a-row') across six training sessions, and repeatedly performed a memory test in which they watched and recalled sequences of moves from the game. We found that participants gradually became better at remembering sequences from the game as their schema developed, driven by improved accuracy for schema-consistent moves. Eye tracking revealed that increased predictive eye movements during encoding, which were most prevalent in expert players, were associated with better memory. Our results identify prediction as a mechanism by which schematic knowledge can improve episodic memory.

Data availability

All the data is openly available through https://osf.io/29cpg/

The following data sets were generated

Article and author information

Author details

  1. Jiawen Huang

    Columbia University, New York, United States
    For correspondence
    jh4290@columbia.edu
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1362-0412
  2. Isabel Velarde

    Columbia University, New York, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5639-0907
  3. Wei Ji Ma

    Department of Psychology, New York University, New York, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-9835-9083
  4. Christopher Baldassano

    Columbia University, New York, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3540-5019

Funding

Columbia University (Graduate Student Fellowship)

  • Jiawen Huang

Columbia University (start-up funding)

  • Christopher Baldassano

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: The experimental protocol was approved by the Institutional Review Board of Columbia University. (AAAS0252) All participants were over 18 years of age with normal or corrected-to-normal vision, and gave informed consent.

Reviewing Editor

  1. Huan Luo, Peking University, China

Version history

  1. Preprint posted: July 20, 2022 (view preprint)
  2. Received: August 10, 2022
  3. Accepted: March 24, 2023
  4. Accepted Manuscript published: March 27, 2023 (version 1)
  5. Version of Record published: April 12, 2023 (version 2)

Copyright

© 2023, Huang et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,207
    Page views
  • 178
    Downloads
  • 2
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Jiawen Huang
  2. Isabel Velarde
  3. Wei Ji Ma
  4. Christopher Baldassano
(2023)
Schema-based predictive eye movements support sequential memory encoding
eLife 12:e82599.
https://doi.org/10.7554/eLife.82599

Share this article

https://doi.org/10.7554/eLife.82599

Further reading

    1. Neuroscience
    Maureen van der Grinten, Jaap de Ruyter van Steveninck ... Yağmur Güçlütürk
    Tools and Resources

    Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or ‘phosphenes’) has limited resolution, and a great portion of the field’s research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator’s suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

    1. Neuroscience
    Simon Lui, Ashleigh K Brink, Laura H Corbit
    Research Article

    Extinction is a specific example of learning where a previously reinforced stimulus or response is no longer reinforced, and the previously learned behaviour is no longer necessary and must be modified. Current theories suggest extinction is not the erasure of the original learning but involves new learning that acts to suppress the original behaviour. Evidence for this can be found when the original behaviour recovers following the passage of time (spontaneous recovery) or reintroduction of the reinforcement (i.e. reinstatement). Recent studies have shown that pharmacological manipulation of noradrenaline (NA) or its receptors can influence appetitive extinction; however, the role and source of endogenous NA in these effects are unknown. Here, we examined the role of the locus coeruleus (LC) in appetitive extinction. Specifically, we tested whether optogenetic stimulation of LC neurons during extinction of a food-seeking behaviour would enhance extinction evidenced by reduced spontaneous recovery in future tests. LC stimulation during extinction trials did not change the rate of extinction but did serve to reduce subsequent spontaneous recovery, suggesting that stimulation of the LC can augment reward-related extinction. Optogenetic inhibition of the LC during extinction trials reduced responding during the trials where it was applied, but no long-lasting changes in the retention of extinction were observed. Since not all LC cells expressed halorhodopsin, it is possible that more complete LC inhibition or pathway-specific targeting would be more effective at suppressing extinction learning. These results provide further insight into the neural basis of appetitive extinction, and in particular the role of the LC. A deeper understanding of the physiological bases of extinction can aid development of more effective extinction-based therapies.