Uncertainty alters the balance between incremental learning and episodic memory

  1. Jonathan Nicholas  Is a corresponding author
  2. Nathaniel D Daw
  3. Daphna Shohamy
  1. Columbia University, United States
  2. Princeton University, United States

Abstract

A key question in decision making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.

Data availability

All code, data, and software needed to reproduce the manuscript can be found here: https://codeocean.com/capsule/2024716/tree/v1; DOI: https://doi.org/10.24433/CO.1266819.v1

The following data sets were generated

Article and author information

Author details

  1. Jonathan Nicholas

    Columbia University, New York, United States
    For correspondence
    jonathan.nicholas@columbia.edu
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-2314-0765
  2. Nathaniel D Daw

    Department of Psychology, Princeton University, Princeton, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5029-1430
  3. Daphna Shohamy

    Columbia University, New York, United States
    Competing interests
    The authors declare that no competing interests exist.

Funding

National Science Foundation (1644869)

  • Jonathan Nicholas

National Science Foundation (1822619)

  • Nathaniel D Daw

National Science Foundation (1822619)

  • Daphna Shohamy

National Institutes of Health (MH121093)

  • Nathaniel D Daw

National Institutes of Health (MH121093)

  • Daphna Shohamy

John Templeton Foundation (60844)

  • Daphna Shohamy

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: Informed consent was obtained online with approval from the Columbia University Institutional Review Board (IRB #1488)

Reviewing Editor

  1. Michael J Frank, Brown University, United States

Version history

  1. Preprint posted: July 6, 2022 (view preprint)
  2. Received: July 7, 2022
  3. Accepted: December 1, 2022
  4. Accepted Manuscript published: December 2, 2022 (version 1)
  5. Version of Record published: January 3, 2023 (version 2)

Copyright

© 2022, Nicholas et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,783
    Page views
  • 294
    Downloads
  • 6
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Jonathan Nicholas
  2. Nathaniel D Daw
  3. Daphna Shohamy
(2022)
Uncertainty alters the balance between incremental learning and episodic memory
eLife 11:e81679.
https://doi.org/10.7554/eLife.81679

Share this article

https://doi.org/10.7554/eLife.81679

Further reading

    1. Developmental Biology
    2. Neuroscience
    Athina Keramidioti, Sandra Schneid ... Charles N David
    Research Article

    The Hydra nervous system is the paradigm of a ‘simple nerve net’. Nerve cells in Hydra, as in many cnidarian polyps, are organized in a nerve net extending throughout the body column. This nerve net is required for control of spontaneous behavior: elimination of nerve cells leads to polyps that do not move and are incapable of capturing and ingesting prey (Campbell, 1976). We have re-examined the structure of the Hydra nerve net by immunostaining fixed polyps with a novel antibody that stains all nerve cells in Hydra. Confocal imaging shows that there are two distinct nerve nets, one in the ectoderm and one in the endoderm, with the unexpected absence of nerve cells in the endoderm of the tentacles. The nerve nets in the ectoderm and endoderm do not contact each other. High-resolution TEM (transmission electron microscopy) and serial block face SEM (scanning electron microscopy) show that the nerve nets consist of bundles of parallel overlapping neurites. Results from transgenic lines show that neurite bundles include different neural circuits and hence that neurites in bundles require circuit-specific recognition. Nerve cell-specific innexins indicate that gap junctions can provide this specificity. The occurrence of bundles of neurites supports a model for continuous growth and differentiation of the nerve net by lateral addition of new nerve cells to the existing net. This model was confirmed by tracking newly differentiated nerve cells.

    1. Neuroscience
    Anna-Maria Grob, Hendrik Heinbockel ... Lars Schwabe
    Research Article

    Maintaining an accurate model of the world relies on our ability to update memory representations in light of new information. Previous research on the integration of new information into memory mainly focused on the hippocampus. Here, we hypothesized that the angular gyrus, known to be involved in episodic memory and imagination, plays a pivotal role in the insight-driven reconfiguration of memory representations. To test this hypothesis, participants received continuous theta burst stimulation (cTBS) over the left angular gyrus or sham stimulation before gaining insight into the relationship between previously separate life-like animated events in a narrative-insight task. During this task, participants also underwent EEG recording and their memory for linked and non-linked events was assessed shortly thereafter. Our results show that cTBS to the angular gyrus decreased memory for the linking events and reduced the memory advantage for linked relative to non-linked events. At the neural level, cTBS targeting the angular gyrus reduced centro-temporal coupling with frontal regions and abolished insight-induced neural representational changes for events linked via imagination, indicating impaired memory reconfiguration. Further, the cTBS group showed representational changes for non-linked events that resembled the patterns observed in the sham group for the linked events, suggesting failed pruning of the narrative in memory. Together, our findings demonstrate a causal role of the left angular gyrus in insight-related memory reconfigurations.