Response-based outcome predictions and confidence regulate feedback processing and learning

  1. Romy Frömer  Is a corresponding author
  2. Matthew R Nassar
  3. Rasmus Bruckner
  4. Birgit Stürmer
  5. Werner Sommer
  6. Nick Yeung
  1. Brown University, United States
  2. Freie University, Germany
  3. International Psychoanalytic University Berlin, Germany
  4. Humboldt Universität zu Berlin, Germany
  5. University of Oxford, United Kingdom

Abstract

Influential theories emphasize the importance of predictions in learning: we learn from feedback to the extent that it is surprising, and thus conveys new information. Here we explore the hypothesis that surprise depends not only on comparing current events to past experience, but also on online evaluation of performance via internal monitoring. Specifically, we propose that people leverage insights from response-based performance monitoring – outcome predictions and confidence – to control learning from feedback. In line with predictions from a Bayesian inference model, we find that people who are better at calibrating their confidence to the precision of their outcome predictions learn more quickly. Further in line with our proposal, EEG signatures of feedback processing are sensitive to the accuracy of, and confidence in, post-response outcome predictions. Taken together, our results suggest that online predictions and confidence serve to calibrate neural error signals to improve the efficiency of learning.

Data availability

Scripts and source data for all analyses are available under https://github.com/froemero/Outcome-Predictions-and-Confidence-Regulate-Learning.

Article and author information

Author details

  1. Romy Frömer

    Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, United States
    For correspondence
    romy_fromer@brown.edu
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-9468-4014
  2. Matthew R Nassar

    Robert J and Nancy D Carney Institute for Brain Science, Brown University, Providence, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5397-535X
  3. Rasmus Bruckner

    Department of Education and Psychology, Freie University, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3033-6299
  4. Birgit Stürmer

    General Psychology and Neurocognitive Psychology, International Psychoanalytic University Berlin, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
  5. Werner Sommer

    Psychology, Humboldt Universität zu Berlin, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
  6. Nick Yeung

    University of Oxford, Oxford, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.

Funding

NIH Office of the Director (R00 AG054732)

  • Matthew R Nassar

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: The study was performed following the guidelines of the ethics committee of the department of Psychology at Humboldt University. Participants gave informed consent to the experiment and were remunerated with course credits or 8 € per hour.

Reviewing Editor

  1. Tadeusz Wladyslaw Kononowicz, Cognitive Neuroimaging Unit, CEA DRF/Joliot, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, France

Version history

  1. Received: September 4, 2020
  2. Accepted: April 30, 2021
  3. Accepted Manuscript published: April 30, 2021 (version 1)
  4. Version of Record published: May 14, 2021 (version 2)
  5. Version of Record updated: November 23, 2021 (version 3)

Copyright

© 2021, Frömer et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,968
    Page views
  • 416
    Downloads
  • 16
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Romy Frömer
  2. Matthew R Nassar
  3. Rasmus Bruckner
  4. Birgit Stürmer
  5. Werner Sommer
  6. Nick Yeung
(2021)
Response-based outcome predictions and confidence regulate feedback processing and learning
eLife 10:e62825.
https://doi.org/10.7554/eLife.62825

Share this article

https://doi.org/10.7554/eLife.62825

Further reading

    1. Neuroscience
    Maureen van der Grinten, Jaap de Ruyter van Steveninck ... Yağmur Güçlütürk
    Tools and Resources

    Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or ‘phosphenes’) has limited resolution, and a great portion of the field’s research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator’s suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

    1. Neuroscience
    Simon Lui, Ashleigh K Brink, Laura H Corbit
    Research Article

    Extinction is a specific example of learning where a previously reinforced stimulus or response is no longer reinforced, and the previously learned behaviour is no longer necessary and must be modified. Current theories suggest extinction is not the erasure of the original learning but involves new learning that acts to suppress the original behaviour. Evidence for this can be found when the original behaviour recovers following the passage of time (spontaneous recovery) or reintroduction of the reinforcement (i.e. reinstatement). Recent studies have shown that pharmacological manipulation of noradrenaline (NA) or its receptors can influence appetitive extinction; however, the role and source of endogenous NA in these effects are unknown. Here, we examined the role of the locus coeruleus (LC) in appetitive extinction. Specifically, we tested whether optogenetic stimulation of LC neurons during extinction of a food-seeking behaviour would enhance extinction evidenced by reduced spontaneous recovery in future tests. LC stimulation during extinction trials did not change the rate of extinction but did serve to reduce subsequent spontaneous recovery, suggesting that stimulation of the LC can augment reward-related extinction. Optogenetic inhibition of the LC during extinction trials reduced responding during the trials where it was applied, but no long-lasting changes in the retention of extinction were observed. Since not all LC cells expressed halorhodopsin, it is possible that more complete LC inhibition or pathway-specific targeting would be more effective at suppressing extinction learning. These results provide further insight into the neural basis of appetitive extinction, and in particular the role of the LC. A deeper understanding of the physiological bases of extinction can aid development of more effective extinction-based therapies.