Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system

  1. Kiyohito Iigaya  Is a corresponding author
  1. University College London, United Kingdom

Abstract

Recent experiments have shown that animals and humans have a remarkable ability to adapt their learning rate according to the volatility of the environment. Yet the neural mechanism responsible for such adaptive learning has remained unclear. To fill this gap, we investigated a biophysically inspired, metaplastic synaptic model within the context of a well-studied decision-making network, in which synapses can change their rate of plasticity in addition to their efficacy according to a reward-based learning rule. We found that our model, which assumes that synaptic plasticity is guided by a novel surprise detection system, captures a wide range of key experimental findings and performs as well as a Bayes optimal model, with remarkably little parameter tuning. Our results further demonstrate the computational power of synaptic plasticity, and provide insights into the circuit-level computation which underlies adaptive decision-making.

Article and author information

Author details

  1. Kiyohito Iigaya

    Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom
    For correspondence
    kiigaya@gatsby.ucl.ac.uk
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-4748-8432

Funding

Schwartz foundation

  • Kiyohito Iigaya

Gatsby Charitable Foundation

  • Kiyohito Iigaya

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Naoshige Uchida, Harvard University, United States

Version history

  1. Received: May 23, 2016
  2. Accepted: August 8, 2016
  3. Accepted Manuscript published: August 9, 2016 (version 1)
  4. Version of Record published: September 1, 2016 (version 2)

Copyright

© 2016, Iigaya

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,517
    views
  • 506
    downloads
  • 41
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Kiyohito Iigaya
(2016)
Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system
eLife 5:e18073.
https://doi.org/10.7554/eLife.18073

Share this article

https://doi.org/10.7554/eLife.18073

Further reading

    1. Neuroscience
    Laura Boi, Yvonne Johansson ... Gilad Silberberg
    Research Article

    Parkinson’s disease (PD) is characterized by motor impairments caused by degeneration of dopamine neurons in the substantia nigra pars compacta. In addition to these symptoms, PD patients often suffer from non-motor comorbidities including sleep and psychiatric disturbances, which are thought to depend on concomitant alterations of serotonergic and noradrenergic transmission. A primary locus of serotonergic neurons is the dorsal raphe nucleus (DRN), providing brain-wide serotonergic input. Here, we identified electrophysiological and morphological parameters to classify serotonergic and dopaminergic neurons in the murine DRN under control conditions and in a PD model, following striatal injection of the catecholamine toxin, 6-hydroxydopamine (6-OHDA). Electrical and morphological properties of both neuronal populations were altered by 6-OHDA. In serotonergic neurons, most changes were reversed when 6-OHDA was injected in combination with desipramine, a noradrenaline (NA) reuptake inhibitor, protecting the noradrenergic terminals. Our results show that the depletion of both NA and dopamine in the 6-OHDA mouse model causes changes in the DRN neural circuitry.

    1. Neuroscience
    Etienne Combrisson, Ruggero Basanisi ... Andrea Brovelli
    Research Article

    How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.