Dopamine neurons learn relative chosen value from probabilistic rewards

  1. Armin Lak  Is a corresponding author
  2. William R Stauffer
  3. Wolfram Schultz
  1. University College London, United Kingdom
  2. University of Cambridge, United Kingdom

Abstract

Economic theories posit reward probability as one of the factors defining reward value. Individuals learn the value of cues that predict probabilistic rewards from experienced reward frequencies. Building on the notion that responses of dopamine neurons increase with reward probability and expected value, we asked how dopamine neurons in monkeys acquire this value signal that may represent an economic decision variable. We found in a Pavlovian learning task that reward probability-dependent value signals arose from experienced reward frequencies. We then assessed neuronal response acquisition during choices among probabilistic rewards. Here, dopamine responses became sensitive to the value of both chosen and unchosen options. Both experiments showed also the novelty responses of dopamine neurones that decreased as learning advanced. These results show that dopamine neurons acquire predictive value signals from the frequency of experienced rewards. This flexible and fast signal reflects a specific decision variable and could update neuronal decision mechanisms.

Article and author information

Author details

  1. Armin Lak

    Institute of Ophthalmology, University College London, London, United Kingdom
    For correspondence
    arminlak@gmail.com
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1926-5458
  2. William R Stauffer

    Department of Physiology, Development, and Neuroscience, University of Cambridge, Cambridge, United Kingdom
    Competing interests
    No competing interests declared.
  3. Wolfram Schultz

    Department of Physiology, Development, and Neuroscience, University of Cambridge, Cambridge, United Kingdom
    Competing interests
    Wolfram Schultz, Reviewing editor, eLife.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-8530-4518

Funding

Wellcome (WT106101)

  • Armin Lak

Wellcome

  • Wolfram Schultz

European Research Council

  • Wolfram Schultz

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: All experimental protocols and procedures were approved by the Home Office of the United Kingdom (project licence number: 80 / 2416).

Copyright

© 2016, Lak et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 3,946
    views
  • 854
    downloads
  • 74
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Armin Lak
  2. William R Stauffer
  3. Wolfram Schultz
(2016)
Dopamine neurons learn relative chosen value from probabilistic rewards
eLife 5:e18044.
https://doi.org/10.7554/eLife.18044

Share this article

https://doi.org/10.7554/eLife.18044

Further reading

    1. Neuroscience
    Vincent Huson, Wade G Regehr
    Research Article

    Unipolar brush cells (UBCs) are excitatory interneurons in the cerebellar cortex that receive mossy fiber (MF) inputs and excite granule cells. The UBC population responds to brief burst activation of MFs with a continuum of temporal transformations, but it is not known how UBCs transform the diverse range of MF input patterns that occur in vivo. Here, we use cell-attached recordings from UBCs in acute cerebellar slices to examine responses to MF firing patterns that are based on in vivo recordings. We find that MFs evoke a continuum of responses in the UBC population, mediated by three different types of glutamate receptors that each convey a specialized component. AMPARs transmit timing information for single stimuli at up to 5 spikes/s, and for very brief bursts. A combination of mGluR2/3s (inhibitory) and mGluR1s (excitatory) mediates a continuum of delayed, and broadened responses to longer bursts, and to sustained high frequency activation. Variability in the mGluR2/3 component controls the time course of the onset of firing, and variability in the mGluR1 component controls the duration of prolonged firing. We conclude that the combination of glutamate receptor types allows each UBC to simultaneously convey different aspects of MF firing. These findings establish that UBCs are highly flexible circuit elements that provide diverse temporal transformations that are well suited to contribute to specialized processing in different regions of the cerebellar cortex.

    1. Neuroscience
    Eleni Hackwell, Sharon R Ladyman ... David R Grattan
    Research Article

    The specific role that prolactin plays in lactational infertility, as distinct from other suckling or metabolic cues, remains unresolved. Here, deletion of the prolactin receptor (Prlr) from forebrain neurons or arcuate kisspeptin neurons resulted in failure to maintain normal lactation-induced suppression of estrous cycles. Kisspeptin immunoreactivity and pulsatile LH secretion were increased in these mice, even in the presence of ongoing suckling stimulation and lactation. GCaMP fibre photometry of arcuate kisspeptin neurons revealed that the normal episodic activity of these neurons is rapidly suppressed in pregnancy and this was maintained throughout early lactation. Deletion of Prlr from arcuate kisspeptin neurons resulted in early reactivation of episodic activity of kisspeptin neurons prior to a premature return of reproductive cycles in early lactation. These observations show dynamic variation in arcuate kisspeptin neuronal activity associated with the hormonal changes of pregnancy and lactation, and provide direct evidence that prolactin action on arcuate kisspeptin neurons is necessary for suppressing fertility during lactation in mice.