A Bayesian model of context-sensitive value attribution

  1. Francesco Rigoli  Is a corresponding author
  2. Karl J Friston
  3. Cristina Martinelli
  4. Mirjana Selaković
  5. Sukhwinder S Shergill
  6. Raymond J Dolan
  1. University College London, United Kingdom
  2. University College London, United Kingdom
  3. King's College London, United Kingdom
  4. Sismanogleio General Hospital, Greece

Abstract

Substantial evidence indicates that incentive value depends on an anticipation of rewards within a given context. However, the computations underlying this context sensitivity remain unknown. To address this question we introduce a normative (Bayesian) account of how rewards map to incentive values. This assumes that the brain inverts a model of how rewards are generated. Key features of our account include (i) an influence of prior beliefs about the context in which rewards are delivered (weighted by their reliability in a Bayes-optimal fashion), (ii) the notion that incentive values correspond to precision-weighted prediction errors, (iii) and contextual information unfolding at different hierarchical levels. This formulation implies that incentive value is intrinsically context-dependent. We provide empirical support for this model by showing that incentive value is influenced by context variability and by hierarchically nested contexts. The perspective we introduce generates new empirical predictions that might help explaining psychopathologies, such as addiction.

Article and author information

Author details

  1. Francesco Rigoli

    The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
    For correspondence
    f.rigoli@ucl.ac.uk
    Competing interests
    The authors declare that no competing interests exist.
  2. Karl J Friston

    The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
  3. Cristina Martinelli

    Department of Psychosis Studies, King's College London, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
  4. Mirjana Selaković

    Department of Psychiatry, Sismanogleio General Hospital, Athens, Greece
    Competing interests
    The authors declare that no competing interests exist.
  5. Sukhwinder S Shergill

    Department of Psychosis Studies, King's College London, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
  6. Raymond J Dolan

    The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.

Ethics

Human subjects: Experiment one was approved by the University College London Research Ethics Committee. Experiment two was approved by the King's College of London Research Ethics Committee. All participants provided written informed consent and were paid for participating.

Copyright

© 2016, Rigoli et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,155
    views
  • 450
    downloads
  • 23
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Francesco Rigoli
  2. Karl J Friston
  3. Cristina Martinelli
  4. Mirjana Selaković
  5. Sukhwinder S Shergill
  6. Raymond J Dolan
(2016)
A Bayesian model of context-sensitive value attribution
eLife 5:e16127.
https://doi.org/10.7554/eLife.16127

Share this article

https://doi.org/10.7554/eLife.16127

Further reading

    1. Neuroscience
    Nico A Flierman, Sue Ann Koay ... Chris I De Zeeuw
    Research Article

    The role of cerebellum in controlling eye movements is well established, but its contribution to more complex forms of visual behavior has remained elusive. To study cerebellar activity during visual attention we recorded extracellular activity of dentate nucleus (DN) neurons in two non-human primates (NHPs). NHPs were trained to read the direction indicated by a peripheral visual stimulus while maintaining fixation at the center, and report the direction of the cue by performing a saccadic eye movement into the same direction following a delay. We found that single-unit DN neurons modulated spiking activity over the entire time course of the task, and that their activity often bridged temporally separated intra-trial events, yet in a heterogeneous manner. To better understand the heterogeneous relationship between task structure, behavioral performance, and neural dynamics, we constructed a behavioral, an encoding, and a decoding model. Both NHPs showed different behavioral strategies, which influenced the performance. Activity of the DN neurons reflected the unique strategies, with the direction of the visual stimulus frequently being encoded long before an upcoming saccade. Moreover, the latency of the ramping activity of DN neurons following presentation of the visual stimulus was shorter in the better performing NHP. Labeling with the retrograde tracer Cholera Toxin B in the recording location in the DN indicated that these neurons predominantly receive inputs from Purkinje cells in the D1 and D2 zones of the lateral cerebellum as well as neurons of the principal olive and medial pons, all regions known to connect with neurons in the prefrontal cortex contributing to planning of saccades. Together, our results highlight that DN neurons can dynamically modulate their activity during a visual attention task, comprising not only sensorimotor but also cognitive attentional components.

    1. Neuroscience
    Robert A Bruce, Matthew Weber ... Kumar Narayanan
    Research Article

    The role of striatal pathways in cognitive processing is unclear. We studied dorsomedial striatal cognitive processing during interval timing, an elementary cognitive task that requires mice to estimate intervals of several seconds and involves working memory for temporal rules as well as attention to the passage of time. We harnessed optogenetic tagging to record from striatal D2-dopamine receptor-expressing medium spiny neurons (D2-MSNs) in the indirect pathway and from D1-dopamine receptor-expressing MSNs (D1-MSNs) in the direct pathway. We found that D2-MSNs and D1-MSNs exhibited distinct dynamics over temporal intervals as quantified by principal component analyses and trial-by-trial generalized linear models. MSN recordings helped construct and constrain a four-parameter drift-diffusion computational model in which MSN ensemble activity represented the accumulation of temporal evidence. This model predicted that disrupting either D2-MSNs or D1-MSNs would increase interval timing response times and alter MSN firing. In line with this prediction, we found that optogenetic inhibition or pharmacological disruption of either D2-MSNs or D1-MSNs increased interval timing response times. Pharmacologically disrupting D2-MSNs or D1-MSNs also changed MSN dynamics and degraded trial-by-trial temporal decoding. Together, our findings demonstrate that D2-MSNs and D1-MSNs had opposing dynamics yet played complementary cognitive roles, implying that striatal direct and indirect pathways work together to shape temporal control of action. These data provide novel insight into basal ganglia cognitive operations beyond movement and have implications for human striatal diseases and therapies targeting striatal pathways.