Dopamine enhances model-free credit assignment through boosting of retrospective model-based inference

  1. Lorenz Deserno  Is a corresponding author
  2. Rani Moran  Is a corresponding author
  3. Jochen Michely
  4. Ying Lee
  5. Peter Dayan
  6. Raymond J Dolan
  1. University of Würzburg, Germany
  2. University College London, United Kingdom
  3. Charité - Universitätsmedizin Berlin, Germany
  4. TU Dresden, Germany
  5. Max Planck Institute for Biological Cybernetics, Germany

Abstract

Dopamine is implicated in representing model-free (MF) reward prediction errors a as well as influencing model-based (MB) credit assignment and choice. Putative cooperative interactions between MB and MF systems include a guidance of MF credit assignment by MB inference. Here, we used a double-blind, placebo-controlled, within-subjects design to test an hypothesis that enhancing dopamine levels boosts the guidance of MF credit assignment by MB inference. In line with this, we found that levodopa enhanced guidance of MF credit assignment by MB inference, without impacting MF and MB influences directly. This drug effect correlated negatively with a dopamine-dependent change in purely MB credit assignment, possibly reflecting a trade-off between these two MB components of behavioural control. Our findings of a dopamine boost in MB inference guidance of MF learning highlights a novel DA influence on MB-MF cooperative interactions.

Data availability

Necessary source data files are openly available at: https://osf.io/4dfkv/.

Article and author information

Author details

  1. Lorenz Deserno

    Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University of Würzburg, Würzburg, Germany
    For correspondence
    deserno_l@ukw.de
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7392-5280
  2. Rani Moran

    Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom
    For correspondence
    rani.moran@gmail.com
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7641-2402
  3. Jochen Michely

    Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3072-2330
  4. Ying Lee

    Psychiatry and Psychology, TU Dresden, Dresden, Germany
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9491-4919
  5. Peter Dayan

    Max Planck Ring 8, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3476-1839
  6. Raymond J Dolan

    The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9356-761X

Funding

Wellcome Trust (098362/Z/12/Z)

  • Raymond J Dolan

Max-Planck-Gesellschaft (Open-access funding)

  • Lorenz Deserno
  • Rani Moran
  • Peter Dayan
  • Raymond J Dolan

Deutsche Forschungsgemeinschaft (402170461)

  • Lorenz Deserno
  • Raymond J Dolan

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: The study was approved by the University College London Research Ethics Committee (Project ID 11285/001). Subjects gave written informed consent before the experiment.

Reviewing Editor

  1. Thorsten Kahnt, Northwestern University, United States

Publication history

  1. Preprint posted: January 17, 2021 (view preprint)
  2. Received: February 22, 2021
  3. Accepted: December 8, 2021
  4. Accepted Manuscript published: December 9, 2021 (version 1)
  5. Accepted Manuscript updated: December 20, 2021 (version 2)
  6. Version of Record published: January 13, 2022 (version 3)

Copyright

© 2021, Deserno et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 918
    Page views
  • 179
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Lorenz Deserno
  2. Rani Moran
  3. Jochen Michely
  4. Ying Lee
  5. Peter Dayan
  6. Raymond J Dolan
(2021)
Dopamine enhances model-free credit assignment through boosting of retrospective model-based inference
eLife 10:e67778.
https://doi.org/10.7554/eLife.67778

Further reading

    1. Neuroscience
    Nahoko Kuga et al.
    Research Article

    The medial prefrontal cortex and amygdala are involved in the regulation of social behavior and associated with psychiatric diseases but their detailed neurophysiological mechanisms at a network level remain unclear. We recorded local field potentials (LFPs) from the dorsal medial prefrontal cortex (dmPFC) and basolateral amygdala (BLA) while male mice engaged on social behavior. We found that in wild-type mice, both the dmPFC and BLA increased 4–7 Hz oscillation power and decreased 30–60 Hz power when they needed to attend to another target mouse. In mouse models with reduced social interactions, dmPFC 4–7 Hz power further increased especially when they exhibited social avoidance behavior. In contrast, dmPFC and BLA decreased 4–7 Hz power when wild-type mice socially approached a target mouse. Frequency-specific optogenetic manipulations replicating social approach-related LFP patterns restored social interaction behavior in socially deficient mice. These results demonstrate a neurophysiological substrate of the prefrontal cortex and amygdala related to social behavior and provide a unified pathophysiological understanding of neuronal population dynamics underlying social behavioral deficits.

    1. Neuroscience
    Nataliia Kozhemiako et al.
    Research Article

    Motivated by the potential of objective neurophysiological markers to index thalamocortical function in patients with severe psychiatric illnesses, we comprehensively characterized key non-rapid eye movement (NREM) sleep parameters across multiple domains, their interdependencies, and their relationship to waking event-related potentials and symptom severity. In 72 schizophrenia (SCZ) patients and 58 controls, we confirmed a marked reduction in sleep spindle density in SCZ and extended these findings to show that fast and slow spindle properties were largely uncorrelated. We also describe a novel measure of slow oscillation and spindle interaction that was attenuated in SCZ. The main sleep findings were replicated in a demographically distinct sample, and a joint model, based on multiple NREM components, statistically predicted disease status in the replication cohort. Although also altered in patients, auditory event-related potentials elicited during wake were unrelated to NREM metrics. Consistent with a growing literature implicating thalamocortical dysfunction in SCZ, our characterization identifies independent NREM and wake EEG biomarkers that may index distinct aspects of SCZ pathophysiology and point to multiple neural mechanisms underlying disease heterogeneity. This study lays the groundwork for evaluating these neurophysiological markers, individually or in combination, to guide efforts at treatment and prevention as well as identifying individuals most likely to benefit from specific interventions.