Pallidal neuromodulation of the explore/exploit trade-off in decision-making

  1. Ana Luisa de A. Marcelino
  2. Owen Gray
  3. Bassam Al-Fatly
  4. William Gilmour
  5. J Douglas Steele
  6. Andrea A Kühn
  7. Tom Gilbertson  Is a corresponding author
  1. Charité - Universitätsmedizin Berlin, Germany
  2. University of Dundee, United Kingdom

Abstract

Every decision that we make involves a conflict between exploiting our current knowledge of an action's value or exploring alternative courses of action that might lead to a better, or worse outcome. The sub-cortical nuclei that make up the basal ganglia have been proposed as a neural circuit that may contribute to resolving this explore-exploit 'dilemma'. To test this hypothesis, we examined the effects of neuromodulating the basal ganglia's output nucleus, the globus pallidus interna, in patients who had undergone deep brain stimulation (DBS) for isolated dystonia. Neuromodulation enhanced the number of exploratory choices to the lower value option in a 2-armed bandit probabilistic reversal-learning task. Enhanced exploration was explained by a reduction in the rate of evidence accumulation (drift rate) in a reinforcement learning drift diffusion model. We estimated the functional connectivity profile between the stimulating DBS electrode and the rest of the brain using a normative functional connectome derived from heathy controls. Variation in the extent of neuromodulation induced exploration between patients was associated with functional connectivity from the stimulation electrode site to a distributed brain functional network. We conclude that the basal ganglia's output nucleus, the globus pallidus interna, can adaptively modify decision choice when faced with the dilemma to explore or exploit.

Data availability

Raw choice and reaction time data, computational model parameter estimates, simulated data and r-maps from connectivity analysis are available via the Open Science Framework https://osf.io/fs36g/

The following data sets were generated

Article and author information

Author details

  1. Ana Luisa de A. Marcelino

    Department of Neurology, Charité - Universitätsmedizin Berlin, Berlin, Germany
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3291-7222
  2. Owen Gray

    Division of Imaging Science and Technology, University of Dundee, Dundee, United Kingdom
    Competing interests
    No competing interests declared.
  3. Bassam Al-Fatly

    Department of Neurology, Charité - Universitätsmedizin Berlin, Berlin, Germany
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0067-6177
  4. William Gilmour

    Division of Imaging Science and Technology, University of Dundee, Dundee, United Kingdom
    Competing interests
    No competing interests declared.
  5. J Douglas Steele

    Division of Imaging Science and Technology, University of Dundee, Dundee, United Kingdom
    Competing interests
    No competing interests declared.
  6. Andrea A Kühn

    Department of Neurology, Charité - Universitätsmedizin Berlin, Berlin, Germany
    Competing interests
    Andrea A Kühn, has received from honoraria from Boston Scientific, Medtronic and Teva..
  7. Tom Gilbertson

    Division of Imaging Science and Technology, University of Dundee, Dundee, United Kingdom
    For correspondence
    tgilbertson@dundee.ac.uk
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-9866-1565

Funding

Chief Scientist Office

  • Tom Gilbertson

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: The which was approved by the local ethics committee (Charité - Universitätsmedizin Berlin, EA1/179/20).

Reviewing Editor

  1. Birte U Forstmann, University of Amsterdam, Netherlands

Publication history

  1. Received: April 21, 2022
  2. Preprint posted: April 22, 2022 (view preprint)
  3. Accepted: February 1, 2023
  4. Accepted Manuscript published: February 2, 2023 (version 1)
  5. Version of Record published: February 20, 2023 (version 2)

Copyright

© 2023, de A. Marcelino et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 732
    Page views
  • 120
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Ana Luisa de A. Marcelino
  2. Owen Gray
  3. Bassam Al-Fatly
  4. William Gilmour
  5. J Douglas Steele
  6. Andrea A Kühn
  7. Tom Gilbertson
(2023)
Pallidal neuromodulation of the explore/exploit trade-off in decision-making
eLife 12:e79642.
https://doi.org/10.7554/eLife.79642

Further reading

    1. Neuroscience
    Xiaosha Wang, Bijun Wang, Yanchao Bi
    Research Article Updated

    One signature of the human brain is its ability to derive knowledge from language inputs, in addition to nonlinguistic sensory channels such as vision and touch. How does human language experience modulate the mechanism by which semantic knowledge is stored in the human brain? We investigated this question using a unique human model with varying amounts and qualities of early language exposure: early deaf adults who were born to hearing parents and had reduced early exposure and delayed acquisition of any natural human language (speech or sign), with early deaf adults who acquired sign language from birth as the control group that matches on nonlinguistic sensory experiences. Neural responses in a semantic judgment task with 90 written words that were familiar to both groups were measured using fMRI. The deaf group with reduced early language exposure, compared with the deaf control group, showed reduced semantic sensitivity, in both multivariate pattern (semantic structure encoding) and univariate (abstractness effect) analyses, in the left dorsal anterior temporal lobe (dATL). These results provide positive, causal evidence that language experience drives the neural semantic representation in the dATL, highlighting the roles of language in forming human neural semantic structures beyond nonverbal sensory experiences.

    1. Neuroscience
    Ayako Yamaguchi, Manon Peltier
    Research Article Updated

    Across phyla, males often produce species-specific vocalizations to attract females. Although understanding the neural mechanisms underlying behavior has been challenging in vertebrates, we previously identified two anatomically distinct central pattern generators (CPGs) that drive the fast and slow clicks of male Xenopus laevis, using an ex vivo preparation that produces fictive vocalizations. Here, we extended this approach to four additional species, X. amieti, X. cliivi, X. petersii, and X. tropicalis, by developing ex vivo brain preparation from which fictive vocalizations are elicited in response to a chemical or electrical stimulus. We found that even though the courtship calls are species-specific, the CPGs used to generate clicks are conserved across species. The fast CPGs, which critically rely on reciprocal connections between the parabrachial nucleus and the nucleus ambiguus, are conserved among fast-click species, and slow CPGs are shared among slow-click species. In addition, our results suggest that testosterone plays a role in organizing fast CPGs in fast-click species, but not in slow-click species. Moreover, fast CPGs are not inherited by all species but monopolized by fast-click species. The results suggest that species-specific calls of the genus Xenopus have evolved by utilizing conserved slow and/or fast CPGs inherited by each species.