Dynamic decision policy reconfiguration under outcome uncertainty
- Carnegie Mellon University, United States
- Northwestern University, United States
- University of Pittsburgh, United States
Abstract
In uncertain or unstable environments, sometimes the best decision is to change your mind. To shed light on this flexibility, we evaluated how the underlying decision policy adapts when the most rewarding action changes. Human participants performed a dynamic two-armed bandit task that manipulated the certainty in relative reward (conflict) and the reliability of action-outcomes (volatility). Continuous estimates of conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height), respectively. At the trialwise level, following a switch in the optimal choice, the drift rate plummets and the boundary height weakly spikes, leading to a slow exploratory state. We find that the drift rate drives most of this response, with an unreliable contribution of boundary height across experiments. Surprisingly, we find no evidence that pupillary responses associated with decision policy changes. We conclude that humans show a stereotypical shift in their decision policies in response to environmental changes.
Data availability
Behavioral data and their computational derivatives are available at https://github.com/kmbond/dynamic_decision_policy_reconfiguration. Code used to generate figures can be found here: https://github.com/kmbond/dynamic_decision_policy_reconfiguration/tree/master/revised_figure_nbs.Raw pupillometry data (DOI: 10.1184/R1/13543133), the features of the task-evoked pupillometry response (DOI: 10.1184/R1/13543067), and the principal components calculated from those features (DOI: 10.1184/R1/13543160) are available at https://kilthub.cmu.edu/projects/Dynamic_decision_policy_reconfiguration_under_outcome_uncertainty/96116.
Article and author information
Author details
Funding
Air Force Research Laboratory (FA9550-18-1-0251)
- Krista Bond
- Timothy Verstynen
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Ethics
Human subjects: Neurologically healthy adults were recruited from thelocal university population. All procedures were approved by the Carnegie Mellon University Institutional Review Board (Approval Code: 2018_00000195; Funding: Air Force Research Laboratory, Grant Office ID: 180119). All research participants provided informed consent to participate in the study and consent to publish any research findings based on their provided data.
Copyright
© 2021, Bond et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 974
- views
-
- 175
- downloads
-
- 7
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Dynamic decision policy reconfiguration under outcome uncertainty
eLife 10:e65540.
https://doi.org/10.7554/eLife.65540
Further reading
-
- Neuroscience
When retrieved, seemingly stable memories can become sensitive to significant events, such as acute stress. The mechanisms underlying these memory dynamics remain poorly understood. Here, we show that noradrenergic stimulation after memory retrieval impairs subsequent remembering, depending on hippocampal and cortical signals emerging during retrieval. In a three-day study, we measured brain activity using fMRI during initial encoding, 24 hr-delayed memory cueing followed by pharmacological elevations of glucocorticoid or noradrenergic activity, and final recall. While post-retrieval glucocorticoids did not affect subsequent memory, the impairing effect of noradrenergic arousal on final recall depended on hippocampal reactivation and category-level reinstatement in the ventral temporal cortex during memory cueing. These effects did not require a reactivation of the original memory trace and did not interact with offline reinstatement during rest. Our findings demonstrate that, depending on the retrieval-related neural reactivation of memories, noradrenergic arousal after retrieval can alter the future accessibility of consolidated memories.
-
- Neuroscience
Insulin plays a key role in metabolic homeostasis. Drosophila insulin-producing cells (IPCs) are functional analogues of mammalian pancreatic beta cells and release insulin directly into circulation. To investigate the in vivo dynamics of IPC activity, we quantified the effects of nutritional and internal state changes on IPCs using electrophysiological recordings. We found that the nutritional state strongly modulates IPC activity. IPC activity decreased with increasing periods of starvation. Refeeding flies with glucose or fructose, two nutritive sugars, significantly increased IPC activity, whereas non-nutritive sugars had no effect. In contrast to feeding, glucose perfusion did not affect IPC activity. This was reminiscent of the mammalian incretin effect, where glucose ingestion drives higher insulin release than intravenous application. Contrary to IPCs, Diuretic hormone 44-expressing neurons in the pars intercerebralis (DH44PINs) responded to glucose perfusion. Functional connectivity experiments demonstrated that these DH44PINs do not affect IPC activity, while other DH44Ns inhibit them. Hence, populations of autonomously and systemically sugar-sensing neurons work in parallel to maintain metabolic homeostasis. Accordingly, activating IPCs had a small, satiety-like effect on food-searching behavior and reduced starvation-induced hyperactivity, whereas activating DH44Ns strongly increased hyperactivity. Taken together, we demonstrate that IPCs and DH44Ns are an integral part of a modulatory network that orchestrates glucose homeostasis and adaptive behavior in response to shifts in the metabolic state.
