Normative decision rules in changing environments
Abstract
Models based on normative principles have played a major role in our understanding of how the brain forms decisions. However, these models have typically been derived for simple, stable conditions, and their relevance to decisions formed under more naturalistic, dynamic conditions is unclear. We previously derived a normative decision model in which evidence accumulation is adapted to fluctuations in the evidence-generating process that occur during a single decision (Glaze et al., 2015), but the evolution of commitment rules (e.g., thresholds on the accumulated evidence) under dynamic conditions is not fully understood. Here we derive a normative model for decisions based on changing contexts, which we define as changes in evidence quality or reward, over the course of a single decision. In these cases, performance (reward rate) is maximized using decision thresholds that respond to and even anticipate these changes, in contrast to the static thresholds used in many decision models. We show that these adaptive thresholds exhibit several distinct temporal motifs that depend on the specific predicted and experienced context changes and that adaptive models perform robustly even when implemented imperfectly (noisily). We further show that decision models with adaptive thresholds outperform those with constant or urgency-gated thresholds in accounting for human response times on a task with time-varying evidence quality and average reward. These results further link normative and neural decision-making while expanding our view of both as dynamic, adaptive processes that update and use expectations to govern both deliberation and commitment.
Data availability
MATLAB code used to generate all results and figures is available at https://github.com/nwbarendregt/AdaptNormThresh.
Article and author information
Author details
Funding
National Institutes of Health (R01-MH-115557)
- Nicholas W Barendregt
- Joshua I Gold
- Krešimir Josić
- Zachary P Kilpatrick
National Institutes of Health (R01-EB029847-01)
- Nicholas W Barendregt
- Zachary P Kilpatrick
National Science Foundation (NSF-DMS-1853630)
- Nicholas W Barendregt
- Zachary P Kilpatrick
National Science Foundation (NSF-DBI-1707400)
- Krešimir Josić
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Copyright
© 2022, Barendregt et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 1,227
- views
-
- 206
- downloads
-
- 3
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.