Humans perseverate on punishment avoidance goals in multigoal reinforcement learning
- The Hebrew University of Jerusalem, Israel
- University College London, United Kingdom
Abstract
Managing multiple goals is essential to adaptation, yet we are only beginning to understand computations by which we navigate the resource-demands entailed in so doing. Here, we sought to elucidate how humans balance reward seeking and punishment avoidance goals, and relate this to variation in its expression within anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). We constructed a computational model of multigoal pursuit to quantify the degree to which participants could disengage from the pursuit goals when instructed to, as well as devote less model-based resources towards goals that were less abundant. In general, participants (n=192) were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, participants often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. In a similar vein, participants showed no significant downregulation of avoidance when punishment avoidance goals were less abundant in the task. Importantly, we show preliminary evidence that individuals with chronic worry may have difficulty disengaging from punishment avoidance when instructed to seek reward. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward. Future studies should test in larger samples whether a difficulty to disengage from punishment avoidance contributes to chronic worry.
Data availability
All data are available in the main text or the supplementary materials. All code and analyses can be found at: github.com/pq1289/multigoal_RL
Article and author information
Author details
Funding
Fulbright Association (PS00318453)
- Paul B Sharp
NIH Blueprint for Neuroscience Research (R01MH124092)
- Eran Eldar
Wellcome Trust (098362/Z/12/Z)
- Paul B Sharp
2Max Planck UCL Centre for Computational Psychiatry and Ageing Research (Open Access Funding)
- Paul B Sharp
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Ethics
Human subjects: Participants gave written informed consent before taking part in the study, which was approved by the university's ethics review board (project ID number 16639/001).
Copyright
© 2022, Sharp et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 2,389
- views
-
- 373
- downloads
-
- 7
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Humans perseverate on punishment avoidance goals in multigoal reinforcement learning
eLife 11:e74402.
https://doi.org/10.7554/eLife.74402
Further reading
-
- Neuroscience
What determines where to move the eyes? We recently showed that pupil size, a well-established marker of effort, also reflects the effort associated with making a saccade (‘saccade costs’). Here, we demonstrate saccade costs to critically drive saccade selection: when choosing between any two saccade directions, the least costly direction was consistently preferred. Strikingly, this principle even held during search in natural scenes in two additional experiments. When increasing cognitive demand experimentally through an auditory counting task, participants made fewer saccades and especially cut costly directions. This suggests that the eye-movement system and other cognitive operations consume similar resources that are flexibly allocated among each other as cognitive demand changes. Together, we argue that eye-movement behavior is tuned to adaptively minimize saccade-inherent effort.
