Decision Making: Remembering to choose the future
From the philosophers of ancient Greece to the self-help books of today, humans have long been interested in choice. Philosophers and ethicists have debated what goals we ought to choose for millennia, and for a century or more economists and psychologists have studied what goals we will choose. However, neuroscience has only recently begun to systematically address how we choose.
Whether we are pondering life-defining decisions about love, career or commitment to a cause, or simply picking which snacks to buy in the grocery store, it is still unclear what regions of the brain are involved in making choices, and what information those regions encode. In everyday language, we often talk about ‘value’ (or in economic terms, ‘utility’) as the driver of such decisions: we consider our options, and select the one with the highest value. Hundreds of functional MRI (or fMRI) studies in healthy humans have identified a consistent set of brain regions which seem to process signals associated with subjective values; this suggests that value is indeed a concept that has biological roots (Bartra et al., 2013). However, the nature of the information that contributes to the neural signals related to value remains a matter of debate (O'Doherty, 2014). In other words, it is not clear what we think about when we think about value.
In fact, scientists know far less about choices based on value than they do about perceptual decisions (such as assessing if a noisy array of moving dots is trending more to the left or to the right; Shadlen and Kiani, 2013). During perceptual choices, external information is repeatedly sampled and the neural representation of this evidence accumulates until a threshold is crossed and a decision is triggered. These tasks are associated with well-known behavioral phenomena – for instance, choices with less perceptual evidence take longer to resolve – which are captured by drift diffusion models (Ratcliff and McKoon, 2008).
It has been proposed that value-based decisions might occur in a similar way (Rangel et al., 2008). However, while it is obvious what knowledge is accumulating as a person gazes at a screen filled with moving dots, it is less clear what information might be sampled to support a decision based on value. Now, in eLife, Akram Bakkour of Columbia University and colleagues report that, at least in part, we may be thinking about past experiences (Bakkour et al., 2019).
Their work makes a strong case that value-based deliberation engages the hippocampus, a small structure within the brain that is involved in long-term memory. Although past experiences are a likely source of relevant information in value-based decisions, to date researchers have focused mostly on other regions of the brain such as the ventral prefrontal cortex and the striatum.
Bakkour et al. – who are based at Columbia and the Memory Disorders Research Center – first used fMRI to establish that activity in the hippocampus is greater for longer deliberations during value-based choice. They then harnessed the power of a lesion experiment to infer that the structure is necessary for such choices (Vaidya et al., 2019). Patients with hippocampal damage were slower to make decisions, and somewhat more variable in what they chose. These hippocampal effects were specific to value-based decisions. Deliberation time in a classic perceptual decision task did not relate to hippocampal signal, nor was it influenced by hippocampal damage. While perceptual decisions involve sampling external evidence, Bakkour et al. propose that deliberation during value-based choice requires sampling internal evidence. This includes – although is presumably not limited to – using the hippocampus to conjure up past experiences with similar options. Ultimately, these results will help to broaden the anatomical scope of decision neuroscience.
Studies have already shown that ‘attention’, while intuitive and attractive as a holistic concept, is in fact composed of dozens of distinct processes with definable characteristics that rely on different neural circuits. It is likely that ‘value’ will also require further decomposition. Armed with this knowledge, it may become possible to better understand how the brain carries out the important value-based decisions that define us as individuals and shape the directions of our societies.
The problem with valueNeuroscience & Biobehavioral Reviews 43:259–268.https://doi.org/10.1016/j.neubiorev.2014.03.027
A framework for studying the neurobiology of value-based decision makingNature Reviews Neuroscience 9:545–556.https://doi.org/10.1038/nrn2357
Lesion studies in contemporary neuroscienceTrends in Cognitive Sciences 23:653–671.https://doi.org/10.1016/j.tics.2019.05.009
Article and author information
- Version of Record published: August 14, 2019 (version 1)
© 2019, Fellows
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
- Page views
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
The basal ganglia (BG) contribute to reinforcement learning (RL) and decision making, but unlike artificial RL agents, it relies on complex circuitry and dynamic dopamine modulaton of opponent striatal pathways to do so. We develop the OpAL* model to assess the normative advantages of this circuitry. In OpAL*, learning induces opponent pathways to differentially emphasize the history of positive or negative outcomes for each action. Dynamic DA modulation then amplifies the pathway most tuned for the task environment. This efficient coding mechanism avoids a vexing explore-exploit tradeoff that plagues traditional RL models in sparse reward environments. OpAL* exhibits robust advantages over alternative models, particularly in environments with sparse reward and large action spaces. These advantages depend on opponent and nonlinear Hebbian plasticity mechanisms previously thought to be pathological. Finally, OpAL* captures risky choice patterns arising from DA and environmental manipulations across species, suggesting that they result from a normative biological mechanism.
Animals can continuously learn different tasks to adapt to changing environments and, therefore, have strategies to effectively cope with inter-task interference, including both proactive interference (Pro-I) and retroactive interference (Retro-I). Many biological mechanisms are known to contribute to learning, memory, and forgetting for a single task, however, mechanisms involved only when learning sequential different tasks are relatively poorly understood. Here, we dissect the respective molecular mechanisms of Pro-I and Retro-I between two consecutive associative learning tasks in Drosophila. Pro-I is more sensitive to an inter-task interval (ITI) than Retro-I. They occur together at short ITI (<20 min), while only Retro-I remains significant at ITI beyond 20 min. Acutely overexpressing Corkscrew (CSW), an evolutionarily conserved protein tyrosine phosphatase SHP2, in mushroom body (MB) neurons reduces Pro-I, whereas acute knockdown of CSW exacerbates Pro-I. Such function of CSW is further found to rely on the γ subset of MB neurons and the downstream Raf/MAPK pathway. In contrast, manipulating CSW does not affect Retro-I as well as a single learning task. Interestingly, manipulation of Rac1, a molecule that regulates Retro-I, does not affect Pro-I. Thus, our findings suggest that learning different tasks consecutively triggers distinct molecular mechanisms to tune proactive and retroactive interference.