A process model account of the role of dopamine in intertemporal choice

Many decisions require trading-off potential benefits (rewards) against the costs of actions, such as the time one has to wait for reward to occur (Soutschek and Tobler, 2018). The neurotransmitter dopamine is thought to play a central role in such cost-benefit trade-offs by increasing the tolerance for action costs in order to maximize net subjective benefits (Beeler, 2012; Robbins and Everitt, 1992; Salamone and Correa, 2012; Schultz, 2015). Tonic dopaminergic activity was hypothesized to implement a ‘cost control’ which moderates whether a reward or goal is considered to be worth its costs (Beeler and Mourra, 2018). Prominent accounts of dopaminergic functioning thus predict that dopamine should strengthen the preference for costly larger-later (LL) over less costly smaller-sooner (SS) rewards. However, empirical studies modulating dopaminergic neurotransmission during intertemporal decision making provided inconsistent evidence for these hypotheses (for a review, see Webber et al., 2021). Blocking dopaminergic activation even seems to increase rather than to reduce the preference for delayed outcomes (Arrondo et al., 2015; Soutschek et al., 2017; Wagner et al., 2020; Weber et al., 2016), in apparent contrast to accounts proposing that lower dopaminergic activity should decrease the attractiveness of costly rewards (Beeler and Mourra, 2018; Robbins and Everitt, 1992; Salamone and Correa, 2012). Thus, the link between dopamine and cost-benefit weighting in intertemporal choice remains elusive. Yet, a plausible account of how dopamine affects cost-benefit weighting is important given that deficits in delay of gratification belong to the core symptoms of several psychiatric disorders and that dopaminergic medication plays a central role in the treatment of these and other disorders (Hasler, 2012; MacKillop et al., 2011).

To account for the conflicting findings on the role of dopamine in intertemporal choice, recent proximity accounts hypothesized that dopamine – in addition to strengthening the pursuit of valuable goals – also increases the preference for proximate over distant rewards (first formulated by Westbrook and Frank, 2018; see also Soutschek et al., 2022). While proximity and action costs often correlate negatively (as cost-free immediate rewards are typically more proximate than costly delayed rewards), they can conceptually be distinguished: perceived costs depend on an individual’s internal state (e.g., available resources to wait for future rewards), whereas proximity is determined by situational factors like familiarity or concreteness (Westbrook and Frank, 2018). The hypothesis that dopamine increases the proximity advantage of sooner over later rewards is consistent with the observed stronger preference for LL options after D2R blockade, which could not be explained by standard accounts of the role of dopamine in cost-benefit decisions (Beeler and Mourra, 2018; Salamone and Correa, 2012).

Still, the question remains as to how the proximity account can be reconciled with the large body of evidence for a motivating role of dopamine in other domains than intertemporal choice (Webber et al., 2021). We recently suggested that both accounts may be unified within the framework of computational process models like the drift diffusion model (DDM) (Soutschek et al., 2022). DDMs assume that decision makers accumulate evidence for two reward options until a decision boundary is reached. The dopamine-mediated cost control may be implemented via dopaminergic effects on the evaluation of reward magnitudes and delay costs during the evidence accumulation process (drift rate), while a proximity advantage for sooner over delayed rewards may shift the starting bias toward the decision boundary for sooner rewards (Soutschek et al., 2022; Westbrook and Frank, 2018). Such proximity effects on the starting bias could reflect an automatic bias toward immediate rewards as posited by dual process models of intertemporal choice (Figner et al., 2010; McClure et al., 2004), whereas the influence of reward and delay on the drift rate involves more controlled and attention-demanding weighting of costs and benefits. Combining these two, in their consequences on overt choices partially opposing, but independent, effects of dopamine in a unified and tractable account could reconcile conflicting findings. In turn, such a process account might provide a knowledge basis to advance our understanding of the neurochemical basis of the decision-making deficits in clinical disorders and improve the effectiveness of pharmaceutical interventions.

Here, we tested central assumptions of the proposed account by re-analyzing the data from two previous studies that investigated how the dopamine D2 receptor antagonist amisulpride and the D1 agonist PF-06412562 impact cost-benefit weighting in intertemporal choice (Soutschek et al., 2017; Soutschek et al., 2020a). D1Rs are prevalent in the direct ‘Go’ pathway and facilitate action selection via mediating the impact of phasic bursts elicited by high above-average rewards (Evers et al., 2017; Kirschner et al., 2020). D2Rs, in contrast, dominate the indirect ‘Nogo’ pathway (which suppresses action) and are more sensitive to small concentration differences in tonic dopamine levels (Missale et al., 1998), which is thought to encode the background, average reward rate (Kirschner et al., 2020; Volkow and Baler, 2015; Westbrook and Frank, 2018; Westbrook et al., 2020). Comparing the influences of the two compounds on the choice process during intertemporal decisions allowed us to test the hypothesized dissociable roles of D1Rs and D2Rs for decision making. Previously reported analyses of these data had shown no influence of D2R blockade or D1R stimulation on the mean preferences for LL over SS options (Soutschek et al., 2017; Soutschek et al., 2020a). However, they had not asked whether the pharmacological agents moderate the influences of reward magnitudes and delay costs on subcomponents of the decision process within the framework of a DDM. We re-analyzed the data sets with hierarchical Bayesian drift diffusion modeling to test central assumptions of the proposed account on dopamine’s role in cost-benefit weighting. First, if D2R activation implements a cost threshold moderating the evaluation of whether a reward is worth the action costs, then blocking D2R activation with amisulpride should increase the influence of reward magnitude on the speed of evidence accumulation, with costly small rewards becoming less acceptable than under placebo. Second, if D2R-mediated tonic dopaminergic activity also moderates the impact of proximity on choices (which affects the starting bias rather than the speed of the evidence accumulation process), D2R blockade should attenuate the effects of waiting costs on the starting bias. Third, we expected D1R stimulation to modulate the sensitivity to rewards during evidence accumulation (via increasing activity in the direct ‘Go’ pathway), without affecting proximity costs which were related to tonic rather than phasic dopaminergic activity (Westbrook and Frank, 2018).

To disentangle how dopamine contributes to distinct subcomponents of the choice process, we re-analyzed a previously published data set where 56 participants had performed an intertemporal choice task under the D2 antagonist amisulpride (400 mg) and placebo in two separate sessions (Soutschek et al., 2017; Figure 1). First, we assessed amisulpride effects on intertemporal choices with conventional model-based and model-free analyses, as they are employed by other pharmacological studies on cost-benefit weighting. Hyperbolic discounting of future rewards was not significantly different under amisulpride (mean log-k=–2.07) compared with placebo (mean log-k=–2.19), Bayesian t-test, HDI_mean = 0.21, HDI_95% = [–0.28; 0.70], and there were also no drug effects on choice consistency (inverse temperature), HDI_mean = –0.28, HDI_95% = [–0.71; 0.13]. Model-free Bayesian mixed generalized linear models (MGLMs) revealed a stronger preference for LL over SS options with increasing differences in reward magnitudes, HDI_mean = 6.32, HDI_95% = [5.03; 7.83], and with decreasing differences in delay of reward delivery, HDI_mean = –1.27, HDI_95% = [–1.87; –0.60]. The impact of delays on choices was significantly reduced under amisulpride compared with placebo, HDI_mean = 0.75, HDI_95% = [0.02; 1.67] (Figure 1C/D and Table 1). When we explored whether dopaminergic effects changed over the course of the experiment, we observed a significant main effect of trial number (more LL choices over time), HDI_mean = 0.58, HDI_95% = [0.19; 0.99]. However, this effect was unaffected by the pharmacological manipulation, HDI_mean = –0.06, HDI_95% = [–0.61; 0.48]. We also re-computed the MGLM reported above on log-transformed decision times, adding predictors for choice (SS vs. LL option) and Magnitude_sum (combined magnitudes of SS and LL rewards). Participants made faster decisions the higher the sum of the two rewards, HDI_mean = –0.12, HDI_95% = [–0.18; –0.06], however we observed no significant drug effects on decision times. Thus, based on these conventional analyses one would conclude that reduction of D2R neurotransmission lowers the sensitivity to delay costs, which on the one hand agrees with one line of previous findings (Arrondo et al., 2015; Wagner et al., 2020; Weber et al., 2016). On the other hand, this result seems to contradict the widely held assumption that dopamine increases the preference for costly over cost-free outcomes (Beeler and Mourra, 2018; Webber et al., 2021; Westbrook et al., 2020), because according to this view lower dopaminergic activity should increase, rather than decrease, the impact of waiting costs on LL choices. However, analyses that consider only the observed choices do not allow disentangling dopaminergic influences on distinct subcomponents of the choice process.

Figure 1

Download asset Open asset

DDMs paint a fuller picture of the decision process than pure choice data by integrating information from observed choices and decision times. DDMs assume that agents accumulate evidence for the choice options (captured by the drift parameter v) from a starting point ζ until the accumulated evidence reaches a decision threshold (boundary parameter a; Figure 1E). Following previous procedures analyzing intertemporal choices with DDMs (Amasino et al., 2019), we assumed that the drift rate ν integrates reward magnitudes and delays of choice options via attribute-wise comparisons (DDM-1). In addition, we also allowed the starting bias to vary as a function of differences in delay costs, in line with recent proximity accounts of dopamine (Westbrook and Frank, 2018).

A sanity check revealed that larger differences between the reward magnitudes of the LL and SS options bias evidence accumulation toward the LL option, HDI_mean = 2.41, HDI_95% = [1.93; 2.95], whereas larger differences in delays bias accumulation in favor of the SS option, HDI_mean = –1.13, HDI_95% = [–1.53; –0.78]. Moreover, we assessed the relationship between the difference in DDM parameters (reward magnitude – delay) and hyperbolic discount parameters log-k as purely choice-based indicator of impulsiveness. Replicating previous findings, we found that across individuals the weights relate to delay discounting, r=–0.61, p<0.001 (Amasino et al., 2019; Figure 1F), such that individuals weighting reward magnitudes more strongly over delays make more patient choices. Thus, our model parameters capture essential subprocesses of intertemporal decision making.

Next, we tested the impact of our dopaminergic manipulation on evidence accumulation: D2R blockade strengthened the impact of differences in reward magnitude on evidence accumulation, Drug×Magnitude_diff: HDI_mean = 0.81, HDI_95% = [0.04; 1.71], while the contribution of differences in delay costs remained unchanged, Drug×Delay_diff: HDI_mean = –0.30, HDI_95% = [–0.85; 0.20] (Figure 2A–F and Table 2). The drug-induced increase in sensitivity to variation in reward magnitude suggests that low rewards are considered less valuable under amisulpride compared with placebo (Figure 2C). This finding is consistent with the cost control hypothesis (Beeler and Mourra, 2018) according to which low dopamine levels reduce the attractiveness of smaller, below-average rewards.

Figure 2

Download asset Open asset

When we assessed dopaminergic effects on the starting bias, we observed that under placebo increasing differences in delay shifted the starting point toward the SS option, HDI_mean = 0.81, HDI_95% = [0.04; 1.71], suggesting that the bias parameter is closer to the proximate (SS) option the stronger the proximity advantage of the SS over the LL option. Amisulpride shifted the starting bias toward the SS option for smaller differences in delay, main effect of Drug: HDI_mean = –0.04, HDI_95% = [–0.08; -0.001], but also attenuated the impact of delay, Drug×Delay_diff: HDI_mean = 0.02, HDI_95% = [0.001; 0.04]. Thus, dopamine appears to moderate the impact of temporal proximity on the starting bias (Figure 2G–K), providing support for recent proximity accounts of dopamine (Soutschek et al., 2022; Westbrook and Frank, 2018). Moreover, compared to the model-free analysis, our process model (which uses not only binary choice but also response time data) provides a fuller picture of the subcomponents of the choice process affected by the dopaminergic manipulation.

Next, we investigated the relation between the drug effects on the drift rate and on the starting bias. We found no evidence that the two effects correlated, r=0.07, p=0.60, suggesting that amisulpride effects on these subprocesses were largely independent of each other. Control analyses revealed no effects of amisulpride on non-decision times, HDI_mean = –0.10, HDI_95% = [–0.24; 0.03], or the decision threshold, HDI_mean = 0.17, HDI_95% = [–0.11; 0.46]. Thus, the results of DDM-1 suggest that dopamine moderates the influence of choice attributes on both the speed of evidence accumulation and on the starting bias, consistent with recent accounts (Soutschek et al., 2022; Westbrook and Frank, 2018) of dopamine’s role in cost-benefit weighting.

To test the robustness of our DDM findings, we computed further DDMs where we either removed the impact of Delay_diff on the starting bias (DDM-2) or the impact of Magnitude_diff and Delay_diff on the drift rate (DDM-3). In a further model (DDM-4), we explored whether the starting bias is affected by the overall proximity of the options (sum of delays, Delay_sum) rather than the difference in proximity (Delay_diff; see Table 3 for an overview over the parameters included in the various models). Importantly, our original DDM-1 (DIC = 9478) explained the data better than DDM-2 (DIC = 9481), DDM-3 (DIC = 10,224), or DDM-4 (DIC = 9492; Figure 3A). Nevertheless, amisulpride moderated the impact of Magnitude_diff on the drift rate also in DDM-2, HDI_mean = 0.86, HDI_95% = [0.18; 1.64], and DDM-4, HDI_mean = 0.83, HDI_95% = [0.04; 1.75], and amisulpride also lowered the impact of Delay_diff on the starting bias in DDM-3, HDI_mean = –0.02, HDI_95% = [–0.04; –0.001]. Thus, the dopaminergic effects on these subcomponents of the choice process are robust to the exact specification of the DDM.

Figure 3

Download asset Open asset

We compared the winning account also with alternative process models of intertemporal choice. While in DDM-1 the drift rate depends on separate comparisons between choice attributes, one might alternatively assume that they compare the discounted subjective reward values of both options (Wagner et al., 2020), as given by the hyperbolic discount functions. However, a DDM where the drift rate was modeled as the difference between the hyperbolically discounted reward values (with the discount factor as free parameter; DDM-5) showed a worse model fit (DIC = 10,720) than DDM-1. This replicates previous findings according to which intertemporal choices can better be explained by attribute-wise than by option-wise comparison strategies (Amasino et al., 2019; Dai and Busemeyer, 2014; Reeck et al., 2017).

Next, we investigated an alternative to the proposal that differences in delay affect the starting bias via proximity effects. Specifically, we tested whether evidence for delay costs are accumulated earlier than for reward magnitude (relative-starting-time (rs)DDM; Amasino et al., 2019; Lombardi and Hare, 2021). From the perspective of rsDDMs, evidence accumulation for delays would start after a shorter non-decision time than for rewards, which is expressed by the variable τ_diff (if τ_diff > 0, non-decision time is shorter for delays than rewards, and vice versa if τ_diff < 0). However, also this rsDDM (DDM-6) explained the data less well (DIC = 9,548) than DDM-1. Thus, DDM-1 explains the current data better than alternative DDMs.

The currently used dose of amisulpride (400 mg) is thought to have predominantly postsynaptic effects on D2Rs, while lower doses (50–300 mg) might show presynaptic rather than postsynaptic effects (Schoemaker et al., 1997). Given that we used the same dose in all participants, one might argue that we may have studied presynaptic effects in individuals with relatively high body mass (which lowers the effective dose). However, we observed no evidence that individual random coefficients for the drug effects on the drift rate or on the starting bias correlated with body weight, all r<0.22, all p>0.10. There were also no significant correlations between DDM parameters and performance in the digit span backward task as proxy for baseline dopamine synthesis capacity (Cools et al., 2008), all r<0.17, all p>0.22. There was thus no evidence that pharmacological effects on intertemporal choices depended on body weight as proxy of effective dose or working memory performance as proxy for baseline dopaminergic activity.

As further check of the explanatory adequacy of DDM-1, we performed posterior predictive checks and parameter recovery analyses. Plotting the observed RTs (split into quintiles according to Magnitude_diff and Delay_diff) against the simulated RTs based on the parameter estimates from the different DDMs suggests that the DDMs provide reasonable accounts of the observed data both on the group and the individual level, at least for DDMs 1–3 (Figure 3B/C and Figure 4). Moreover, the squared differences between observed and simulated RTs were smaller for DDM-1 (0.83) than for alternative DDMs (DDM-2: 0.85; DDM-3: 0.98; DDM-4: 1.05, DDM-5: 0.89; DDM-6: 1.63). To assess parameter recovery, we re-computed DDM-1 on 10 simulated data sets based on the original DDM-1 parameter estimates. All group-level parameters from the simulated data were within the 95% HDI of the original parameter estimates, except for the non-decision time τ (which suggests that our model tends to overestimate the duration of decision-unrelated processes). Nevertheless, all parameters determining the outcome of the decision process (i.e., the choice made) as well as the dopaminergic effects on the parameters could reliably be recovered by DDM-1.

Figure 4

Download asset Open asset

To assess the receptor specificity of our findings, we conducted the same analyses on the data from a study (published previously in Soutschek et al., 2020a) testing the impact of three doses of a D1 agonist (6, 15, 30 mg) relative to placebo on intertemporal choices (between-subject design). In the intertemporal choice task used in this experiment, the SS reward was always immediately available (delay = 0), contrary to the task in the D2 experiment where the delay of the SS reward varied from 0 to 30 days. Again, the data in the D1 experiment were best explained by DDM-1 (DIC_DDM-1=19,657) compared with all other DDMs (DIC_DDM-2=20,934; DIC_DDM-3=21,710; DIC_DDM-5=21,982; DIC_DDM-6=19,660; note that DDM-4 was identical with DDM-1 for the D1 agonist study because the delay of the SS reward was 0). Neither the best-fitting nor any other model yielded significant drug effects on any drift diffusion parameter (see Table 4 for the best-fitting model). Also model-free analyses conducted in the same way as for the D2 antagonist study revealed no significant drug effects (all HDI_95% included zero). There was thus no evidence for any influence of D1R stimulation on intertemporal decisions.

Dopamine is hypothesized to play a central role in human cost-benefit decision making, but existing empirical evidence does not conclusively support the widely shared assumption that dopamine promotes the pursuit of high benefit-high cost options (for reviews, see Soutschek et al., 2022; Webber et al., 2021). By manipulating dopaminergic activity with the D2 antagonist amisulpride, we provide empirical evidence for a novel process model of cost-benefit weighting that reconciles conflicting views by assuming dissociable effects of dopamine on distinct subcomponents of the decision process.

D2R blockade (relative to placebo) increased the sensitivity to variation in reward magnitudes during evidence accumulation, such that only relatively large future rewards were considered to be worth the waiting cost, whereas small delayed rewards were perceived as less valuable than sooner rewards. This dopaminergic impact on the drift rate is consistent with the view that D2R-mediated tonic dopamine levels implement a cost control determining whether a reward is worth the required action costs (Beeler and Mourra, 2018). From this perspective, lowering D2R activity with amisulpride resulted in a stricter cost control such that only rather large delayed rewards were able to overcome D2R-mediated cortical inhibition (Lerner and Kreitzer, 2011). While this effect is consistent with the standard view according to which dopamine increases the preference for large costly rewards (Robbins and Everitt, 1992; Salamone and Correa, 2012; Schultz, 2015), the dopaminergic effects on the starting bias parameter yielded a different pattern. Here, inhibition of D2R activation reduced the impact of delay costs on the starting bias, such that for shorter delays (where the immediate reward has only a small proximity advantage) D2R inhibition shifts the bias toward the SS option. This finding represents first evidence for the hypothesis that tonic dopamine moderates the impact of proximity (e.g., more concrete vs. more abstract rewards) on cost-benefit decision making (Soutschek et al., 2022; Westbrook and Frank, 2018). Pharmacological manipulation of D1R activation, in contrast, showed no significant effects on the decision process. This provides evidence for the receptor specificity of dopamine’s role in intertemporal decision making (though as caveat it is worth keeping the differences between the tasks administered in the D1 and the D2 studies in mind).

Conceptually, the assumption of proximity effects on the starting bias is consistent with dual process models of intertemporal choice assuming that individuals are (at least partially) biased toward selecting immediate over delayed rewards (Figner et al., 2010; McClure et al., 2004). This automatic favoring of immediate rewards is reflected in a shift of the starting bias and thus occurs before the evidence accumulation process, which relies on attention-demanding cost-benefit weighting (Zhao et al., 2019). In agreement with this notion, DDM-1 with temporal proximity-dependent bias showed better fit than DDM-5 with variable non-decision times for rewards and delays. We note that the hierarchical modeling approach allowed us to compare models on the group-level only, such that in some individuals behavior might better be explained by a different model than DDM-1. Such model comparisons on the individual level, however, were beyond the scope of the current study and might not yield robust results given the limited number of trials per individual. We also emphasize that alternative process models like the linear ballistic accumulator (LBA) model make different assumptions than DDMs, for example by positing the existence of separate option-specific accumulators rather than only one as assumed by DDMs. However, proximity effects as investigated in the current study might be incorporated in LBA models as well by varying the starting points of the accumulators as function of proximity.

A dopaminergic modulation of proximity effects provides an elegant explanation for the fact that in most D2 antagonist studies D2R reduction increased the preference for LL options (Arrondo et al., 2015; Soutschek et al., 2017; Wagner et al., 2020; Weber et al., 2016), contrary to the predictions of energization accounts (Beeler and Mourra, 2018; Salamone and Correa, 2012). Noteworthy, the dopaminergic effects on evidence accumulation and on the starting bias promote potentially different action tendencies, as the impact of amisulpride on evidence accumulation lowered the weight assigned to small future rewards, whereas the amisulpride effects on the starting bias increased the likelihood of LL options being chosen. Rather than generally biasing impulsive or patient choices, the impact of dopamine on decision making may therefore crucially depend on the rewards at stake and the associated waiting costs (Figure 4). In our model, lower dopamine levels strengthen the preference for high reward-high cost options predominantly in two situations. First, if differences in reward magnitude are high (e.g., choosing between your favorite meal vs. a clearly less liked dish) and, second, if the less costly option has a clear proximity advantage over the costlier one (having dinner in a restaurant close-by or a preferred restaurant on the other side of town). Conversely, if differences in both expected reward and waiting costs are small, lower dopamine may bias choices in favor of low-cost rewards over high-cost rewards. By extension, higher dopamine levels should increase the preference for an SS option if the SS option has a pronounced proximity advantage over the LL option, and bias the acceptance of LL options if both options are associated with similar waiting costs. We note though that the effects of increasing dopamine levels are less predictable than the effects of lowering dopaminergic activity due to possible inverted-U-shaped dopamine-response curves (Floresco, 2013); potentially, the dopaminergic effects on drift rate and starting bias might even follow different dose-response functions. Taken together, our process model of the dopaminergic involvement in cost-benefit decisions allows reconciling conflicting theoretical accounts and (apparently) inconsistent empirical findings by showing that dopamine moderates the effects of reward magnitudes and delay costs on different subcomponents of the choice process.

We note that the moderating roles of differences in delays are also reflected in the significant interaction between drug and delay from the model-free analysis, although this analysis could provide no insights into which subcomponents of the choice process are affected by dopamine. As the influence of dopamine on decision making varies as a function of the differences in reward magnitude and waiting costs, the outcomes of standard analyses like mean percentage of LL choices or hyperbolic discount parameters may be specific to the reward magnitudes and delays administered in a given study. For example, if an experimental task includes large differences between rewards and delays, dopamine antagonists may reduce delay discounting, whereas studies with smaller differences between these choice attributes may observe no effect of dopaminergic manipulations (Figure 5). Standard analyses that measure patience by one behavioral parameter only (e.g., discount factors) may thus result in misleading findings. In contrast, process models of decision making do not just assess whether a neural manipulation increases or reduces patience; instead, they quantify the influence of a manipulation on the weights assigned to rewards and waiting costs during different phases of the choice process, with these weights being less sensitive to the administered choice options in a given experiment. Process models may thus provide a less option-specific picture of the impact of pharmacological and neural manipulations.

Figure 5

Download asset Open asset

As potential alternative explanation for the enhanced influence of reward magnitude under amisulpride, one might argue that D2R blockade generally increases the signal-to-noise ratio for decision-relevant information. However, this notion is inconsistent with the proposed role of D2R activation for precise action selection (Keeler et al., 2014), because this view would have predicted amisulpride to result in noisier (less precise action selection) rather than less noisy evidence accumulation. Moreover, our data provide no evidence for drug effects on the inverse temperature parameter measuring choice consistency, and there were also no significant correlations between amisulpride effects on reward and delay processing, contrary to what one should expect if these effects were driven by the same mechanism.

While higher doses of amisulpride (as administered in the current study) antagonize postsynaptic D2Rs, lower doses (50–300 mg) were found to primarily block presynaptic dopamine receptors (Schoemaker et al., 1997), which may result in amplified phasic dopamine release and thus increased sensitivity to benefits (Frank and O’Reilly, 2006). At first glance, the stronger influence of differences in reward magnitude on drift rates under amisulpride compared with placebo might therefore speak in favor of presynaptic (higher dopamine levels) rather than postsynaptic mechanisms of action in the current study. However, amisulpride vs. placebo increased evidence accumulation toward LL rewards (more positive drift rate) only for larger differences between larger (later) and smaller (sooner) rewards, whereas for smaller reward differences amisulpride enhanced evidence accumulation toward SS choices (more negative drift rate; see Figure 2C). The latter finding appears inconsistent with presynaptic effects, as higher dopamine levels are thought to increase the preference for costly larger rewards (Webber et al., 2021). Instead, the stronger influence of reward differences on drift rates under amisulpride could be explained by a stricter cost control (Beeler and Mourra, 2018). In this interpretation, individuals more strongly distinguish between larger rewards that are worth the waiting costs (large difference between LL and SS rewards) and larger rewards that are not worth the same waiting costs (small difference between LL and SS rewards). While this speaks in favor of postsynaptic effects, we acknowledge that the amisulpride effects for larger reward differences are compatible with presynaptic mechanisms.

The result pattern for the starting bias parameter, in turn, suggests the presence of two distinct response biases, reflected by the intercept and the delay-dependent slope of the bias parameter (see Figure 2K), which are both under dopaminergic control but in opposite directions. First, participants seem to have a general bias toward the LL option in the current task (intercept), which is reduced under amisulpride compared with placebo, consistent with the assumption that dopamine strengthens the preference for larger rewards (Beeler and Mourra, 2018; Salamone and Correa, 2012; Schultz, 2015). Second, amisulpride reduced the impact of increasing differences in delay on the starting bias, as predicted by the proximity account of tonic dopamine (Westbrook and Frank, 2018). Both of these effects are compatible with postsynaptic effects of amisulpride. However, we note that in principle one might make the assumption that proximity effects are stronger for smaller than for larger differences in delay, and under this assumption the results would be consistent with presynaptic effects. On balance, the current results thus appear more likely under the assumption of postsynaptic rather than presynaptic effects but the latter cannot be entirely excluded. Unfortunately, the lack of a significant amisulpride effect on decision times (which should be reduced or increased as consequence of presynaptic or postsynaptic effects, respectively) sheds no additional light on the issue. Lastly, while the actions of amisulpride on D2/D3 receptors are relatively selective, it also affects serotonergic 5-HT7 receptors (Abbas et al., 2009). Because serotonin has been related to impulsive behavior (Mori et al., 2018), it is worth keeping in mind that amisulpride effects on serotonergic, in addition to dopaminergic, activity might contribute to the observed result pattern.

An important question refers to whether our findings for delay costs can be generalized to other types of costs as well, including risk, social costs (i.e., inequity), effort, and opportunity costs. We recently proposed that dopamine might also moderate proximity effects for reward options differing in risk and social costs, whereas the existing literature provides no evidence for a proximity advantage of effort-free over effortful rewards (Soutschek et al., 2022). However, these hypotheses need to be tested more explicitly by future investigations. Dopamine has also been ascribed a role for moderating opportunity costs, with lower tonic dopamine reducing the sensitivity to opportunity costs (Niv et al., 2007). While this appears consistent with our finding that amisulpride (under the assumption of postsynaptic effects) reduced the impact of delay on the starting bias, it is important to note that choosing delayed rewards did not involve any opportunity costs in our paradigm, given that participants could pursue other rewards during the waiting time. Thus, it needs to be clarified whether our findings for delayed rewards without experienced waiting time can be generalized to choice situations involving experienced opportunity costs.

To conclude, our findings may shed a new light on the role of dopamine in psychiatric disorders that are characterized by deficits in impulsiveness or cost-benefit weighting in general (Hasler, 2012), and where dopaminergic drugs belong to the standard treatments for deficits in value-related and other behavior. Dopaminergic manipulations yielded mixed results on impulsiveness in psychiatric and neurologic disorders (Acheson and de Wit, 2008; Antonelli et al., 2014; Foerde et al., 2016; Kayser et al., 2017), and our process model regarding the role of dopamine for delaying gratification explains some of the inconsistencies between empirical findings (on top of factors like non-linear dose-response relationships). As similarly inconsistent findings were observed also in the domains of risky and social decision making (Soutschek et al., 2022; Webber et al., 2021), the proposed process model may account for the function of dopamine in these domains of cost-benefit weighting as well. By deepening the understanding of the role of dopamine in decision making, our findings provide insights into how abnormal dopaminergic activation, and its pharmacological treatment, in psychiatric disorders may affect distinct aspects of decision making.

The data supporting the findings of this study and the data analysis code are available on Open Science Framework (https://osf.io/dp2me/).

1. 1. Soutschek A

   (2023) Open Science Framework

   ID dp2me. Intertemporal Choice_amisulpride.

   https://osf.io/dp2me/

1. 1. Abbas AI
   2. Hedlund PB
   3. Huang XP
   4. Tran TB
   5. Meltzer HY
   6. Roth BL

   (2009) Amisulpride is a potent 5-HT7 antagonist: relevance for antidepressant actions in vivo

   Psychopharmacology 205:119–128.

   https://doi.org/10.1007/s00213-009-1521-8

   • PubMed
   • Google Scholar
2. 1. Acheson A
   2. de Wit H

   (2008) Bupropion improves attention but does not affect impulsive behavior in healthy young adults

   Experimental and Clinical Psychopharmacology 16:113–123.

   https://doi.org/10.1037/1064-1297.16.2.113

   • PubMed
   • Google Scholar
3. 1. Ahn WY
   2. Haines N
   3. Zhang L

   (2017) Revealing neurocomputational mechanisms of reinforcement learning and decision-making with the hbayesdm package

   Computational Psychiatry 1:24–57.

   https://doi.org/10.1162/CPSY_a_00002

   • PubMed
   • Google Scholar
4. 1. Amasino DR
   2. Sullivan NJ
   3. Kranton RE
   4. Huettel SA

   (2019) Amount and time exert independent influences on intertemporal choice

   Nature Human Behaviour 3:383–392.

   https://doi.org/10.1038/s41562-019-0537-2

   • PubMed
   • Google Scholar
5. 1. Antonelli F
   2. Ko JH
   3. Miyasaki J
   4. Lang AE
   5. Houle S
   6. Valzania F
   7. Ray NJ
   8. Strafella AP

   (2014) Dopamine-agonists and impulsivity in Parkinson’s disease: impulsive choices vs. impulsive actions

   Human Brain Mapping 35:2499–2506.

   https://doi.org/10.1002/hbm.22344

   • PubMed
   • Google Scholar
6. 1. Arrondo G
   2. Aznárez-Sanado M
   3. Fernández-Seara MA
   4. Goñi J
   5. Loayza FR
   6. Salamon-Klobut E
   7. Heukamp FH
   8. Pastor MA

   (2015) Dopaminergic modulation of the trade-off between probability and time in economic decision-making

   European Neuropsychopharmacology 25:817–827.

   https://doi.org/10.1016/j.euroneuro.2015.02.011

   • PubMed
   • Google Scholar
7. 1. Beeler JA

   (2012) Thorndike’s law 2.0: dopamine and the regulation of thrift

   Frontiers in Neuroscience 6:116.

   https://doi.org/10.3389/fnins.2012.00116

   • PubMed
   • Google Scholar
8. 1. Beeler JA
   2. Mourra D

   (2018) To do or not to do: dopamine, affordability and the economics of opportunity

   Frontiers in Integrative Neuroscience 12:6.

   https://doi.org/10.3389/fnint.2018.00006

   • PubMed
   • Google Scholar
9. 1. Bürkner PC

   (2017) Brms: an R package for bayesian multilevel models using stan

   Journal of Statistical Software 80:1–28.

   https://doi.org/10.18637/jss.v080.i01

   • Google Scholar
10. 1. Cools R
    2. Gibbs SE
    3. Miyakawa A
    4. Jagust W
    5. D’Esposito M

    (2008) Working memory capacity predicts dopamine synthesis capacity in the human striatum

    The Journal of Neuroscience 28:1208–1212.

    https://doi.org/10.1523/JNEUROSCI.4475-07.2008

    • PubMed
    • Google Scholar
11. 1. Dai J
    2. Busemeyer JR

    (2014) A probabilistic, dynamic, and attribute-wise model of intertemporal choice

    Journal of Experimental Psychology. General 143:1489–1514.

    https://doi.org/10.1037/a0035976

    • PubMed
    • Google Scholar
12. 1. Evers EA
    2. Stiers P
    3. Ramaekers JG

    (2017) High reward expectancy during methylphenidate depresses the dopaminergic response to gain and loss

    Social Cognitive and Affective Neuroscience 12:311–318.

    https://doi.org/10.1093/scan/nsw124

    • PubMed
    • Google Scholar
13. 1. Figner B
    2. Knoch D
    3. Johnson EJ
    4. Krosch AR
    5. Lisanby SH
    6. Fehr E
    7. Weber EU

    (2010) Lateral prefrontal cortex and self-control in intertemporal choice

    Nature Neuroscience 13:538–539.

    https://doi.org/10.1038/nn.2516

    • PubMed
    • Google Scholar
14. 1. Floresco SB

    (2013) Prefrontal dopamine and behavioral flexibility: shifting from an “ inverted-U” toward a family of functions

    Frontiers in Neuroscience 7:62.

    https://doi.org/10.3389/fnins.2013.00062

    • PubMed
    • Google Scholar
15. 1. Foerde K
    2. Figner B
    3. Doll BB
    4. Woyke IC
    5. Braun EK
    6. Weber EU
    7. Shohamy D

    (2016) Dopamine modulation of intertemporal decision-making: evidence from Parkinson disease

    Journal of Cognitive Neuroscience 28:657–667.

    https://doi.org/10.1162/jocn_a_00929

    • PubMed
    • Google Scholar
16. 1. Fontanesi L
    2. Gluth S
    3. Spektor MS
    4. Rieskamp J

    (2019) A reinforcement learning diffusion decision model for value-based decisions

    Psychonomic Bulletin & Review 26:1099–1121.

    https://doi.org/10.3758/s13423-018-1554-2

    • PubMed
    • Google Scholar
17. 1. Frank MJ
    2. O’Reilly RC

    (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol

    Behavioral Neuroscience 120:497–517.

    https://doi.org/10.1037/0735-7044.120.3.497

    • PubMed
    • Google Scholar
18. 1. Hasler G

    (2012) Can the neuroeconomics revolution revolutionize psychiatry?

    Neuroscience and Biobehavioral Reviews 36:64–78.

    https://doi.org/10.1016/j.neubiorev.2011.04.011

    • PubMed
    • Google Scholar
19. 1. Kayser AS
    2. Vega T
    3. Weinstein D
    4. Peters J
    5. Mitchell JM

    (2017) Right inferior frontal cortex activity correlates with tolcapone responsivity in problem and pathological gamblers

    NeuroImage. Clinical 13:339–348.

    https://doi.org/10.1016/j.nicl.2016.12.022

    • PubMed
    • Google Scholar
20. 1. Keeler JF
    2. Pretsell DO
    3. Robbins TW

    (2014) Functional implications of dopamine D1 vs. D2 receptors: a “ prepare and select ” model of the striatal direct vs. indirect pathways

    Neuroscience 282:156–175.

    https://doi.org/10.1016/j.neuroscience.2014.07.021

    • PubMed
    • Google Scholar
21. 1. Kirschner M
    2. Rabinowitz A
    3. Singer N
    4. Dagher A

    (2020) From apathy to addiction: insights from neurology and psychiatry

    Progress in Neuro-Psychopharmacology & Biological Psychiatry 101:109926.

    https://doi.org/10.1016/j.pnpbp.2020.109926

    • PubMed
    • Google Scholar
22. 1. Konovalov A
    2. Krajbich I

    (2019) Revealed strength of preference: inference from response times

    Judgment and Decision Making 14:381–394.

    https://doi.org/10.1017/S1930297500006082

    • Google Scholar
23. 1. Kruschke JK

    (2013) Bayesian estimation supersedes the T test

    Journal of Experimental Psychology. General 142:573–603.

    https://doi.org/10.1037/a0029146

    • PubMed
    • Google Scholar
24. 1. Lerner TN
    2. Kreitzer AC

    (2011) Neuromodulatory control of striatal plasticity and behavior

    Current Opinion in Neurobiology 21:322–327.

    https://doi.org/10.1016/j.conb.2011.01.005

    • PubMed
    • Google Scholar
25. Preprint

    1. Lombardi G
    2. Hare T

    (2021) Piecewise Constant Averaging Methods Allow for Fast and Accurate Hierarchical Bayesian Estimation of Drift Diffusion Models with Time-Varying Evidence Accumulation Rates

    PsyArXiv.

    https://doi.org/10.31234/osf.io/5azyx

    • Google Scholar
26. 1. Lu ZH
    2. Chow SM
    3. Loken E

    (2017) A comparison of Bayesian and frequentist model selection methods for factor analysis models

    Psychological Methods 22:361–381.

    https://doi.org/10.1037/met0000145

    • PubMed
    • Google Scholar
27. 1. MacKillop J
    2. Amlung MT
    3. Few LR
    4. Ray LA
    5. Sweet LH
    6. Munafò MR

    (2011) Delayed reward discounting and addictive behavior: a meta-analysis

    Psychopharmacology 216:305–321.

    https://doi.org/10.1007/s00213-011-2229-0

    • PubMed
    • Google Scholar
28. 1. McClure SM
    2. Laibson DI
    3. Loewenstein G
    4. Cohen JD

    (2004) Separate neural systems value immediate and delayed monetary rewards

    Science 306:503–507.

    https://doi.org/10.1126/science.1100907

    • PubMed
    • Google Scholar
29. 1. Missale C
    2. Nash SR
    3. Robinson SW
    4. Jaber M
    5. Caron MG

    (1998) Dopamine receptors: from structure to function

    Physiological Reviews 78:189–225.

    https://doi.org/10.1152/physrev.1998.78.1.189

    • PubMed
    • Google Scholar
30. 1. Mori M
    2. Tsutsui-Kimura I
    3. Mimura M
    4. Tanaka KF

    (2018) 5-Ht3 antagonists decrease discounting rate without affecting sensitivity to reward magnitude in the delay discounting task in mice

    Psychopharmacology 235:2619–2629.

    https://doi.org/10.1007/s00213-018-4954-0

    • PubMed
    • Google Scholar
31. 1. Niv Y
    2. Daw ND
    3. Joel D
    4. Dayan P

    (2007) Tonic dopamine: opportunity costs and the control of response vigor

    Psychopharmacology 191:507–520.

    https://doi.org/10.1007/s00213-006-0502-4

    • PubMed
    • Google Scholar
32. Conference

    1. Plummer M.

    (2003)

    JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling

    Paper presented at the Proceedings of the 3rd international workshop on distributed statistical computing.

    • Google Scholar
33. 1. Reeck C
    2. Wall D
    3. Johnson EJ

    (2017) Search predicts and changes patience in intertemporal choice

    PNAS 114:11890–11895.

    https://doi.org/10.1073/pnas.1707040114

    • PubMed
    • Google Scholar
34. 1. Robbins TW
    2. Everitt BJ

    (1992) Functions of dopamine in the dorsal and ventral striatum

    Seminars in Neuroscience 4:119–127.

    https://doi.org/10.1016/1044-5765(92)90010-Y

    • Google Scholar
35. 1. Salamone JD
    2. Correa M

    (2012) The mysterious motivational functions of mesolimbic dopamine

    Neuron 76:470–485.

    https://doi.org/10.1016/j.neuron.2012.10.021

    • PubMed
    • Google Scholar
36. 1. Schoemaker H
    2. Claustre Y
    3. Fage D
    4. Rouquier L
    5. Chergui K
    6. Curet O
    7. Oblin A
    8. Gonon F
    9. Carter C
    10. Benavides J
    11. Scatton B

    (1997)

    Neurochemical characteristics of amisulpride, an atypical dopamine D2/D3 receptor antagonist with both presynaptic and limbic selectivity

    The Journal of Pharmacology and Experimental Therapeutics 280:83–97.

    • PubMed
    • Google Scholar
37. 1. Schultz W

    (2015) Neuronal reward and decision signals: from theories to data

    Physiological Reviews 95:853–951.

    https://doi.org/10.1152/physrev.00023.2014

    • PubMed
    • Google Scholar
38. 1. Soutschek A
    2. Burke CJ
    3. Raja Beharelle A
    4. Schreiber R
    5. Weber SC
    6. Karipidis II
    7. Ten Velden J
    8. Weber B
    9. Haker H
    10. Kalenscher T
    11. Tobler PN

    (2017) The dopaminergic reward system underpins gender differences in social preferences

    Nature Human Behaviour 1:819–827.

    https://doi.org/10.1038/s41562-017-0226-y

    • PubMed
    • Google Scholar
39. 1. Soutschek A.
    2. Tobler PN

    (2018) Motivation for the greater good: neural mechanisms of overcoming costs

    Current Opinion in Behavioral Sciences 22:96–105.

    https://doi.org/10.1016/j.cobeha.2018.01.025

    • Google Scholar
40. 1. Soutschek A
    2. Gvozdanovic G
    3. Kozak R
    4. Duvvuri S
    5. de Martinis N
    6. Harel B
    7. Gray DL
    8. Fehr E
    9. Jetter A
    10. Tobler PN

    (2020a) Dopaminergic d₁ receptor stimulation affects effort and risk preferences

    Biological Psychiatry 87:678–685.

    https://doi.org/10.1016/j.biopsych.2019.09.002

    • PubMed
    • Google Scholar
41. 1. Soutschek A
    2. Kozak R
    3. de Martinis N
    4. Howe W
    5. Burke CJ
    6. Fehr E
    7. Jetter A
    8. Tobler PN

    (2020b) Activation of D1 receptors affects human reactivity and flexibility to valued cues

    Neuropsychopharmacology 45:780–785.

    https://doi.org/10.1038/s41386-020-0617-z

    • PubMed
    • Google Scholar
42. 1. Soutschek A
    2. Jetter A
    3. Tobler PN

    (2022) Toward a unifying account of dopamine’s role in cost-benefit decision making

    Biological Psychiatry Global Open Science 010.

    https://doi.org/10.1016/j.bpsgos.2022.02.010

    • Google Scholar
43. 1. Toubia O
    2. Johnson E
    3. Evgeniou T
    4. Delquié P

    (2013) Dynamic experiments for estimating preferences: an adaptive method of eliciting time and risk parameters

    Management Science 59:613–640.

    https://doi.org/10.1287/mnsc.1120.1570

    • Google Scholar
44. 1. Volkow ND
    2. Baler RD

    (2015) Now vs later brain circuits: implications for obesity and addiction

    Trends in Neurosciences 38:345–352.

    https://doi.org/10.1016/j.tins.2015.04.002

    • PubMed
    • Google Scholar
45. 1. Wabersich D
    2. Vandekerckhove J

    (2014) Extending JAGS: a tutorial on adding custom distributions to JAGS (with a diffusion model example)

    Behavior Research Methods 46:15–28.

    https://doi.org/10.3758/s13428-013-0369-3

    • PubMed
    • Google Scholar
46. 1. Wagner B
    2. Clos M
    3. Sommer T
    4. Peters J

    (2020) Dopaminergic modulation of human intertemporal choice: a diffusion model analysis using the D2-receptor antagonist haloperidol

    The Journal of Neuroscience 40:7936–7948.

    https://doi.org/10.1523/JNEUROSCI.0592-20.2020

    • PubMed
    • Google Scholar
47. 1. Webber HE
    2. Lopez-Gamundi P
    3. Stamatovich SN
    4. de Wit H
    5. Wardle MC

    (2021) Using pharmacological manipulations to study the role of dopamine in human reward functioning: a review of studies in healthy adults

    Neuroscience and Biobehavioral Reviews 120:123–158.

    https://doi.org/10.1016/j.neubiorev.2020.11.004

    • PubMed
    • Google Scholar
48. 1. Weber SC
    2. Beck-Schimmer B
    3. Kajdi M-E
    4. Müller D
    5. Tobler PN
    6. Quednow BB

    (2016) Dopamine D2/3- and μ-opioid receptor antagonists reduce cue-induced responding and reward impulsivity in humans

    Translational Psychiatry 6:e850.

    https://doi.org/10.1038/tp.2016.113

    • PubMed
    • Google Scholar
49. 1. Westbrook A
    2. Frank M

    (2018) Dopamine and proximity in motivation and cognitive control

    Current Opinion in Behavioral Sciences 22:28–34.

    https://doi.org/10.1016/j.cobeha.2017.12.011

    • PubMed
    • Google Scholar
50. 1. Westbrook A
    2. van den Bosch R
    3. Määttä JI
    4. Hofmans L
    5. Papadopetraki D
    6. Cools R
    7. Frank MJ

    (2020) Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work

    Science 367:1362–1366.

    https://doi.org/10.1126/science.aaz5891

    • PubMed
    • Google Scholar
51. 1. Zhao WJ
    2. Diederich A
    3. Trueblood JS
    4. Bhatia S

    (2019) Automatic biases in intertemporal choice

    Psychonomic Bulletin & Review 26:661–668.

    https://doi.org/10.3758/s13423-019-01579-9

    • PubMed
    • Google Scholar

Author details

1. Alexander Soutschek

   Department of Psychology, Ludwig Maximilian University Munich, Munich, Germany

   Contribution

   Conceptualization, Formal analysis, Investigation, Visualization, Methodology, Writing - original draft

   For correspondence

   alexander.soutschek@psy.lmu.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-8438-7721

2. Philippe N Tobler

   1. Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zürich, Switzerland
   2. Neuroscience Center Zurich, University of Zurich, Swiss Federal Institute of Technology Zurich, Zurich, Switzerland

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-4915-9448

• 1,089

  views

• 133

  downloads

• 7

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Citations by DOI

• 7

  citations for umbrella DOI https://doi.org/10.7554/eLife.83734