Decisions are not isolated events, but are embedded in a sequence of choices. Choices, or their outcomes (e.g. rewards), exert a large influence on subsequent decisions (Sutton and Barto, 1998; Sugrue et al., 2004). This holds even for low-level perceptual choices (Fernberger, 1920; Rabbitt and Rodgers, 1977; Treisman and Williams, 1984). In most perceptual choice tasks used in the laboratory, the decision should only be based on current sensory input, the momentary ‘evidence’ for the decision. Thus, most work on their computational and neurophysiological mechanisms has largely focused on the transformation of sensory evidence into choice (Shadlen and Kiani, 2013). Yet, perceptual decisions are strongly influenced by experimental history: whether or not previous choices led to positive outcomes (Rabbitt and Rodgers, 1977; Dutilh et al., 2012), the confidence in them (Desender et al., 2018), and the content of the previous choice (i.e. which stimulus category was selected; Akaishi et al., 2014; Fründ et al., 2014; Urai et al., 2017). The latter type of sequential effect, which we call ‘choice history bias’, refers to the selective tendency to repeat (or alternate) previous choices. It is distinct and dissociable from effects of reward, performance feedback or subjective error awareness in previous trials.

Choice history biases are prevalent in human (Fründ et al., 2014; Urai et al., 2017), monkey (Gold et al., 2008) and rodent (Busse et al., 2011; Odoemene et al., 2018) perceptual decision-making. Remarkably, this holds even for environments lacking any correlations between stimuli presented on successive trials – the standard in psychophysical laboratory experiments. Choice history biases vary substantially across individuals (Abrahamyan et al., 2016; Urai et al., 2017). Neural signals reflecting previous choices have been found across the sensorimotor pathways of the cerebral cortex, from sensory to associative and motor regions (Gold et al., 2008; de Lange et al., 2013; Akaishi et al., 2014; Pape and Siegel, 2016; Purcell and Kiani, 2016a; St John-Saaltink et al., 2016; Thura et al., 2017; Hwang et al., 2017; Scott et al., 2017).

By which mechanism are choice history signals incorporated into the formation of a decision? Current models of perceptual decision-making posit the temporal accumulation of sensory evidence, resulting in an internal decision variable that grows with time (Bogacz et al., 2006; Gold and Shadlen, 2007; Ratcliff and McKoon, 2008; Brody and Hanks, 2016). When this decision variable reaches one of two decision bounds, a choice is made and the corresponding motor response is initiated. In this framework, a bias can arise in two ways: (i) by shifting the starting point of accumulation toward one of the bounds or (ii) by selectively changing the rate at which evidence for one versus the other choice alternative is accumulated. Figure 1 illustrates these two biasing mechanisms for a simple and widely used form of accumulation-to-bound model: the drift diffusion model (DDM). Similar principles apply to more complex accumulation-to-bound models. The starting point shift can be thought of as adding an offset to the perceptual interpretation of the current sensory evidence. By contrast, the evidence accumulation bias corresponds to biasing that perceptual interpretation toward one of the two stimulus categories.

Figure 1

Download asset Open asset

It is currently unknown which of those two principal mechanisms accounts for the choice history biases observed in overt behavior. Previous theoretical accounts have postulated a shift in the starting point of the decision variables toward the bound of the previous choice (Yu and Cohen, 2008; Zhang et al., 2014; Glaze et al., 2015). This is based on the assumption that the representation of the decision variable decays slowly, leaving a trace of the observer’s choice in the next trial (Cho et al., 2002; Gao et al., 2009; Gao et al., 2009; Bonaiuto et al., 2016). However, choice history biases might also originate from a slower (i.e. tens of seconds) across-trial accumulation of internal decision variables – analogous to the accumulation of external outcomes in value-based decisions (Sutton and Barto, 1998; Sugrue et al., 2004). Previous experimental work on perceptual choice history biases either did not analyze the within-trial decision dynamics (Busse et al., 2011; de Lange et al., 2013; Akaishi et al., 2014; Fründ et al., 2014; Urai et al., 2017; Braun et al., 2018), or only tested for starting point biases, not accumulation biases (Cho et al., 2002; Gold et al., 2008; Yu and Cohen, 2008; Gao et al., 2009; Wilder et al., 2009; Bode et al., 2012; Jones et al., 2013; Zhang et al., 2014).

Here, we untangle how history-dependent changes in evidence accumulation and starting point contribute to history biases in overt choice behavior. Across a range of perceptual choice tasks, we found that individual differences in choice repetition are explained by history-dependent biases in accumulation, not starting point. Thus, the interaction between choice history and decision formation seems to be more complex than previously thought: choices may bias later evidence accumulation processes towards (or away from) the previous chosen perceptual interpretation of the sensory input.

We fit different bounded-accumulation models to human behavioral data:choices and response times (RT). The DDM estimates model parameters from joint choices and RT distributions, and provides good fits to behavioral data from a large array of two-choice task (Ratcliff and McKoon, 2008). We estimated the following parameters: non-decision time (the time needed for sensory encoding and response execution), starting point of the decision variable, separation of the decision bounds, mean drift rate, and a stimulus-independent constant added to the mean drift. We refer to the latter parameter (termed ‘drift criterion’ by Ratcliff and McKoon, 2008) as ‘drift bias’.

Within the DDM, choice behavior can be selectively biased toward repetition or alternation by two mechanisms: shifting the starting point, or biasing the drift toward (or away from) the bound for the previous choice (Figure 1). These biasing mechanisms are hard to differentiate based on the proportion of choices alone, but they are readily distinguishable by the relationship between choice bias and RT (Figure 1c). Specifically, the conditional bias function (White and Poldrack, 2014) shows the fraction of choice repetitions as a function of their RT (binned in quantiles). A shift in starting point is most influential early in the decision process: it affects the leading edge of the RT distribution and shifts its mode. It predicts that the majority of history-dependent choice biases occur on trials with fast RTs (Figure 1c, green). A drift bias is instead accumulated along with the evidence and therefore grows as a function of elapsed time. Thus, drift bias strongly affects the trailing edge of the RT distribution with only a minor effect on the mode, altering choice fractions across the whole range of RTs (Figure 1c, blue). History-dependent changes in bound separation or mean drift rate may also occur, but they can only change overall RT and accuracy: those changes are by themselves not sufficient to bias the accumulation process toward one or the other bound, and thus toward choice repetition or alternation (see Figure 4—figure supplement 1).

We fit different variants of the DDM (Figure 3—figure supplement 1) to data from six experiments. These covered a range of task protocols and sensory modalities commonly used in studies of perceptual decision-making (Figure 2a): two alternative forced-choice, two interval forced-choice, and yes-no (simple forced choice) tasks; RT and so-called fixed duration tasks; visual motion direction and coherence discrimination, visual contrast and auditory detection; and experiments with and without single-trial performance feedback. As found in previous work (Fründ et al., 2014; Abrahamyan et al., 2016; Urai et al., 2017), observers exhibited a wide range of idiosyncratic choice history biases across all experiments (Figure 2b,c). To ensure that the DDM is an appropriate (simplified) modeling framework for these data, we first fit a basic version of the DDM that contained the above-described parameters, without allowing bias parameters to vary with choice history. We then fit the DDM while also allowing starting point, drift bias, or both to vary as a function of the observer’s choice on the previous trial.

Figure 2

Download asset Open asset

The DDM fits matched several aspects of the behavioral data (Figure 3—figure supplement 1). First, RT distributions matched the model predictions reasonably well (shown separately for each combination of stimuli and choices in Figure 3—figure supplement 1, darker colors indicate predicted RTs obtained through model simulations). Second, for the fits obtained with a hierarchical Bayesian fitting procedure (see Figure 3—figure supplement 1 and Materials and methods), used for Figures 3–5, the ${\displaystyle \hat{R}}$ for group-level parameters ranged between 0.9997 and 1.0406 across datasets, indicating good convergence of the sampling procedure (Wiecki et al., 2013). Third, individual drift rate estimates correlated with individual perceptual sensitivity (d’, Figure 3—figure supplement 1a) and monotonically increased with stronger sensory evidence (Figure 3—figure supplement 1a). In fixed duration tasks, the decision-maker does not need to set a bound for terminating the decision (Bogacz et al., 2006), so the bounded diffusion process described by the DDM might seem inappropriate. Yet, the success of the DDM in fitting these data was consistent with previous work (e.g. Ratcliff, 2006; Bode et al., 2012; Jahfari et al., 2012) and might have reflected the fact that observers set implicit decision bounds also when they do not control the stimulus duration (Kiani et al., 2008; but see Tsetsos et al., 2015).

History-dependent accumulation bias, not starting point bias, explains individual differences in choice repetition behavior

Models with history-dependent biases better explained the data than the baseline model without such history dependence (Figure 3a), corroborating the observation that observers’ behavior showed considerable dependence on previous choices (Figure 2f). The model with both history-dependent starting point and drift bias provided the best fit to five out of six datasets (Figure 3a), based on the Akaike Information Criterion (AIC; Akaike, 1974 - note that we obtained the same results when instead using the hierarchical Deviance Information Criterion).

Figure 3 with 2 supplements see all

Download asset Open asset

The above model comparison pointed to the importance of including a history-dependency into the model. We further examined the ability of each model to explain specific diagnostic features in the data (Palminteri et al., 2017) that distinguished starting point from drift bias. A history-dependent shift in the starting point leads to biased choices primarily when responses are fast (early RT quantiles), whereas a history-dependent shift in drift leads to biased choices across all trials, including those with slow responses (Figure 1). We simulated choices and RTs from the three different model variants and computed so-called ‘conditional bias functions’ (White and Poldrack, 2014): the fraction of choices in line with each observer’s choice repetition tendency (i.e. repetition probability), in each quantile of their RT distribution. For observers whose choice repetition probability was >0.5, this was the fraction of repetitions; for the other observers, this was the fraction of alternations. Consistent with a shift in drift bias, observers exhibited history-dependent choice biases across the entire range of RTs (Figure 3b). In particular, the biased choices on slow RTs could only be captured by models that included a history-dependent shift in drift bias (Figure 3c, blue and dark green bars).

We used the parameter estimates obtained from the full model (with both history-dependent starting point and drift bias) to investigate how history-dependent variations in starting point and drift bias related to each individual’s tendency to repeat their previous choices. We call each bias parameter’s dependence on the previous choice its ‘history shift’. For instance, in the left vs. right motion discrimination task, the history shift in starting point was computed as the difference between the starting point estimate for previous ‘left’ and previous ‘right’ choices, irrespective of the category of the current stimulus. The history shift in drift bias, but not the history shift in starting point, was robustly correlated to the individual probability of choice repetition (Figure 4a, significant correlations indicated with solid regression lines). In five out of six datasets, the correlation with the history shift in drift bias was significantly stronger than the correlation with the history shift in starting point (Figure 4b, Δ${\displaystyle \rho }$ values).

Figure 4 with 5 supplements see all

Download asset Open asset

We quantified the total evidence by computing a Bayes factor for each correlation (Wetzels and Wagenmakers, 2012), and multiplying these across datasets (Scheibehenne et al., 2016). This further confirmed that individual choice history biases were not captured by history shifts in starting point, but consistently captured by history shifts in drift (Figure 4b). Specifically, the Bayes factor for the history shift in starting point approached zero, indicating strong evidence for the null hypothesis of no correlation. The Bayes factor for the history shift in drift indicated strong evidence for a correlation (Kass and Raftery, 1995).

Correlations between estimated history shifts in starting point and drift bias were generally negative (mean ${\displaystyle \rho }$: −0.2884, range −0.4130 to −0.0757), but reached statistical significance (p<0.05) in only one dataset. The combined Bayes Factor (BF₁₀) was 0.0473, indicating strong evidence for H₀. We thus remain agnostic about the relationship between the history shifts of both parameters.

The same qualitative pattern of results was obtained with an alternative fitting procedure (non-hierarchical G² optimization, Figure 4—figure supplement 2a), as well as a model that allowed for additional across-trial variability in non-decision time (Figure 4—figure supplement 2b). Letting non-decision time vary with each level of sensory evidence strength (in the two datasets including multiple such levels) did not change the pattern of model comparison and correlation results (Figure 4—figure supplement 2c). These findings are thus robust to specifics of the model and fitting method. The Visual motion 2IFC #2 also included pharmacological interventions in two sub-groups of participants (see Materials and methods); we found the same effects for both drug groups as well as the placebo group (Figure 4—figure supplement 3). A significant positive correlation between history shift in drift bias and P(repeat) was present for two sub-groups of participants, defined as ‘repeaters’ and ‘alternators’ (based on P(repeat) being larger or smaller than 0.5, respectively; Figure 4—figure supplement 4).

The lack of a correlation between history-dependent starting point shifts and individual choice repetition is surprising in light of previous accounts (Yu and Cohen, 2008; Gao et al., 2009). History shifts in starting point were mostly negative (a tendency toward choice alternation) across participants, regardless of their individual tendency toward choice repetition or alternation (Figure 4—figure supplement 5, significant in two out of six datasets). This small but consistent effect likely explains why our formal model comparison favored a model with both history-dependent drift and starting point over one with only drift bias (see also Discussion). Critically, only the history-dependent shift in drift bias accounted for individual differences in choice repetition (Figure 4).

History-dependent accumulation bias explains individual choice repetition behavior irrespective of previous choice outcome

In four out of six tasks, participants received explicit outcome feedback (correct, error) after each choice. It is possible that participants experienced positive feedback as rewarding and (erroneously) assumed that a rewarded choice is more likely to be rewarded on the next trial. Manipulations of reward (probability or magnitude) have been found to change starting point (Voss et al., 2008; Leite and Ratcliff, 2011; Mulder et al., 2012), but might also bias drift (Liston and Stone, 2008; Afacan-Seref et al., 2018; Fan et al., 2018). Given that there were far more correct (i.e. rewarded) choices than errors, the history-dependent drift bias could reflect the expectation of reward for the choice that was correct on the previous trial.

Two findings refute this idea. First, the same result holds in the two datasets without single-trial outcome feedback (Figure 4a, bottom row), implying that external feedback is not necessary for history shifts in drift bias. Second, we found similar results when separately estimating the model parameters (history shift in starting point and drift bias) and model-free measures (choice repetition probability) after both correct and error trials (Figure 5a). Across datasets, individual repetition probability was best explained by history shifts in drift bias, not starting point, after both correct (Figure 5b) and error (Figure 5c) trials. Thus, even erroneous choices bias evidence accumulation on the next trial, in the same direction as correct choices. Indeed, most participants were predominantly biased by their previous choice (95 ‘stay’, 30 ‘switch’), while a third was biased by a combination of the previous choice and its correctness (26 ‘win-stay lose-switch’, 42 ‘win-switch lose-stay’; Figure 2c).

Figure 5

Download asset Open asset

Correlations tended to be smaller for previous erroneous choices. However, directly comparing the correlation coefficients between post-correct and post-error trials (after subsampling the former to ensure equal trial numbers per participant) did not allow us to refute nor confirm a difference (Figure 5d). In sum, history-dependent drift biases did not require external feedback about choice outcome and were predominantly induced by the previous choice. These choice history-dependent biases in evidence accumulation were accompanied by effects on drift rate and boundary separation (Figure 4—figure supplement 1), in line with previous work on post-error slowing (Dutilh et al., 2012; Goldfarb et al., 2012; Purcell and Kiani, 2016a).

Accumulation bias correlates with several past choices

Does the history shift in evidence accumulation depend on events from one past trial only? Recent work has exposed long-lasting choice history biases that span several trials and tens of seconds (Urai et al., 2017; Braun et al., 2018; Hermoso-Mendizabal et al., 2018). We thus estimated the influence of past events on the evidence accumulation process in a more comprehensive fashion. We fit a family of models in which correct and incorrect choices from up to six previous trials were used as predictors, and estimated their contribution to current starting point and drift bias.

Inclusion of further lags improved the model’s ability to account for the data, up to a lag of 2–4 after which model fits (ΔAIC) began to deteriorate (Figure 6—figure supplement 1). In 4/6 datasets, the best-fitting model contained only history-dependent changes in drift, not starting point, over a scale of the previous 2–4 trials. In the other two datasets, the best-fitting model was a hybrid where both drift and starting point varied as a function of choice history, up to two to trials into the past (Figure 6—figure supplement 1). We computed ‘history kernels’ across the different lags, separately for starting point and drift bias. These are analogous to the kernels obtained from a history-dependent regression analysis of the psychometric function that ignores decision time (Fründ et al., 2014), and which have been widely used in the recent literature on choice history biases (Fründ et al., 2014; Urai et al., 2017; Braun et al., 2018). To interpret these group-level kernels in light of substantial individual variability, we expressed each regression weight with respect to individual repetition probability at lag 1 (i.e. switching the sign for alternators).

Previous choices shifted drift bias in line with individual history bias across several trials, whereas starting point did not consistently shift in the direction of history bias. The hybrid models showed that the effect of choice history on drift bias decayed over approximately three past trials (Figure 6a), with a slower decay than for starting point (Figure 6a). The regression weights for past trials (from lag two through each dataset’s best-fitting lag) for drift bias – but not starting point - significantly correlated with the probability of repeating past choices at these same lags (Figure 6b). This was true after both correct and error trials (Figure 6b), similarly to the effects at lag 1 (Figure 5b–c).

Figure 6 with 1 supplement see all

Download asset Open asset

In sum, the biasing effect of choice history on evidence accumulation is long-lasting (longer than the effects on starting point), dependent on preceding choices several trials into the past, but independent of their correctness. This analysis corroborates the previous findings from our simpler models focusing on only the preceding trial, and further dissociate the effects of choice history on starting point and evidence accumulation.

History-dependent accumulation bias explains individual choice repetition behavior irrespective of specifics of bounded-accumulation models

We next set out to test the generality of our conclusions and gain deeper mechanistic insight into the nature of the dynamic (i.e. time-increasing) bias. We used a variety of bounded-accumulation models with more complex dynamics than the standard DDM. We focused on the preceding trial only, which our previous analyses had identified as exerting the same effect on history bias as the longer lags (Figure 6). These models included variants of the DDM (i.e. a perfect accumulator) with more complex dynamics of the bias or the decision bounds, as well as variants of a leaky accumulator (Busemeyer and Townsend, 1993; Usher and McClelland, 2001; Brunton et al., 2013). We focused on the Visual motion 2AFC (FD) dataset because it entailed small random dot stimuli (diameter 5° of visual angle), leading to large within- and across-trial fluctuations in the sensory evidence which we estimated through motion energy filtering (Adelson and Bergen, 1985; Urai and Wimmer, 2016; Figure 7—figure supplement 1). These fluctuating motion energy estimates were used as time-varying sensory input to the models, providing key additional constraints over and above nominal sensory evidence levels, choices and RT distributions (Brunton et al., 2013).

We first re-fit the standard DDM where the two biasing parameters were allowed to vary with previous choice (see Figure 1), now using single-trial motion energy estimates and a non-hierarchical fitting procedure (see Materials and methods). This made these fits directly comparable to both the hierarchical fits in Figures 3–4, and the more complex models described below. As expected (Figure 3a), the data were better explained by a history-dependent bias in the drift, rather than the starting point (Figure 7b1). In these non-hierarchical fits, the hybrid DDM (i.e. both bias terms free to vary as a function of previous choice) lost against the drift bias-only model (indicated by its higher AIC). Yet thi hybrid model allowed for a direct comparison of the correlations between these (jointly fit) bias parameters and individual choice repetition probability. As in our previous analysis (Figure 4), individual choice repetition probability was more strongly predicted by drift than starting point bias (Figure 6c1).

Figure 7 with 3 supplements see all

Download asset Open asset

A previous study of reward effects on speeded decisions reported that reward asymmetries induced supra-linear bias dynamics (Afacan-Seref et al., 2018). Temporal integration of a constant drift bias produces a linearly growing effective bias in the decision variable (Figure 1b), whereas integration of a ramping drift bias produces a supra-linear growth of effective bias (Figure 7a, yellow). In our data, a standard DDM with constant drift bias provided a better fit than DDMs with either a ramping drift bias, or a combination of constant and ramping drift bias (Figure 7b2). Furthermore, in the latter (hybrid) model, the constant drift bias best predicted individual choice repetition behavior (Figure 7c2), in line with the constant accumulation bias inferred from the standard DDM fits. For the fits shown in Figure 7b2/c2, we used the same fitting protocol as for the standard DDM, in which the time-varying sensory evidence fluctuations during stimulus presentation were replaced by their average over time to compute a single-trial drift rate (called ‘default protocol’, Materials and methods section Extended bounded accumulation models: General assumptions and procedures). The same qualitative pattern of results also held for another fitting protocol (‘dynamic protocol’, see Materials and methods), in which the time-varying sensory evidence was fed into the integrator (ΔAIC relative to no-history model: −1103, –985, −995, for constant drift bias, ramping drift bias, and hybrid, respectively; correlation with P(repeat): ρ(30)= 0.5458, p=0.0012; ρ(30)= 0.3600, p=0.0429 for constant and ramping drift bias, respectively). We next used this dynamic protocol for a set of more complex dynamical models.

It has been proposed that decision bounds might collapse over time, implementing an ‘urgency signal’ (Figure 6a, middle; Churchland et al., 2008; Cisek et al., 2009). Indeed, adding collapsing bounds substantially improved our model fits (Figure 7b3). This indicates the presence of a strong urgency signal in this task, which had a relatively short stimulus presentation (750 ms) and a tight response deadline (1.25 s after stimulus offset). Critically, a history-dependent drift bias best fit the data (Figure 7b3) and captured individual choice repetition behavior (Figure 7c3) also in the DDM with collapsing bounds. In other words, while there is evidence for collapsing bounds in this dataset, our conclusion about the impact of history bias on decision formation does not depend on its inclusion in the model.

In the brain, a neural representation of the momentary sensory evidence feeds into a set of accumulators. These consist of circuits of excitatory and inhibitory populations of cortical neurons, which give rise to persistent activity and competitive winner-take-all dynamics (Usher and McClelland, 2001; Wang, 2002). Under certain parameter regimes, these circuit dynamics can be reduced to lower-dimensional models (Bogacz et al., 2006; Wong, 2006). In such models, the effective accumulation time constant 1/λ (with λ being the effective leak) results from the balance of leak within each accumulator (due to self-excitation and passive decay) and mutual inhibition between two accumulators encoding different choices (Usher and McClelland, 2001). Evidence accumulation can then be biased through an internal representation of the sensory input, or through the way this sensory representation is accumulated (Figure 7a, right). We here used a reduced competing accumulator model, where the decision variable was computed as the difference of two leaky accumulators (Busemeyer and Townsend, 1993; Zhang and Bogacz, 2010; see also Brunton et al., 2013) to compare these two accumulation biases and a biased accumulator starting point.

We fit a family of bounded, leaky accumulator models, in which the starting point of the accumulators, their input, or their effective leak λ could be biased as a function of previous choice (Figure 7a, right). Note that a bias of the accumulator starting point would also translate into an accumulation bias, due to the model dynamics (see Materials and methods section Extended bounded accumulation models: General assumptions and procedures). Even so, comparing this regime with other two biasing mechanism was informative. Also note that we here use the term ‘leaky accumulator model’ to denote that the model dynamics consisted of a free effective leak parameter λ, without implying that λ < 0 (corresponding to activation decay). Our fits allowed λ to take either negative (‘forgetful’ regime) or positive (‘unstable’ regime) values (Figure 7—figure supplement 1d; see also Brunton et al., 2013). Critically, in order to test for choice history-dependent accumulation bias, we allowed λ of each accumulator to vary as a function of the previous choice, before computing the difference between the two accumulator activations. Choice-history dependent biases in accumulator starting point or accumulator input were directly applied to the accumulator difference (akin to starting point and drift bias within the DDM). Due to the simplicity of its dynamics, the DDM cannot distinguish between input and leak bias. Indeed, when simulating behavior of leaky accumulator models with either of these two accumulation biases and fitting it with the DDM, both input and λ bias loaded onto DDM drift bias (Figure 7—figure supplement 2). Critically, the leaky accumulator with biased accumulator input best explained the data, among all the models considered (Figure 7b4). Furthermore, the individually estimated input bias predicted individual choice repetition (Figure 7c4). This suggests that choice history might specifically bias the internal representation of sensory evidence feeding into the evidence accumulation process.

Quantitative treatments of perceptual decision-making commonly attribute trial-to-trial variability of overt choices to noisy decision computations (Shadlen et al., 1996; Renart and Machens, 2014; Wyart and Koechlin, 2016). Those accounts typically assume that systematic decision biases remain constant over time. Instead, the choice history biases studied here vary continuously over the course of the experiment, as a function of the previous choices (and choice outcome information). Our current results indicate that choice history explains trial-to-trial variability specifically in evidence accumulation, in a number of widely used perceptual choice tasks. Ignoring such trial-to-trial variations will lead to an overestimation of the noise in the evidence accumulation process and resulting behavior.

History biases in perceptual choice have long been known in perceptual psychophysics (Fernberger, 1920) and neuroscience (Gold et al., 2008). However, the underlying dynamic mechanisms have remained elusive. We here show that individual differences in overt choice repetition behavior are explained by the degree to which choices bias the evidence accumulation, not the starting point, of subsequent decisions. This accumulation bias is associated with choices made several trials into the past, and it grows steadily as the current decision unfolds. This insight calls for a revision of current models of choice history biases (Yu and Cohen, 2008; Zhang et al., 2014).

It is instructive to relate our results to previous studies manipulating the probability of the occurrence of a particular category (i.e. independently of the sequence of categories) or the asymmetry between rewards for both choices. Most of these studies explained the resulting behavioral biases in terms of starting point shifts (Leite and Ratcliff, 2011; Mulder et al., 2012; White and Poldrack, 2014; Rorie et al., 2010; Gao et al., 2011; but only for decisions without time pressure, see Afacan-Seref et al., 2018). Yet, one study with variations of evidence strength found an effect of asymmetric target probability on accumulation bias (Hanks et al., 2011) similar to the one we here identified for choice history. In all this previous work, biases were under experimental control: probability or reward manipulations were signaled via explicit task instructions or single-trial cues (in humans) or block structure (in animals). By contrast, the choice history biases we studied here emerge spontaneously and in an idiosyncratic fashion (Figure 2e), necessitating our focus on individual differences.

Our modeling addressed the question of how prior information is combined with new evidence during decision formation (see in particular the section Dynamics of effective bias signal approximates rational combination of prior information with current evidence). But why did participants use choice history as a prior for their decisions? In all our experiments, the sensory evidence was uncorrelated across trials – as is the case in the majority of perceptual choice tasks used in the literature. Thus, any history bias can only reduce performance below the level that could be achieved, given the observer’s sensitivity. It may seem irrational that people use history biases in such settings. However, real-world sensory evidence is typically stable (i.e. auto-correlated) across various timescales (Yu and Cohen, 2008). Thus, people might (erroneously) apply an internal model of this environmental stability to randomized laboratory experiments (Yu and Cohen, 2008), which will push them toward choice repetition or alternation (Glaze et al., 2015). Indeed, people flexibly adjust their choice history biases to environments with different levels of stability (Glaze et al., 2015; Kim et al., 2017; Braun et al., 2018), revealing the importance of such internal models on perceptual decision-making. In sum, with our conclusions from the time course of the effective bias signal, these considerations suggest that participants may have applied a rational strategy, but based on erroneous assumptions about the structure of the environment.

While we found that choice history-dependent variations of accumulation bias were generally more predictive of individual choice repetition behavior, the DDM starting point was consistently shifted away from the previous response for a majority of participants (i.e. negative values along x-axis of Figure 4a). This shift was statistically significant in three out or six datasets (Figure 4—figure supplement 5a), and might explain the advantage of the dual parameter model over the pure drift-bias model in our model comparisons (Figure 3a). The starting point shift may be due to at least two scenarios, which are not mutually exclusive. First, it might reflect a stereotypical response alternation tendency originating from neural dynamics in motor cortex – for example, a post-movement ‘rebound’ of beta-band oscillations (Pfurtscheller et al., 1996). Indeed, previous work found that beta rebound is related to response alternation in a perceptual choice task, which precluded (in contrast to our tasks) motor planning during evidence accumulation (Pape and Siegel, 2016). This stereotypical response alternation tendency (via starting point) may have conspired with the more flexible history bias of evidence accumulation (via drift bias) to shape choice behavior. Because starting point shifts will predominantly contribute to fast decisions, this scenario is consistent with the average choice alternation tendency we observed for RTs < 600 ms (Figure 4—figure supplement 5c). Because the response alternation tendency in motor cortex is likely to be induced only by the immediately preceding response, this scenario is also consistent with the shorter timescales we estimated for the starting point effects (1.39 trials) than the drift rate effects (2.38 trials; Figure 6a, exponential fits). Second, the starting point shift may also reflect decision dynamics more complex than described by the standard DDM: non-linear drift biases (Figure 7—figure supplement 2, third column) or biases in the leak of decision accumulators (Figure 7—figure supplement 3, third column). Both give rise to opposite effects on drift bias and starting point bias when fit with the standard DDM, thus yielding negative correlations between DDM starting point and drift bias estimates. Such negative correlations were present in our data, but weak and not statistically significant (Spearman’s rho −0.4130 to −0.0757, combined BF₁₀= 0.0473). It is possible that both of the scenarios discussed here conspired to yield the starting point effects observed in model comparisons and individual parameter estimates. Future work is needed to illuminate this issue, for example through manipulating decision speed and/or the delays between subsequent motor responses, and modeling choice-related neural dynamics in motor cortex.

We propose that choice history biases evidence accumulation, but there are alternative scenarios. First, it is possible that participants’ choices were due to computations altogether different from those incorporated in the bounded accumulation models assessed here. All our models imply simple neural accumulators with persistent activity. At least on a subset of trials, participants may make fast guesses (Noorbaloochi et al., 2015), or engage in automatic decision processing (Servant et al., 2014; Ulrich et al., 2015) or post-accumulation biases (Erlich et al., 2015). The decision computation may also entail noise-driven attractor dynamics (Wang, 2002; Braun and Mattia, 2010) possibly with sudden ‘jumps’ between neural activity states (Latimer et al., 2015), instead of linear accumulation to a threshold level. Even if the accumulation dynamics postulated in our models cannot be reduced to the dynamics of single neurons, the history-dependent accumulation bias we inferred here would constitute a valid description of the collective computational properties of the neural system producing choice behavior. Second, within bounded accumulation models, any directed change in the decision variable can be mimicked by some selective (i.e. asymmetric) change in one of the decision bounds. For example, combining the DDM with a linearly collapsing bound for the favored choice and a linearly expanding bound for the other choice has the same effect on choice fractions and RT distributions as a drift bias. We are not aware of any empirical evidence for such asymmetric changes in decision bounds. Decision-related cortical ramping activity seems to always reach a fixed level just before motor response, irrespective of prior probabilities (Hanks et al., 2011) or speed-accuracy trade-offs (Hanks et al., 2014; Murphy et al., 2016). Instead, the build-up of this activity is biased by prior information (Hanks et al., 2011).

A plausible mechanism underlying the choice history-dependent shift in accumulation bias is a bias of the neural representations of the sensory evidence towards (or away from) a previously selected category (Nienborg and Cumming, 2009; St John-Saaltink et al., 2016; Urai and Wimmer, 2016). This is precisely the ‘input bias’ scenario entailed in our best fitting model (Figure 7). The primate brain is equipped with powerful machinery to bias sensory representations in a top-down fashion (Desimone and Duncan, 1995; Reynolds and Heeger, 2009). In the laboratory, these top-down mechanisms have been probed by explicitly instructing subjects to shift their attention to a particular sensory feature or location. Such instructions induce biased activity states in regions of prefrontal and parietal association cortex, which are propagated down the cortical hierarchy to sensory cortex via selective feedback projections, where they boost the corresponding feature representations and suppress others (Desimone and Duncan, 1995). The same prefrontal and parietal regions accumulate sensory evidence and seem to carry choice history signals. It is tempting to speculate that choice history signals in these regions cause the same top-down modulation of sensory cortex as during explicit manipulations of attention. In other words, agents’ choices might be one factor directing their top-down attention under natural conditions, in a way analogous to explicit attention cues in laboratory tasks. An alternative, but related possibility is that the direction of selective attention fluctuates spontaneously during the course of a perceptual choice experiment, preferentially sampling features supporting one choice for a streak of trials, and then switching to sampling support for the other category. The corresponding top-down modulations would bias evidence accumulation and choice in a serially correlated fashion. These ideas are not mutually exclusive and can be tested by means of multi-area neurophysiological recordings combined with local perturbations.

A growing body of evidence points to the interplay of multiple timescales for neural computation in the cortex. One line of behavioral work has revealed effective (within-trial) evidence accumulation over timescales ranging from a few hundred milliseconds (Kiani et al., 2008; Tsetsos et al., 2015) to several seconds (Tsetsos et al., 2012; Wyart et al., 2012; Cheadle et al., 2014). Another line of work, including the current study, revealed the slow accumulation of internal decision variables or external outcome information across trials (tens of seconds) to build up time-varying biases, or priors (Sugrue et al., 2004; Abrahamyan et al., 2016; Purcell and Kiani, 2016b; Braun et al., 2018). Relatedly, neurophysiological work on ongoing activity has inferred multiple hierarchically organized timescales in different cortical regions (Honey et al., 2012; Murray et al., 2014; Chaudhuri et al., 2015; Runyan et al., 2017; Scott et al., 2017). The history-dependent evidence accumulation biases that we have uncovered here might index the interplay between these different effective timescales, with long-timescale accumulators at higher stages biasing short-timescale accumulators at intermediate stages of the cortical hierarchy.

All data is available at https://doi.org/10.6084/m9.figshare.7268558, and analysis code at https://github.com/anne-urai/2018_Urai_choice-history-ddm (copy archived at https://github.com/elifesciences-publications/2018_Urai_choice-history-ddm).

1. 1. Urai AE
   2. de Gee JW
   3. Tsetsos K
   4. Donner TH

   (2018) figshare

   Choice history biases subsequent evidence accumulation.

   https://doi.org/10.6084/m9.figshare.7268558

1. 1. Murphy PR
   2. Vandekerckhove J
   3. Nieuwenhuis S

   (2014) Dryad Digital Repository

   Pupil-linked arousal determines variability in perceptual decision making.

   https://doi.org/10.5061/dryad.tb542
2. 1. Urai AE
   2. Braun A
   3. Donner TH

   (2017) figshare

   Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias.

   https://doi.org/10.6084/m9.figshare.4300043
3. 1. de Gee JW
   2. Knapen T
   3. Donner TH

   (2014) figshare

   Decision-related pupil dilation reflects upcoming choice and individual bias.

   https://doi.org/10.6084/m9.figshare.4806559
4. 1. de Gee JW
   2. Colizoli O
   3. Kloosterman NA
   4. Knapen T
   5. Nieuwenhuis S
   6. Donner TH

   (2017) figshare

   Dynamic modulation of decision biases by brainstem arousal systems.

   https://doi.org/10.6084/m9.figshare.4806562

1. 1. Abrahamyan A
   2. Silva LL
   3. Dakin SC
   4. Carandini M
   5. Gardner JL

   (2016) Adaptable history biases in human perceptual decisions

   PNAS 113:E3548–E3557.

   https://doi.org/10.1073/pnas.1518786113

   • PubMed
   • Google Scholar
2. 1. Adelson EH
   2. Bergen JR

   (1985) Spatiotemporal energy models for the perception of motion

   Journal of the Optical Society of America A 2:284–299.

   https://doi.org/10.1364/JOSAA.2.000284

   • Google Scholar
3. 1. Afacan-Seref K
   2. Steinemann NA
   3. Blangero A
   4. Kelly SP

   (2018) Dynamic interplay of value and sensory information in high-speed decision making

   Current Biology 28:795–802.

   https://doi.org/10.1016/j.cub.2018.01.071

   • PubMed
   • Google Scholar
4. 1. Akaike H

   (1974) A new look at the statistical model identification

   IEEE Transactions on Automatic Control 19:716–723.

   https://doi.org/10.1109/TAC.1974.1100705

   • Google Scholar
5. 1. Akaishi R
   2. Umeda K
   3. Nagase A
   4. Sakai K

   (2014) Autonomous mechanism of internal choice estimate underlies decision inertia

   Neuron 81:195–206.

   https://doi.org/10.1016/j.neuron.2013.10.018

   • PubMed
   • Google Scholar
6. 1. Arieli A
   2. Sterkin A
   3. Grinvald A
   4. Aertsen A

   (1996) Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses

   Science 273:1868–1871.

   https://doi.org/10.1126/science.273.5283.1868

   • PubMed
   • Google Scholar
7. 1. Bode S
   2. Sewell DK
   3. Lilburn S
   4. Forte JD
   5. Smith PL
   6. Stahl J

   (2012) Predicting perceptual decision biases from early brain activity

   Journal of Neuroscience 32:12488–12498.

   https://doi.org/10.1523/JNEUROSCI.1708-12.2012

   • PubMed
   • Google Scholar
8. 1. Bogacz R
   2. Brown E
   3. Moehlis J
   4. Holmes P
   5. Cohen JD

   (2006) The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks

   Psychological Review 113:700–765.

   https://doi.org/10.1037/0033-295X.113.4.700

   • PubMed
   • Google Scholar
9. 1. Bonaiuto JJ
   2. Berker A
   3. Bestmann S

   (2016) Response repetition biases in human perceptual decisions are explained by activity decay in competitive attractor models

   eLife 5:e20047.

   https://doi.org/10.7554/eLife.20047

   • PubMed
   • Google Scholar
10. 1. Braun A
    2. Urai AE
    3. Donner TH

    (2018) Adaptive history biases result from Confidence-Weighted accumulation of past choices

    The Journal of Neuroscience 38:2418–2429.

    https://doi.org/10.1523/JNEUROSCI.2189-17.2017

    • PubMed
    • Google Scholar
11. 1. Braun J
    2. Mattia M

    (2010) Attractors and noise: twin drivers of decisions and multistability

    NeuroImage 52:740–751.

    https://doi.org/10.1016/j.neuroimage.2009.12.126

    • PubMed
    • Google Scholar
12. 1. Brody CD
    2. Hanks TD

    (2016) Neural underpinnings of the evidence accumulator

    Current Opinion in Neurobiology 37:149–157.

    https://doi.org/10.1016/j.conb.2016.01.003

    • PubMed
    • Google Scholar
13. 1. Brunton BW
    2. Botvinick MM
    3. Brody CD

    (2013) Rats and humans can optimally accumulate evidence for decision-making

    Science 340:95–98.

    https://doi.org/10.1126/science.1233912

    • PubMed
    • Google Scholar
14. 1. Busemeyer JR
    2. Townsend JT

    (1993) Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment

    Psychological Review 100:432–459.

    https://doi.org/10.1037/0033-295X.100.3.432

    • PubMed
    • Google Scholar
15. 1. Busse L
    2. Ayaz A
    3. Dhruv NT
    4. Katzner S
    5. Saleem AB
    6. Schölvinck ML
    7. Zaharia AD
    8. Carandini M

    (2011) The detection of visual contrast in the behaving mouse

    Journal of Neuroscience 31:11351–11361.

    https://doi.org/10.1523/JNEUROSCI.6689-10.2011

    • PubMed
    • Google Scholar
16. 1. Chamberlain SR
    2. Hampshire A
    3. Müller U
    4. Rubia K
    5. Del Campo N
    6. Craig K
    7. Regenthal R
    8. Suckling J
    9. Roiser JP
    10. Grant JE
    11. Bullmore ET
    12. Robbins TW
    13. Sahakian BJ

    (2009) Atomoxetine modulates right inferior frontal activation during inhibitory control: a pharmacological functional magnetic resonance imaging study

    Biological Psychiatry 65:550–555.

    https://doi.org/10.1016/j.biopsych.2008.10.014

    • PubMed
    • Google Scholar
17. 1. Chaudhuri R
    2. Knoblauch K
    3. Gariel MA
    4. Kennedy H
    5. Wang XJ

    (2015) A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex

    Neuron 88:419–431.

    https://doi.org/10.1016/j.neuron.2015.09.008

    • PubMed
    • Google Scholar
18. 1. Cheadle S
    2. Wyart V
    3. Tsetsos K
    4. Myers N
    5. de Gardelle V
    6. Herce Castañón S
    7. Summerfield C

    (2014) Adaptive gain control during human perceptual choice

    Neuron 81:1429–1441.

    https://doi.org/10.1016/j.neuron.2014.01.020

    • PubMed
    • Google Scholar
19. 1. Cho RY
    2. Nystrom LE
    3. Brown ET
    4. Jones AD
    5. Braver TS
    6. Holmes PJ
    7. Cohen JD

    (2002) Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task

    Cognitive, Affective, & Behavioral Neuroscience 2:283–299.

    https://doi.org/10.3758/CABN.2.4.283

    • PubMed
    • Google Scholar
20. 1. Churchland AK
    2. Kiani R
    3. Shadlen MN

    (2008) Decision-making with multiple alternatives

    Nature Neuroscience 11:693–702.

    https://doi.org/10.1038/nn.2123

    • Google Scholar
21. 1. Cisek P
    2. Puskas GA
    3. El-Murr S

    (2009) Decisions in changing conditions: the urgency-gating model

    Journal of Neuroscience 29:11560–11571.

    https://doi.org/10.1523/JNEUROSCI.1844-09.2009

    • PubMed
    • Google Scholar
22. 1. de Gee JW
    2. Knapen T
    3. Donner TH

    (2014) Decision-related pupil dilation reflects upcoming choice and individual bias

    PNAS 111:E618–E625.

    https://doi.org/10.1073/pnas.1317557111

    • PubMed
    • Google Scholar
23. 1. de Gee JW
    2. Colizoli O
    3. Kloosterman NA
    4. Knapen T
    5. Nieuwenhuis S
    6. Donner TH

    (2017) Dynamic modulation of decision biases by brainstem arousal systems

    eLife 6:e23232.

    https://doi.org/10.7554/eLife.23232

    • PubMed
    • Google Scholar
24. 1. de Lange FP
    2. Rahnev DA
    3. Donner TH
    4. Lau H

    (2013) Prestimulus oscillatory activity over motor cortex reflects perceptual expectations

    Journal of Neuroscience 33:1400–1410.

    https://doi.org/10.1523/JNEUROSCI.1094-12.2013

    • PubMed
    • Google Scholar
25. Preprint

    1. Desender K
    2. Murphy PR
    3. Boldt A
    4. Verguts T
    5. Yeung N

    (2018) A post-decisional neural marker of confidence predicts information-seeking

    bioRxiv.

    https://doi.org/10.1101/433276

    • Google Scholar
26. 1. Desimone R
    2. Duncan J

    (1995) Neural mechanisms of selective visual attention

    Annual Review of Neuroscience 18:193–222.

    https://doi.org/10.1146/annurev.ne.18.030195.001205

    • PubMed
    • Google Scholar
27. Preprint

    1. Drugowitsch J
    2. Pouget A

    (2018) Learning optimal decisions with confidence

    bioRxiv.

    https://doi.org/10.1101/244269

    • Google Scholar
28. 1. Dutilh G
    2. van Ravenzwaaij D
    3. Nieuwenhuis S
    4. van der Maas HLJ
    5. Forstmann BU
    6. Wagenmakers E-J

    (2012) How to measure post-error slowing: a confound and a simple solution

    Journal of Mathematical Psychology 56:208–216.

    https://doi.org/10.1016/j.jmp.2012.04.001

    • Google Scholar
29. 1. Erlich JC
    2. Brunton BW
    3. Duan CA
    4. Hanks TD
    5. Brody CD

    (2015) Distinct effects of prefrontal and parietal cortex inactivations on an accumulation of evidence task in the rat

    eLife 4:e05457.

    https://doi.org/10.7554/eLife.05457

    • Google Scholar
30. 1. Faisal AA
    2. Selen LP
    3. Wolpert DM

    (2008) Noise in the nervous system

    Nature Reviews Neuroscience 9:292–303.

    https://doi.org/10.1038/nrn2258

    • PubMed
    • Google Scholar
31. 1. Fan Y
    2. Gold JI
    3. Ding L

    (2018) Ongoing, rational calibration of reward-driven perceptual biases

    eLife 7:e36018.

    https://doi.org/10.7554/eLife.36018

    • PubMed
    • Google Scholar
32. 1. Fernberger SW

    (1920) Interdependence of judgments within the series for the method of constant stimuli

    Journal of Experimental Psychology 3:126–150.

    https://doi.org/10.1037/h0065212

    • Google Scholar
33. 1. Fründ I
    2. Wichmann FA
    3. Macke JH

    (2014) Quantifying the effect of intertrial dependence on perceptual decisions

    Journal of Vision 14:9.

    https://doi.org/10.1167/14.7.9

    • PubMed
    • Google Scholar
34. 1. Gao J
    2. Wong-Lin K
    3. Holmes P
    4. Simen P
    5. Cohen JD

    (2009) Sequential effects in two-choice reaction time tasks: decomposition and synthesis of mechanisms

    Neural Computation 21:2407–2436.

    https://doi.org/10.1162/neco.2009.09-08-866

    • PubMed
    • Google Scholar
35. 1. Gao J
    2. Tortell R
    3. McClelland JL

    (2011) Dynamic integration of reward and stimulus information in perceptual decision-making

    PLOS ONE 6:e16749.

    https://doi.org/10.1371/journal.pone.0016749

    • PubMed
    • Google Scholar
36. 1. Glaze CM
    2. Kable JW
    3. Gold JI

    (2015) Normative evidence accumulation in unpredictable environments

    eLife 4:e08825.

    https://doi.org/10.7554/eLife.08825

    • Google Scholar
37. 1. Gold JI
    2. Law CT
    3. Connolly P
    4. Bennur S

    (2008) The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning

    Journal of Neurophysiology 100:2653–2668.

    https://doi.org/10.1152/jn.90629.2008

    • PubMed
    • Google Scholar
38. 1. Gold JI
    2. Shadlen MN

    (2007) The neural basis of decision making

    Annual Review of Neuroscience 30:535–574.

    https://doi.org/10.1146/annurev.neuro.29.051605.113038

    • PubMed
    • Google Scholar
39. 1. Goldfarb S
    2. Wong-Lin K
    3. Schwemmer M
    4. Leonard NE
    5. Holmes P

    (2012) Can post-error dynamics explain sequential reaction time patterns?

    Frontiers in Psychology 3:213.

    https://doi.org/10.3389/fpsyg.2012.00213

    • PubMed
    • Google Scholar
40. Book

    1. Green DM
    2. Swets JA

    (1966)

    Signal Detection Theory and Psychophysics

    John Wiley and Sons.

    • Google Scholar
41. 1. Hanks TD
    2. Mazurek ME
    3. Kiani R
    4. Hopp E
    5. Shadlen MN

    (2011) Elapsed decision time affects the weighting of prior probability in a perceptual decision task

    Journal of Neuroscience 31:6339–6352.

    https://doi.org/10.1523/JNEUROSCI.5613-10.2011

    • PubMed
    • Google Scholar
42. 1. Hanks T
    2. Kiani R
    3. Shadlen MN

    (2014) A neural mechanism of speed-accuracy tradeoff in macaque area LIP

    eLife 3:e02260.

    https://doi.org/10.7554/eLife.02260

    • Google Scholar
43. Preprint

    1. Hermoso-Mendizabal A
    2. Hyafil A
    3. Rueda-Orozco PE
    4. Jaramillo S
    5. Robbe D
    6. de la Rocha J

    (2018) Response outcomes gate the impact of expectations on perceptual decisions

    bioRxiv.

    https://doi.org/10.1101/433409

    • Google Scholar
44. 1. Honey CJ
    2. Thesen T
    3. Donner TH
    4. Silbert LJ
    5. Carlson CE
    6. Devinsky O
    7. Doyle WK
    8. Rubin N
    9. Heeger DJ
    10. Hasson U

    (2012) Slow cortical dynamics and the accumulation of information over long timescales

    Neuron 76:423–434.

    https://doi.org/10.1016/j.neuron.2012.08.011

    • PubMed
    • Google Scholar
45. 1. Hwang EJ
    2. Dahlen JE
    3. Mukundan M
    4. Komiyama T

    (2017) History-based action selection bias in posterior parietal cortex

    Nature Communications 8:1242.

    https://doi.org/10.1038/s41467-017-01356-z

    • PubMed
    • Google Scholar
46. 1. Jahfari S
    2. Verbruggen F
    3. Frank MJ
    4. Waldorp LJ
    5. Colzato L
    6. Ridderinkhof KR
    7. Forstmann BU

    (2012) How preparation changes the need for top-down control of the basal ganglia when inhibiting premature actions

    Journal of Neuroscience 32:10870–10878.

    https://doi.org/10.1523/JNEUROSCI.0902-12.2012

    • PubMed
    • Google Scholar
47. 1. Jones M
    2. Curran T
    3. Mozer MC
    4. Wilder MH

    (2013) Sequential effects in response time reveal learning mechanisms and event representations

    Psychological Review 120:628–666.

    https://doi.org/10.1037/a0033180

    • PubMed
    • Google Scholar
48. 1. Kass RE
    2. Raftery AE

    (1995) Bayes factors

    Journal of the American Statistical Association 90:773–795.

    https://doi.org/10.1080/01621459.1995.10476572

    • Google Scholar
49. 1. Katahira K

    (2016) How hierarchical models improve point estimates of model parameters at the individual level

    Journal of Mathematical Psychology 73:37–58.

    https://doi.org/10.1016/j.jmp.2016.03.007

    • Google Scholar
50. 1. Kiani R
    2. Hanks TD
    3. Shadlen MN

    (2008) Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment

    Journal of Neuroscience 28:3017–3029.

    https://doi.org/10.1523/JNEUROSCI.4761-07.2008

    • PubMed
    • Google Scholar
51. 1. Kim TD
    2. Kabir M
    3. Gold JI

    (2017) Coupled decision processes update and maintain saccadic priors in a dynamic environment

    The Journal of Neuroscience 37:3632–3645.

    https://doi.org/10.1523/JNEUROSCI.3078-16.2017

    • PubMed
    • Google Scholar
52. 1. Latimer KW
    2. Yates JL
    3. Meister ML
    4. Huk AC
    5. Pillow JW

    (2015) Single-trial spike trains in parietal cortex reveal discrete steps during decision-making

    Science 349:184–187.

    https://doi.org/10.1126/science.aaa4056

    • PubMed
    • Google Scholar
53. 1. Leite FP
    2. Ratcliff R

    (2011)

    What cognitive processes drive response biases? A diffusion model analysis

    Judgment and Decision Making 6:651–687.

    • Google Scholar
54. 1. Liston DB
    2. Stone LS

    (2008) Effects of prior information and reward on oculomotor and perceptual choices

    Journal of Neuroscience 28:13866–13875.

    https://doi.org/10.1523/JNEUROSCI.3120-08.2008

    • PubMed
    • Google Scholar
55. 1. Mazurek ME
    2. Roitman JD
    3. Ditterich J
    4. Shadlen MN

    (2003) A role for neural integrators in perceptual decision making

    Cerebral Cortex 13:1257–1269.

    https://doi.org/10.1093/cercor/bhg097

    • PubMed
    • Google Scholar
56. 1. McGinley MJ
    2. David SV
    3. McCormick DA

    (2015) Cortical membrane potential signature of optimal states for sensory signal detection

    Neuron 87:179–192.

    https://doi.org/10.1016/j.neuron.2015.05.038

    • PubMed
    • Google Scholar
57. 1. Moran R

    (2015) Optimal decision making in heterogeneous and biased environments

    Psychonomic Bulletin & Review 22:38–53.

    https://doi.org/10.3758/s13423-014-0669-3

    • PubMed
    • Google Scholar
58. 1. Mulder MJ
    2. Wagenmakers EJ
    3. Ratcliff R
    4. Boekel W
    5. Forstmann BU

    (2012) Bias in the brain: a diffusion model analysis of prior probability and potential payoff

    Journal of Neuroscience 32:2335–2343.

    https://doi.org/10.1523/JNEUROSCI.4156-11.2012

    • PubMed
    • Google Scholar
59. 1. Murphy PR
    2. Vandekerckhove J
    3. Nieuwenhuis S

    (2014) Pupil-linked arousal determines variability in perceptual decision making

    PLOS Computational Biology 10:e1003854.

    https://doi.org/10.1371/journal.pcbi.1003854

    • PubMed
    • Google Scholar
60. 1. Murphy PR
    2. Boonstra E
    3. Nieuwenhuis S

    (2016) Global gain modulation generates time-dependent urgency during perceptual choice in humans

    Nature Communications 7:13526.

    https://doi.org/10.1038/ncomms13526

    • PubMed
    • Google Scholar
61. 1. Murray JD
    2. Bernacchia A
    3. Freedman DJ
    4. Romo R
    5. Wallis JD
    6. Cai X
    7. Padoa-Schioppa C
    8. Pasternak T
    9. Seo H
    10. Lee D
    11. Wang XJ

    (2014) A hierarchy of intrinsic timescales across primate cortex

    Nature Neuroscience 17:1661–1663.

    https://doi.org/10.1038/nn.3862

    • PubMed
    • Google Scholar
62. 1. Nienborg H
    2. Cumming BG

    (2009) Decision-related activity in sensory neurons reflects more than a neuron's causal effect

    Nature 459:89–92.

    https://doi.org/10.1038/nature07821

    • PubMed
    • Google Scholar
63. 1. Noorbaloochi S
    2. Sharon D
    3. McClelland JL

    (2015) Payoff information biases a fast guess process in perceptual decision making under deadline pressure: evidence from behavior, evoked potentials, and quantitative model comparison

    Journal of Neuroscience 35:10989–11011.

    https://doi.org/10.1523/JNEUROSCI.0017-15.2015

    • PubMed
    • Google Scholar
64. 1. Odoemene O
    2. Pisupati S
    3. Nguyen H
    4. Churchland AK

    (2018) Visual evidence accumulation guides decision-making in unrestrained mice

    The Journal of Neuroscience 38:10143–10155.

    https://doi.org/10.1523/JNEUROSCI.3478-17.2018

    • PubMed
    • Google Scholar
65. 1. Palminteri S
    2. Wyart V
    3. Koechlin E

    (2017) The importance of falsification in computational cognitive modeling

    Trends in Cognitive Sciences 21:425–433.

    https://doi.org/10.1016/j.tics.2017.03.011

    • PubMed
    • Google Scholar
66. 1. Pape AA
    2. Siegel M

    (2016) Motor cortex activity predicts response alternation during sensorimotor decisions

    Nature Communications 7:13098.

    https://doi.org/10.1038/ncomms13098

    • PubMed
    • Google Scholar
67. 1. Pfurtscheller G
    2. Stancák A
    3. Neuper C

    (1996) Post-movement beta synchronization. A correlate of an idling motor area?

    Electroencephalography and Clinical Neurophysiology 98:281–293.

    https://doi.org/10.1016/0013-4694(95)00258-8

    • PubMed
    • Google Scholar
68. 1. Purcell BA
    2. Kiani R

    (2016a) Neural mechanisms of post-error adjustments of decision policy in parietal cortex

    Neuron 89:658–671.

    https://doi.org/10.1016/j.neuron.2015.12.027

    • PubMed
    • Google Scholar
69. 1. Purcell BA
    2. Kiani R

    (2016b) Hierarchical decision processes that operate over distinct timescales underlie choice and changes in strategy

    PNAS 113:E4531–E4540.

    https://doi.org/10.1073/pnas.1524685113

    • PubMed
    • Google Scholar
70. 1. Rabbitt P
    2. Rodgers B

    (1977) What does a man do after he makes an error? an analysis of response programming

    Quarterly Journal of Experimental Psychology 29:727–743.

    https://doi.org/10.1080/14640747708400645

    • Google Scholar
71. 1. Ratcliff R

    (2006) Modeling response signal and response time data

    Cognitive Psychology 53:195–237.

    https://doi.org/10.1016/j.cogpsych.2005.10.002

    • PubMed
    • Google Scholar
72. 1. Ratcliff R
    2. Childers R

    (2015) Individual differences and fitting methods for the two-choice diffusion model of decision making

    Decision 2:237–279.

    https://doi.org/10.1037/dec0000030

    • Google Scholar
73. 1. Ratcliff R
    2. McKoon G

    (2008) The diffusion decision model: theory and data for two-choice decision tasks

    Neural Computation 20:873–922.

    https://doi.org/10.1162/neco.2008.12-06-420

    • PubMed
    • Google Scholar
74. 1. Ratcliff R
    2. Tuerlinckx F

    (2002) Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability

    Psychonomic Bulletin & Review 9:438–481.

    https://doi.org/10.3758/BF03196302

    • PubMed
    • Google Scholar
75. 1. Renart A
    2. Machens CK

    (2014) Variability in neural activity and behavior

    Current Opinion in Neurobiology 25:211–220.

    https://doi.org/10.1016/j.conb.2014.02.013

    • PubMed
    • Google Scholar
76. 1. Reynolds JH
    2. Heeger DJ

    (2009) The normalization model of attention

    Neuron 61:168–185.

    https://doi.org/10.1016/j.neuron.2009.01.002

    • PubMed
    • Google Scholar
77. 1. Rogers SL
    2. Friedhoff LT

    (1998) Pharmacokinetic and pharmacodynamic profile of donepezil HCl following single oral doses

    British Journal of Clinical Pharmacology 46:1–6.

    https://doi.org/10.1046/j.1365-2125.1998.0460s1001.x

    • PubMed
    • Google Scholar
78. 1. Rokem A
    2. Silver MA

    (2010) Cholinergic enhancement augments magnitude and specificity of visual perceptual learning in healthy humans

    Current Biology 20:1723–1728.

    https://doi.org/10.1016/j.cub.2010.08.027

    • PubMed
    • Google Scholar
79. 1. Rorie AE
    2. Gao J
    3. McClelland JL
    4. Newsome WT

    (2010) Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey

    PLoS ONE 5:e9308.

    https://doi.org/10.1371/journal.pone.0009308

    • PubMed
    • Google Scholar
80. 1. Runyan CA
    2. Piasini E
    3. Panzeri S
    4. Harvey CD

    (2017) Distinct timescales of population coding across cortex

    Nature 548:92–96.

    https://doi.org/10.1038/nature23020

    • PubMed
    • Google Scholar
81. 1. Sauer JM
    2. Ring BJ
    3. Witcher JW

    (2005) Clinical pharmacokinetics of atomoxetine

    Clinical Pharmacokinetics 44:571–590.

    https://doi.org/10.2165/00003088-200544060-00002

    • PubMed
    • Google Scholar
82. 1. Scheibehenne B
    2. Jamil T
    3. Wagenmakers EJ

    (2016) Bayesian evidence synthesis can reconcile seemingly inconsistent results: the case of hotel towel reuse

    Psychological Science 27:1043–1046.

    https://doi.org/10.1177/0956797616644081

    • PubMed
    • Google Scholar
83. 1. Scott BB
    2. Constantinople CM
    3. Akrami A
    4. Hanks TD
    5. Brody CD
    6. Tank DW

    (2017) Fronto-parietal cortical circuits encode accumulated evidence with a diversity of timescales

    Neuron 95:385–398.

    https://doi.org/10.1016/j.neuron.2017.06.013

    • PubMed
    • Google Scholar
84. 1. Servant M
    2. Montagnini A
    3. Burle B

    (2014) Conflict tasks and the diffusion framework: insight in model constraints based on psychological laws

    Cognitive Psychology 72:162–195.

    https://doi.org/10.1016/j.cogpsych.2014.03.002

    • PubMed
    • Google Scholar
85. 1. Shadlen MN
    2. Britten KH
    3. Newsome WT
    4. Movshon JA

    (1996) A computational analysis of the relationship between neuronal and behavioral responses to visual motion

    The Journal of Neuroscience 16:1486–1510.

    https://doi.org/10.1523/JNEUROSCI.16-04-01486.1996

    • PubMed
    • Google Scholar
86. 1. Shadlen MN
    2. Kiani R

    (2013) Decision making as a window on cognition

    Neuron 80:791–806.

    https://doi.org/10.1016/j.neuron.2013.10.047

    • PubMed
    • Google Scholar
87. 1. St John-Saaltink E
    2. Kok P
    3. Lau HC
    4. de Lange FP

    (2016) Serial dependence in perceptual decisions is reflected in activity patterns in primary visual cortex

    The Journal of Neuroscience 36:6186–6192.

    https://doi.org/10.1523/JNEUROSCI.4390-15.2016

    • PubMed
    • Google Scholar
88. 1. Stanislaw H
    2. Todorov N

    (1999) Calculation of signal detection theory measures

    Behavior Research Methods, Instruments, & Computers 31:137–149.

    https://doi.org/10.3758/BF03207704

    • Google Scholar
89. 1. Steiger JH

    (1980) Tests for comparing elements of a correlation matrix

    Psychological Bulletin 87:245–251.

    https://doi.org/10.1037/0033-2909.87.2.245

    • Google Scholar
90. 1. Sugrue LP
    2. Corrado GS
    3. Newsome WT

    (2004) Matching behavior and the representation of value in the parietal cortex

    Science 304:1782–1787.

    https://doi.org/10.1126/science.1094765

    • PubMed
    • Google Scholar
91. Book

    1. Sutton RS
    2. Barto AG

    (1998)

    Reinforcement Learning: An Introduction

    Cambridge: A Bradford Book.

    • Google Scholar
92. 1. Teodorescu AR
    2. Moran R
    3. Usher M

    (2016) Absolutely relative or relatively absolute: violations of value invariance in human decision making

    Psychonomic Bulletin & Review 23:22–38.

    https://doi.org/10.3758/s13423-015-0858-8

    • PubMed
    • Google Scholar
93. 1. Thura D
    2. Guberman G
    3. Cisek P

    (2017) Trial-to-trial adjustments of speed-accuracy trade-offs in premotor and primary motor cortex

    Journal of Neurophysiology 117:665–683.

    https://doi.org/10.1152/jn.00726.2016

    • PubMed
    • Google Scholar
94. 1. Treisman M
    2. Williams TC

    (1984) A theory of criterion setting with an application to sequential dependencies

    Psychological Review 91:68–111.

    https://doi.org/10.1037/0033-295X.91.1.68

    • Google Scholar
95. 1. Tsetsos K
    2. Chater N
    3. Usher M

    (2012) Salience driven value integration explains decision biases and preference reversal

    PNAS 109:9659–9664.

    https://doi.org/10.1073/pnas.1119569109

    • PubMed
    • Google Scholar
96. 1. Tsetsos K
    2. Pfeffer T
    3. Jentgens P
    4. Donner TH

    (2015) Action planning and the timescale of evidence accumulation

    PLoS ONE 10:e0129473.

    https://doi.org/10.1371/journal.pone.0129473

    • PubMed
    • Google Scholar
97. 1. Ulrich R
    2. Schröter H
    3. Leuthold H
    4. Birngruber T

    (2015) Automatic and controlled stimulus processing in conflict tasks: superimposed diffusion processes and delta functions

    Cognitive Psychology 78:148–174.

    https://doi.org/10.1016/j.cogpsych.2015.02.005

    • PubMed
    • Google Scholar
98. 1. Urai AE
    2. Braun A
    3. Donner TH

    (2017) Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias

    Nature Communications 8:14637.

    https://doi.org/10.1038/ncomms14637

    • PubMed
    • Google Scholar
99. Software

    1. Urai AE
    2. de Gee JW

    (2019) 2018_Urai_choice-history-ddm

    GitHub.

    https://github.com/anne-urai/2018_Urai_choice-history-ddm
100. Software

     1. Urai AE
     2. Wimmer K

     (2016) Spatiotemporal motion energy filtering: a Matlab implementation

     Zenodo.

     http://doi.org/10.5281/zenodo.594505
101. 1. Usher M
     2. McClelland JL

     (2001) The time course of perceptual choice: the leaky, competing accumulator model

     Psychological Review 108:550–592.

     https://doi.org/10.1037/0033-295X.108.3.550

     • PubMed
     • Google Scholar
102. 1. Voss A
     2. Rothermund K
     3. Brandtstädter J

     (2008) Interpreting ambiguous stimuli: separating perceptual and judgmental biases

     Journal of Experimental Social Psychology 44:1048–1056.

     https://doi.org/10.1016/j.jesp.2007.10.009

     • Google Scholar
103. 1. Wang XJ

     (2002) Probabilistic decision making by slow reverberation in cortical circuits

     Neuron 36:955–968.

     https://doi.org/10.1016/S0896-6273(02)01092-9

     • PubMed
     • Google Scholar
104. 1. Wetzels R
     2. Wagenmakers EJ

     (2012) A default bayesian hypothesis test for correlations and partial correlations

     Psychonomic Bulletin & Review 19:1057–1064.

     https://doi.org/10.3758/s13423-012-0295-x

     • PubMed
     • Google Scholar
105. 1. White CN
     2. Poldrack RA

     (2014) Decomposing bias in different types of simple decisions

     Journal of Experimental Psychology: Learning, Memory, and Cognition 40:385–398.

     https://doi.org/10.1037/a0034851

     • Google Scholar
106. 1. Wiecki TV
     2. Sofer I
     3. Frank MJ

     (2013) HDDM: hierarchical bayesian estimation of the Drift-Diffusion model in python

     Frontiers in Neuroinformatics 7:.

     https://doi.org/10.3389/fninf.2013.00014

     • PubMed
     • Google Scholar
107. Book

     1. Wilder M
     2. Jones M
     3. Mozer MC

     (2009)

     Sequential effects reflect parallel learning of multiple environmental regularities

     In: Bengio Y, Schuurmans D, Lafferty J. D, Williams C. K. I, Culotta A, editors. Advances in Neural Information Processing Systems 22. Curran Associates, Inc. pp. 2053–2061.

     • Google Scholar
108. 1. Wong K-F

     (2006) A recurrent network mechanism of time integration in perceptual decisions

     Journal of Neuroscience 26:1314–1328.

     https://doi.org/10.1523/JNEUROSCI.3733-05.2006

     • Google Scholar
109. 1. Wyart V
     2. Nobre AC
     3. Summerfield C

     (2012) Dissociable prior influences of signal probability and relevance on visual contrast sensitivity

     PNAS 109:3593–3598.

     https://doi.org/10.1073/pnas.1120118109

     • PubMed
     • Google Scholar
110. 1. Wyart V
     2. Koechlin E

     (2016) Choice variability and suboptimality in uncertain environments

     Current Opinion in Behavioral Sciences 11:109–115.

     https://doi.org/10.1016/j.cobeha.2016.07.003

     • Google Scholar
111. 1. Yu AJ
     2. Cohen JD

     (2008)

     Sequential effects: superstition or rational behavior?

     Advances in Neural Information Processing Systems 21:1873–1880.

     • PubMed
     • Google Scholar
112. 1. Zhang S
     2. Huang CH
     3. Aj Y

     (2014)

     Sequential effects: a bayesian analysis of prior bias on reaction time and behavioral choice

     Cognitive Science Society.

     • Google Scholar
113. 1. Zhang J
     2. Bogacz R

     (2010) Bounded Ornstein–Uhlenbeck models for two-choice time controlled tasks

     Journal of Mathematical Psychology 54:322–333.

     https://doi.org/10.1016/j.jmp.2010.03.001

     • Google Scholar

Author details

1. Anne E Urai

   1. Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
   2. Department of Psychology, University of Amsterdam, Amsterdam, Netherlands

   Present address

   Cold Spring Harbor Laboratory, Cold Spring Harbor, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing

   For correspondence

   anne.urai@gmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-5270-6513

2. Jan Willem de Gee

   1. Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
   2. Department of Psychology, University of Amsterdam, Amsterdam, Netherlands

   Present address

   Department of Neuroscience, Baylor College of Medicine, Houston, United States

   Contribution

   Formal analysis, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5875-8282

3. Konstantinos Tsetsos

   Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

   Contribution

   Formal analysis, Writing—review and editing

   Contributed equally with

   Tobias H Donner

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2709-7634

4. Tobias H Donner

   1. Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
   2. Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
   3. Amsterdam Brain and Cognition, University of Amsterdam, Amsterdam, Netherlands

   Contribution

   Conceptualization, Resources, Supervision, Writing—original draft, Writing—review and editing

   Contributed equally with

   Konstantinos Tsetsos

   For correspondence

   t.donner@uke.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7559-6019

• 6,377

  views

• 743

  downloads

• 151

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.