Introduction

Judging whether the world has changed is ubiquitous, from public health officials grappling with whether a pandemic surge has peaked, central banks figuring out whether inflation is easing, investors discerning whether the electric car market is getting traction, or romantic partners divining whether a relationship has soured. In all of these examples, individuals must update their beliefs that the world has changed based on a noisy signal, such as a drop in positive pandemic cases or a romantic partner’s suddenly mysterious behavior. In some cases, epidemiological or statistical models provide guidance. However, in many, if not most cases, the determination of whether a regime shift has occurred is made intuitively (Sanders & Manrodt, 2003).

We investigate intuitive judgments of regime-shift detection using a simple empirical paradigm (Massey & Wu, 2005; Seifert et al., 2023). Although this paradigm abstracts away some complications of real-world change detection, it maintains the most central features of the problem: normatively, regime-shift judgments reflect the signals from the environment as well as knowledge about the system that produces the signals. The most recent time series of inflation rates, pandemic cases, and sales of electric cars are all examples of signals. When pandemic cases continue to decline in recent weeks, one might infer a shift from pandemic to non-pandemic regime, only to learn a few weeks later that pandemic has resurged. Indeed, signals such as the latest pandemic cases are seldom precise indications of the true state of the world. Put differently, signals are, by and large, noisy. The noisier the signals are, the less diagnostic they are of the underlying regime.

In addition, signals are affected by how likely the regime shifts from one to another (transition probability). These two fundamental features or system parameters—the diagnosticity of the signals and transition probability—can be conceptualized as two independent aspects of the system that generates the signals. Previous works on regime-shift detection has found that people tend to overreact to change when they receive noisy signals (low signal diagnosticity) but nonetheless are in a stable environment (small transition probability). By contrast, precise signals (high signal diagnosticity) in an unstable environment (large transition probability) typically results in underreaction (Benjamin, 2019; Brown & Steyvers, 2009; Massey & Wu, 2005).

Massey and Wu (2005) proposed that over- and underreactions reflect system neglect—the tendency to respond primarily to signals and secondarily to the system parameters that produces the signals. The system-neglect hypothesis was derived from theoretical accounts of the determinants of confidence by Griffin and Tversky (1992). To explain system neglect, consider someone who is making judgments on whether a stock market had shifted from the bear to the bull market regime and has been given information about recent stock returns (signals), how frequent regime shifts happen (transition probability), and how similar the two regimes are (signal diagnosticity). If her judgments are solely based on the signals and not affected by transition probability and signal diagnosticity, she shows a complete neglect of the system parameters. Broadly, system neglect describes a lack of sensitivity—compared with normative Bayesian updating—to the system parameters. In the case of regime-shift detection, this leads to insufficient belief revision (i.e., underreaction) in diagnostic and unstable environments, where Bayesian updating requires a larger change in beliefs, and excessive belief change in noisy and stable environments (i.e., overreaction), where Bayesian updating calls for less pronounced belief revision. Empirical patterns akin to system neglect is not only observed in regime-shift detection, but also in other domains such as confidence judgments (Griffin & Tversky, 1992; Kraemer & Weber, 2004), demand forecasting (Kremer et al., 2011), and pricing decisions (Seifert et al., 2023). Under- and overreactions have been an active research topic in financial economics, often measured as reactions to stock market changes or firm news (Baker & Wurgler, 2007; Barberis et al., 1998; Daniel et al., 1998; De Bondt & Thaler, 1985; Nelson et al., 2001).

At the neurobiological level, change detection has been investigated in the context of reinforcement learning in dynamic environments where changes in the state of the world, such as reward distributions, take place during the experiments (Soltani & Izquierdo, 2019). Different behavioral paradigms, most notably reversal learning, and computational models were developed to investigate its neurocomputational substrates (Behrens et al., 2007; Izquierdo et al., 2017; Payzan-LeNestour et al., 2011, 2013; Nasser et al., 2010; McGuire et al., 2014). Key findings on the neural implementations for such learning include identifying brain areas and networks that track volatility in the environment (rate of change) (Behrens et al., 2007), participants’ beliefs about change (Payzan-LeNestour et al., 2011; McGuire et al., 2014; Kao et al., 2020), and their uncertainty about whether a change had occurred (McGuire et al., 2014; Kao et al., 2020). Evidence from several of the aforementioned studies (Behrens et al., 2007; McGuire et al., 2014; Kao et al., 2020) suggests that the dorsomedial frontal cortex (DMFC) is critical to learning in dynamic environments, as information about volatility, subjective beliefs and uncertainty about change converge in this brain region.

But how do biases in change detection arise in the brain? Although reinforcement learning studies provide valuable insights into change detection in the learning process, it remains unclear how biases in change detection—under- and overreactions to change—arise at the neural algorithmic and implementation levels. For example, it is unclear how a certain brain area, such as DMFC, that had been shown to represent environmental volatility, would contribute to under- and overreactions to change. In order to systematically characterize under- and overreactions, it would be critical to (1) adopt a well-established behavioral paradigm that robustly elicits these behavioral phenomena and (2) have computational frameworks suitable for developing neural hypotheses regarding under- and overreactions. To address these issues, in this study, we adopted the regime-shift detection task from Massey and Wu (2005) and their system-neglect computational framework. At the behavioral level, the regime-shift task is a well-established paradigm that robustly elicits under- and overreactions to change. At the algorithmic and implementation levels, the system-neglect framework provides a straightforward neurocomputational hypothesis regarding under- and overreactions. It predicts that for brain areas involved in regime-shift detection, under- and overreactions arise from their sensitivity or lack thereof in response to the system parameters.

We replicated previous behavioral findings on under- and overreactions (Massey & Wu, 2005). Using blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI), we reported three key findings. First, we identified two distinct brain networks involved in regime-shift detection, with the ventromedial prefrontal cortex (vmPFC) and ventral striatum in representing subjects’ reported beliefs about change and a frontoparietal network in evaluating the strength of change evidence. Second, we found that these two networks selectively respond to different system parameters: while the frontoparietal network represents individual subjects’ sensitivity to signal diagnosticity but not transition probability, the vmPFC shows the opposite pattern. Third, the neural sensitivity profiles were signal-dependent: frontoparietal network only represented individuals’ sensitivity to signal diagnosticity when signals consistent with change appeared. By contrast, vmPFC represented individuals’ sensitivity to transition probability regardless of whether subjects received signals consistent or inconsistent with change. Such signal-dependent representations led us to further examine and subsequently verify that they are indeed key properties of our system-neglect computational model. Together, these results suggest that regime-shift detection is implemented jointly by a belief-updating network (vmPFC-striatum) and evidence evaluation network (frontoparietal network) and that their sensitivity in response to different environmental parameters contribute to under- and overreactions to change. More broadly, we showed that neural data can reveal important properties of computational models that are overlooked in theoretical treatments and behavioral analyses.

Results

In our regime-shift detection task (Fig. 1A), in each trial, subjects saw a series of sequentially presented sensory signals (red or blue balls). They were told that the signals came from one of two regimes, the red regime or the blue regime (Fig. 1B). Regimes were symmetric, for example, with a red regime consisting of 60 red balls and 40 blue balls and the corresponding blue regime consisting of 60 blue balls and 40 red balls. Each trial started with the red regime but could shift to the blue regime before each of the 10 periods in a trial. After seeing a new signal in each period, subjects provided a probability estimate that the current regime was the blue regime, i.e., a posterior probability of a regime shift. They were also instructed that once the regime has shifted from the red to the blue during a trial, the regime would remain in the blue regime until the end of the trial, i.e., the blue regime was a trapping or absorbing state. Our experimental paradigm hence follows Massey and Wu (2005). Note that, during a trial, subjects did not receive feedback—after making probability estimates in each period—on whether the regime had shifted and the monetary bonus earned as a result of accuracy in probability estimates (see Methods for details). Hence, subjects had no access to information about accuracy and rewards as she or he was making probability estimates.

Figure 1.

We manipulated two system parameters, transition probability and signal diagnosticity (Fig. 1C). Transition probability, q, with possible values are 0.01, 0.05, and 0.1, specified the probability that the regime would shift from the red to the blue regime in any period. Signal diagnosticity, d, with possible values of 1.5, 3, and 9, captured the degree to which the two regimes differed. For example, an environment with high signal diagnosticity (e.g., d = 9) indicated that there were 9 times more red balls than blue balls in the red regime (a 90: 10 Red to Blue ratio) and 9 times more blue balls than the red balls in the blue regime (a 90: 10 Blue to Red ratio). Therefore, the weight that a signal (blue or red ball) carried was captured by the signal diagnosticity: in a low diagnostic environment, a blue signal most likely reflects no change in regime (d = 1.5, example on the left of Fig. 1D). By contrast, in a highly diagnostic environment, a blue signal very likely reveals a shift in regime (d = 9, example on the right of Fig. 1D). At the beginning of each trial, subjects were informed about the transition probability and signal diagnosticity in that trial. In the example trial sequence (Fig. 1A), the transition probability (indicated by “switch probability” in Fig. 1A) is 0.1 while the signal diagnosticity (indicated by “color ratio” in Fig. 1A) is 1.5, with the red regime consisting of 60 red balls and 40 blue balls and the blue regime consisting of 40 red balls and 60 blue balls.

To establish the neural representations for regime-shift estimation, we performed three fMRI experiments (n = 30 subjects for each experiment, 90 subjects in total). Experiment 1 was the main experiment, while Experiments 2 to 3 were control experiments that ruled out two important confounds (Fig. 1E). Experiment 2 was designed to establish whether the results from Experiment 1 was truly about regime-shift probability estimation or simply reflected probability estimation. In Experiment 1, the subjects had to estimate the probability that the current regime is the blue regime. Experiment 2 was identical to Experiment 1 except that the environments were stationary (no transition from one regime to another was possible) in Experiment 2 as in Edwards (1968) classic “bookbag-and-poker chip” studies. Subjects in both experiments had to estimate the probability that the current regime is the blue regime, but this estimation corresponded to the estimates of regime change only in Experiment 1. Therefore, activity that correlated with probability estimates in Experiment 1 but not in Experiment 2 can be uniquely attributed to representing the probability estimates of regime shift. Experiment 3 was specifically designed to address the motor confounds—whether neural signals correlated with probability estimates arose from motor preparation and/or execution, as the subjects made button presses to enter their probability estimates. By comparing the fMRI results of these experiments, we were therefore able to establish the neural representations that can be uniquely attributed to the probability estimates of regime-shift.

Behavioral evidence for over- and underreactions to change

Our analyses used subjects’ probability estimates of a regime shift, P_t, for each period, t = 1, …, 10. We found that subjects were in general responsive to the system parameters, with higher P_t when the transition probability was larger. Subjects also tended to provide more extreme P_t, P_t that were closer to 0 or 1, when the signal diagnosticity was large (Fig. S1 in Supplementary Information, SI). In addition, we used a measure of belief revision, Δ P_t = P_t − P_{t −1}. In Fig. 2A we show examples of P_t and ΔP_t from a subject. On the left, the subject was in a stable environment (small transition probability, q = 0.01) and faced two regimes that were very similar to each other (low signal diagnosticity, d = 1.5). The red and blue signals (10 periods) were what the subject encountered during a trial. On the right, the subject was in an unstable environment (q = 0.1) and faced two regimes that were very different (d = 9).

Figure 2.

To examine over- and underreactions to change, we compared subjects’ belief revision, ΔP_t = P_t P_t−1, t, = 2, …, 10, with belief revision predicted by the Bayesian model, (see Fig. 2A for illustrations). P_t and respectively capture how much subjects and a normative Bayesian change probability estimates in response to a new signal. When , it indicates larger belief revision than the normative Bayesian, i.e., an overreaction. By contrast, indicates smaller belief revision, i.e., an underreaction. We therefore use as an Index of Overreaction (IO). We found that subjects tended to overreact to change (IO > 0) when they received noisy signals (i.e., low signal diagnosticity, d = 1.5) and when the environment was stable (small transition probability, q = 0.01). By contrast, underreaction (IO < 0) was most commonly observed when they were in unstable environments (large transition probability, q = 0.1) and with clear signals (i.e., high signal diagnosticity, d = 9) (Fig. 2B). These patterns of over- and underreactions were consistent with findings in Massey and Wu (2005) and the system-neglect hypothesis, which posits a tendency to respond primarily to the signals and secondarily to the system that generates the signals (Massey & Wu, 2005; Seifert et al., 2023). According to the system-neglect hypothesis, responding secondarily to the system is synonymous with a lack of sensitivity to the system parameters, which leads to underreactions in unstable environments with precise signals, and overreactions in stable environments with noisy signals.

Following Massey and Wu (2005), we quantitatively model these belief revisions using the system-neglect model (see Methods). The model is a parameterized version of the normative Bayesian model that allows for distortion of the system parameters via weighting parameters for transition probability (α) and signal diagnosticity (β). In short, α reflects distortion of transition probability, with α × q in the system-neglect model capturing a decision maker’s effective transition probability (q). For example, if α = 4 when q = 0.01, the decision maker effectively treats a 0.01 transition probability as if it were 0.04. By contrast, β captures the extent to which the decision maker overweighs or underweighs signal diagnosticity (d^β) when faced with a signal. For example, if β = 2 when d = 1.5, subjects would treat a blue signal by updating the odds ratio for change by 1.5², or 2.25 rather than 1.5.

In the system-neglect model, we estimated the weighting parameters separately for each level of transition probability and signal diagnosticity, i.e., α_i × q_i and , where α₁, α₂, and α₃ correspond respectively to transition probabilities of 0.01, 0.05, and 0.1, and β₁, β₂, and β₃ correspond respectively to signal diagnosticity of 1.5, 3, and 9. In contrast to the Bayesian model which implies α_i = β_j = 1 for all i, j, the system-neglect model requires that that α_i > α_i+1 and β_j > β_j+1 because it would effectively capture a lack of sensitivity to the system parameters.

We fit the model to P_t for each subject separately and found parameter estimates consistent with system neglect (Fig. 2C). The mean estimates of α were 3.69, 1.04 and 0.65 respectively for q = 0.01, 0.05 and 0.10. The parameters indicated that, on average, when q = 0.01, subjects treated as if it were 0.0369. By contrast, when q = 0.10, the subjects treated it as if it were 0.065. Thus, a factor of 10 in actual transition probability (0.01 vs. 0.1) was reduced to a factor of less than 2 (0.0369 vs. 0.065) in effective transition probability. For signal diagnosticity, the mean parameter estimates of β were 1.69, 0.77 and 0.57 for d = 1.5, 3, and 9, respectively. Thus, subjects updated their beliefs 1.5^{1.6 9} = 1. 98 when d₁ = 1.5 and 9^0.57 = 3.50 when d₃ = 9. Normatively, the change in odds ratio between the two conditions should have been d₃/ d₁ = 6 but, consistent with system neglect, was considerably smaller, 3.50/1. 98 = 1.76. Together, large parameter estimates (α > 1, β > 1) at low signal diagnosticity (noisy signals) and low transition probability (stable environments) capture overreactions to changes, while small parameter estimates (α < 1, β< 1) at large signal diagnosticity (precise signals) and large transition probability (unstable environments) reflect underreactions to change. These results replicate the findings by Massey and Wu (2005), with the pattern of over- and underreactions as predicted by the system neglect hypothesis. Critically, the degree of system neglect can be captured by the negative trend of the parameter estimate as a function of the system parameter levels (Fig. 2C): the steeper the slope, the larger the system neglect. We found a similar pattern on β in Experiment 2 (one of the control experiments) where environments were stationary (no transition probability) and signal diagnosticity was manipulated (Fig. S2 in SI) (Benjamin, 2019; Tversky et al., 1990).

Finally, we performed a parameter recovery analysis to examine whether the fitting procedure gave reasonable parameter estimates (Wilson & Collins, 2019). First, we simulated each subject’s probability estimation data based on the system-neglect model by using that subject’s parameter estimates. Second, we fitted the system-neglect model to the simulated data. Third, we computed the correlation across subjects between the estimated parameters and the parameter values we used to simulate data. Fourth, we repeated the above steps by adding independent white noise to the simulated data. Across different levels of noise, we found good parameter recovery (Pearson’s D for transition probability r ≥ . 9 988 across different noise levels, Pearson’s r for signal diagnosticity r ≥ . 9 97 9 across different noise levels). (Figs. S3 and S4 in SI).

We also examine whether the way subjects respond to different system parameters is similar. It is possible, for example, that subjects who showed stronger (or weaker) distortion of transition probability (captured by α parameter) also showed stronger (or weaker) distortion of signal diagnosticity (captured by β parameter). There was no significant correlation between α and β parameters (Fig. 2D). However, we did find within-parameter correlation: subjects who had a higher α_i for a transition probability level i also tended to have a higher for a second transition probability level (Fig. 2E), with the same pattern also holding for signal diagnosticity (Fig. 2F). Together, these results suggested that the way an individual decision maker responds to information about the probability of change in the environment (transition probability) has little to do with how she or he responds to information about the similarity between different regimes (signal diagnosticity). But individuals are consistent in responding to a particular system parameter (transition probability or signal diagnosticity) across different levels of the parameter.

fMRI results

We focus our fMRI analyses on addressing three questions. First, what are the brain regions that correlated with subjects’ probability estimates of change and belief revision? Second, what are the neural representations for the computational variables contributing to these probability estimates? Third, how might neural responses in the identified brain areas be associated with under- and overreactions to change?

Ventromedial prefrontal cortex and ventral striatum represent probability estimates and belief revision on regime shift

Our first analysis is aimed at identifying brain regions that represented our subjects’ regime-shift estimation. To address this question, we used two behavioral measures, namely the period-by-period probability estimates of regime shift, P_t, and the change in P_t between successive periods, ΔP_t. P_t can be regarded as the subjects’ posterior probability estimates of regime shift, whereas ΔP_t captures the change in belief (belief revision) about regime shift in the presence of a new signal (see Fig. 3A for an example on P_t and ΔP_t).

Figure 3.

For P_t, we found that the ventromedial prefrontal cortex (vmPFC), ventral striatum, and many other brain regions, including the motor, insular, occipital cortices, and the cerebellum significantly correlated with P_t (Fig. 3B; see GLM-1 in Methods, and Tables S1 to S3 in SI, respectively, for significant clusters of activation using Gaussian random field theory, permutation test on threshold-free-cluster-enhancement statistic, and permutations test on cluster-extent statistic).

For ΔP_t, we also found that the vmPFC and ventral striatum were associated with regime shift belief revision (Fig. 3; see GLM-1 in Methods and Tables S1 to S3 in SI for significant clusters of activation). While many brain regions correlated with regime-shift probability estimates (P_t), only the vmPFC and ventral striatum also correlated with belief revisions, ΔP_t. We further examined the robustness of P_t representations in these two regions with different transformations of P_t: a GLM that replaced P_t with the log odds of P_t, ln (P_t/(1 − P_t)) (Fig. S5 in SI), a GLM (GLM-2 in Methods) that included various task-related variables as regressors (Fig. S6 in SI), and a GLM that examined P_t separately on periods when blue and red signals appeared (Fig. S7 in SI). Each of these analyses showed the same pattern of correlations between P_t and activation in vmPFC and ventral striatum.

Brain regions shown to correlate with regime-shift probability estimates, P_t, could be driven by motor response because larger estimates predominantly involved right-hand finger presses (see Methods for details). To rule out motor confounds, we conducted two control experiments (Experiments 2 and 3) and performed two analyses. First, we examined the neural correlates of probability estimates (P_t in GLM-1) in the control experiments (Experiments 2 and 3). Second, we compared the effect of P_t (GLM-1) between the main experiment (Experiment 1) and the control experiments. In the first analysis, we found that in both control experiments, vmPFC and ventral striatum did not significantly correlate with probability estimates at the whole-brain level (in Experiment 2, no significant clusters of activation at the whole-brain level; see Table S4 for Experiment 3 in SI). In the second analysis, we found that both vmPFC and ventral striatum correlated more significantly with P_t in Experiment 1 than Experiment 2 (Fig. 3C) and Experiment 3 (Fig. 3D; also see Tables S5 and S6 in SI). These results were subsequently confirmed in a separate independent ROI analysis on vmPFC and ventral striatum (Fig. S8 in SI). Finally, we note that in GLM-1, we implemented an “action-handedness” regressor to directly address the motor-confound issue, that higher probability estimates preferentially involved right-handed responses for entering higher digits. The action-handedness regressor was parametric, coding -1 if both finger presses involved the left hand (e.g., a subject pressed “23” as her probability estimate when seeing a signal), 0 if using one left finger and one right finger (e.g., “75”), and 1 if both finger presses involved the right hand (e.g., “90”). Taken together, these results suggested that vmPFC and ventral striatum represent subjects’ probability estimates of change (regime shifts) and belief revision.

A frontoparietal network represents key variables for estimating regime shifts

Our second analysis is aimed at identifying brain regions that represented key variables contributing to regime-shift estimation. Guided by our theoretical framework and computational models, we focused on two variables, the interaction between signals and signal diagnosticity and intertemporal prior probability of change, and set up a new GLM (GLM-2 in Methods) to examine these effects.

Our theoretical framework makes two fundamental predictions. First, a signal should be weighted differently depending on signal diagnosticity, i.e., a blue ball is stronger evidence for change in a highly diagnostic environment (e.g., d = 9) than a system in which the red and blue regimes are very similar (e.g., d = 1.5). To capture the interaction between signals and signal diagnosticity, we code a blue signal as 1 and a red signal as -1 and multiply the signal code (s =1 or -1) by the natural logarithm of signal diagnosticity, ln(d) (two examples are shown in Fig. 4A). We term this interaction, s × ln(d), the strength of evidence in favor of change or strength of change evidence for short. The Bayesian model, as described in Methods, critically depends on d^s, computing posterior odds by multiplying prior odds by the likelihood ratio. Thus, the log posterior odds were calculated from both the prior odds and s × ln(d). At the whole-brain level, we found that a frontoparietal network including the dorsal medial prefrontal cortex (dmPFC), lateral prefrontal cortex (bilateral inferior frontal gyrus, IFG), and the posterior parietal cortex (bilateral intraparietal sulcus, IPS) represented s × ln(d) (Fig. 4A). These brain regions overlap with what is commonly referred to as the frontoparietal control network (Buckner et al., 2013; Seeley et al., 2007; Yeo et al., 2011). Among them, dmPFC sits in the vicinity of dorsomedial frontal cortex (DMFC) shown to represent change probability and uncertainty about change in reinforcement learning (McGuire et al., 2014).

Figure 4.

The second prediction our theoretical framework offers concerns the prior probability of a regime shift over time. Specifically, the Bayesian model predicts that the prior probability should increase over time (see two examples in Fig. 4B), with the intertemporal prior, in log odds terms, defined as the natural logarithm of the odds in favor of regime shift in the t-th period, ln K, where q is transition probability and t = 1, …, 10 is the period (see Eq. 1 in Methods). We found that most brain regions in the frontoparietal network also correlated with the intertemporal prior (Fig. 4B; dmPFC: t(2 9) = −1.6 9, p = 0.10; left IFG: t(2 9) = 2.20, p = 0.04; right IFG: t(2 9) = −2.64, p = 0.01; left IPS: t(2 9) = −2.35, p = 0.03; right IPS: t(2 9) = −2.07, p = 0.05). In contrast, brain regions that represented the intertemporal prior, which we found to be in the right fusiform cortex in the occipitotemporal regions, did not correlate with the strength of change evidence (Fig. S9 in SI).

Finally, we emphasize that these effects—the strength of change evidence and intertemporal prior—cannot be otherwise attributed to probability estimates (P_t) or belief revision (ΔP_t) because both P_t and ΔP_t were included in GLM-2 where these effects were examined. Taken together, these results suggest that the frontoparietal network is critically involved in representing the two key variables for estimating regime shifts, strength of change evidence and intertemporal prior. See Supplementary Tables S7 to S9, respectively, for information about significant clusters of activation using Gaussian random field theory, permutation test on threshold-free-cluster-enhancement (TFCE) statistic, and permutation test on cluster-extent statistic.

Under- and overreactions are associated with selectivity and sensitivity of neural responses to system parameters

The system-neglect hypothesis posits that under- and overreactions arise from a lack of sensitivity to the system parameters. We can measure individual subjects’ sensitivity to system parameters using behavioral data (subjects’ probability estimates). Meanwhile, we can also measure sensitivity using neural data. In the following analysis, we examined whether there is a match between the behavioral and neural measures of sensitivity to the system parameters. This would allow us to examine, through the system-neglect framework, whether sensitivity in neural responses to the system parameters are associated with under- and overreactions to change.

We focused on the vmPFC-striatum network and frontoparietal network, as they were shown to be involved in regime-shift detection (Figs. 3 and 4). We examined whether these brain networks show selective preference for a particular system parameter, which we refer to as parameter selectivity. We also asked whether parameter selectivity is signal-dependent, i.e., different for signals suggesting change (blue) or no change (red).

We started by defining a behavioral measure of sensitivity to the system parameter. To visualize this measure, we consider two extreme decision makers, a Bayesian and someone who reacts to signals identically across all systems, which we term complete neglect. In Fig. 5A (left graph), we use signal diagnosticity (d) to illustrate the pattern of these two decision makers. The vertical axis is β ln(d) and the horizontal axis is the signal-diagnosticity level (d), where β is the weighting parameter on signal diagnosticity in the system-neglect model. A Bayesian (open circles) does not overweight or underweight d, and thus β = 1. We can then define the Bayesian slope by regressing β_i ln(d_i) against ln(d_i). In this formulation, the Bayesian slope is 1 and it reflects the sensitivity of a Bayesian decision maker to signal diagnosticity. On the other hand, a complete-neglect decision maker is unresponsive to signal diagnosticity, i.e.,β₁ ln(d₁) = β₂ ln(d₃) = β₃ ln(d₃). Hence, the complete-neglect slope should be 0. These two slopes, the Bayesian slope and the complete-neglect slope, provide the boundaries for system neglect. For each subject, we computed β_i ln(d_i) at each d_i level, where β_i is the estimate for diagnosticity d_i fitted to the system-neglect model (see β_i in Fig. 2B). We then estimated each subject’s behavioral slope (to distinguish it from the neural slope reported later) and use it as a behavioral measure of sensitivity to signal diagnosticity.

Figure 5.

For each subject, we estimated two behavioral slopes, one for d, the signal diagnosticity (top row in Fig. 5A), and the other for q, the transition probability (bottom row in Fig. 5A). The right graphs in Fig. 5A shows the behavioral slope for each of the 30 subjects (top: signal diagnosticity; bottom: transition probability). For signal diagnosticity, 28 out of 30 subjects’ behavioral slope were within the boundaries. For transition probability, 27 out of 30 subjects’ behavioral slope were within the boundaries (between complete neglect and Bayesian). One subject’s (subject 6) behavioral slope for q (transition probability) was approximately 2 and clearly outside the boundaries. This subject’s data were excluded for further analysis on q (the right two columns in Fig. 5CD).

We found that, across subjects, system neglect was unique from either normative Bayesian or complete neglect. Subjects’ sensitivity to transition probability, as captured by the behavioral slope in Fig. 5A, deviated significantly from the Bayesian slope (comparing subjects’ slope with 1, t(2 9) = −10.8, p < .01, two-tailed) and from complete neglect slope (comparing subjects’ slope with 0, t(2 9) = 4.8, p < .01, two-tailed). For signal dignosticity, subjects’ sensitivity to signal diagnosticity was also significantly different from both Bayesian (t(2 9) = −12.5, p < .01, two-tailed) and complete neglect (t(2 9) = 6.1, p < .01, two-tailed).

However, subjects were closer to complete neglect than to the normative Bayesian. We tested this by examining whether the behavioral slope, N, was significantly greater or smaller than 0.5, the midpoint between complete neglect (slope of 0) and Bayesian (slope of 1). γ − .5 > 0 indicates that subjects’ behavior was in closer alignment with Bayesian. By contrast, γ − .5 < 0 implies behavior closer to complete neglect. We found that, for both transition probability and signal diagnosticity, the behavioral slope was closer to complete neglect than to Bayesian (transition probability: t(2 9) = −2. 97, p < .01; signal diagnosticity: t(2 9) = −3.23, p < .01, two-tailed). Together, these results suggested that, while subjects did respond to the system parameters in regime-shift estimation in the correct direction predicted by the Bayesian model, their sensitivity to the system parameters was closer to complete neglect than to normative Bayesian.

For the neural data, we defined a neural measure of sensitivity to the system parameters by estimating how neural responses change as a function of those parameters. Using the signal diagnosticity parameter as an example, for each subject and each ROI separately, we regressed average brain activity at each diagnosticity level against ln(d). The slope estimate, termed the neural slope, from the linear regression gave us a neural measure of sensitivity to signal diagnosticity. To investigate whether the neural sensitivity was signal-dependent, i.e., neural sensitivity in response to the blue signals (consistent with change) was different from the red (inconsistent with change) signals, we separately estimated the neural slope in response to blue and red signals.

After obtaining both the behavioral and neural measures of sensitivity to the system parameters, we then computed the Pearson correlation coefficient between them. We found that the vmPFC-striatum network and frontoparietal network showed clear dissociations in how they corresponded with the system parameters. First, the frontoparietal network represented individual subjects’ sensitivity to signal diagnosticity (left two columns in Fig. 5C), but not transition probability (right two columns in Fig. 5C). Notably, patterns of parameter selectivity were remarkably consistent across brain regions in the frontoparietal network: when blue signals (consistent with change) appeared, the neural measure of sensitivity from all brain regions in the frontoparietal network except the right IPS significantly correlated with the behavioral measure of sensitivity (second column from the left in Fig. 5B; dmPFC: r = 0.48, p = 0.007; lIFG: r = 0.5, p = 0.00 9; rIFG: r = 0.4, p = 0.027; lIPS: r = 0.58, p = 0.001; rIPS: r = 0.32, p = 0.082). By contrast, when red signals (inconsistent with change) appeared, all regions within the frontoparietal network did not significantly correlate with the behavioral measure of sensitivity (first column from the left in Fig 5B; dmPFC: r = 0.32, p = 0.083; lIFG: r = 0.04, p = 0.848; rIFG: r = 0.1 9, p = 0.312; lIPS: r = 0.05, p = 0.787; rIPS: r = −0.02, p = 0. 914).

Second, in contrast to the frontoparietal network, vmPFC in the vmPFC-striatum network showed the opposite pattern of parameter selectivity: vmPFC selectively represented individual subjects’ sensitivity to transition probability (right two columns in Fig. 5D), but not signal diagnosticity (left two columns in Fig. 5D). Unlike the frontoparietal network, selectivity in vmPFC was not signal-dependent: regardless of red or blue signals, it represented sensitivity to transition probability (r = −0.38, p = 0.043 for red signals; r = −0.37, p = 0.047 for blue signals). For the ventral striatum, it did not show selectivity to either the transition probability or signal diagnosticity (transition probability: r = −0.26, p = 0.175 for red signals; r = 0.03, p = 0.870 for blue signals; signal diagnosticity: r = 0.33, p = 0.077 for red signals; r = 0.27, p = 0.150 for blue signals). Therefore, only vmPFC in the vmPFC-striatum network showed parameter selectivity.

fMRI results revealed hidden properties of the system-neglect model

A key insight emerging from Fig. 5 was an asymmetry in the signal-dependent nature of neural sensitivity to the system parameters: neural sensitivity to signal diagnosticity was signal-dependent, whereas neural sensitivity to transition probability was not. For signal diagnosticity, the frontoparietal network represented individual subjects’ behavioral measure of sensitivity only when signals in favor of change (blue signals) appeared. By contrast, the vmPFC—shown to represent sensitivity to transition probability—did not show signal dependency. Neural sensitivity to transition probability in vmPFC, measured separately at the time of blue signals and red signals, both correlated with individuals’ behavioral measure of sensitivity to transition probability.

This neural insight makes two important predictions about the system-neglect model. First, it predicts that the β parameter estimates (for signal diagnosticity) are driven primarily by subjects’ probability estimates in response to blue signals (consistent with change), but not by probability estimates in response to red signals (inconsistent with change). Second, it predicts that the α parameter estimates (for transition probability) are driven by both kinds of signals.

To examine these two predictions, we started by estimating how subjects responded to the system parameters in a model-free fashion, that is, without implementing any computational model. To do that, for each subject separately, we performed multiple regression with probability estimates as data (see model-free sensitivity analysis in Methods). The regression model estimated, separately for the blue and red signals, how probability estimates changed as a function of transition probability and signal diagnosticity. We referred to these estimates as the model-free behavioral slopes. In contrast to the model-free behavioral slope, the behavioral slopes in Fig. 5 were model-based because they were obtained based on the system-neglect model. Hence, we also referred to them as the model-based behavioral slopes (Fig. 6). If our neurally inspired predictions were true, the model-based slopes for signal diagnosticity would significantly correlate only with the model-free slopes estimated in response to blue signals. By contrast, model-based slopes for transition probability would correlate with model-free slopes estimated in response to both blue and red signals.

Figure 6.

Consistent with our predictions, we found that model-based behavioral slopes for signal diagnosticity correlated with model-free behavioral slopes estimated at blue signals but not red signals (Fig. 6A). By contrast, model-based slopes for transition probability correlated with model-free slopes estimated both blue and red signals (Fig. 6B). Similar results were obtained by performing the same analysis on simulated probability estimation data (Fig. 6CD). Together, these results indicated that neural insight from fMRI results revealed “hidden” properties of the system-neglect model that were not obvious in the model setup and were overlooked in our previous behavioral analyses. Specifically, the hidden properties are that the β parameter estimate (capturing distortion of signal diagnosticity) in the system-neglect model was primarily driven by subjects’ responses to signals consistent with change, whereas the α parameter estimate (capturing distortion of transition probability) was driven by responses to both signals consistent with change (blue signals) and signals inconsistent with change (red signals).

Discussion

We investigated the neurocomputational substrates for regime-shift detection and mechanisms that give rise to under- and overreactions to change. Regime shifts—the transition from one state of the world to another—are present in many daily situations, from the stock market (a change from the bull to the bear market) to the state of a pandemic. Detecting regime shifts can be challenging for at least two reasons. First, the signals we receive from the environments are often noisy. A signal in favor of potential change, for example a drop in pandemic cases, can either inform a true shift in regime or simply reflect noisy fluctuations. Second, the signals we receive reflect the volatility of the environment: while some environments are more prone to changes, others are not. To capture these two key features in regime-shift detection, we designed an fMRI task based on Massey and Wu (2005) where subjects made probability judgments about regime shifts and where we manipulated the signal diagnosticity and transition probability. Signal diagnosticity captures the level of noise inherent in the signals, while transition probability reflects the volatility of the environment. Replicating Massey and Wu (2005), we found that overreactions to regime shifts take place when participants received noisy signals (low signal diagnosticity) but when the environments were stable (low transition probability). By contrast, when the signals are more precise but the environments were unstable, participants tended to underreact to changes. These results suggest system neglect— people respond primarily to signals and secondarily to the system that generates the signals (Massey & Wu, 2005).

At the neurobiological level, we found that regime-shift detection is jointly implemented by two networks, the vmPFC-striatum network and a frontoparietal network. The vmPFC-striatum network represented subjects’ probability estimates of change and the revision of probability estimates in the presence of new signals (belief revision). By contrast, the frontoparietal network represented strength of change evidence and intertemporal prior probability of change—two key variables contributing to probability estimation. Guided by the system-neglect framework, we found that under- and overreactions to change are closely associated with the sensitivity of these networks in response to the system parameters—transition probability and signal diagnosticity—that impact regime changes. In particular, the vmPFC represented individual subjects’ sensitivity to transition probability, whereas the frontoparietal network represented sensitivity to signal diagnosticity. Together, these findings suggest that selectivity and sensitivity of neural responses to system parameters are key mechanisms that give rise to under- and overreactions.

Our work is closely related to the reversal-learning paradigm—the standard paradigm in neuroscience and psychology to study change detection. In a typical reversal-learning task, human or animal subjects choose between two options that differ in the value or probability of receiving a reward. Through reward feedback the participants gradually learn the reward contingencies associated with the options and have to update the contingencies when they switched in order to maximize rewards. While a switch in reward contingencies can be regarded as a kind of regime shift, there are three major differences between the reversal-learning paradigm and our regime-shift task. The first difference is about learning. In the reversal-learning paradigm, the subjects must learn both the reward contingencies and the switches through experience. By contrast, in the regime-shift task, the subjects were explicitly informed about the makeup of different regimes and the transition probability. Therefore, participants do not need to learn about the different regimes and the transition probability through experience. The second difference is the kind of behavioral data collected. In our task, we asked the subjects to estimate the probability of change, whereas in the reversal-learning task, the subjects indicate their choice preferences. The third difference is on reward contingency. In the reversal-learning task, a change is specifically about change in reward contingencies, which is not the case in our task.

We believe that these major differences in task design led to three key insights into change detection from our study. The first insight is on over- and underreactions to change. At the behavioral level, we were able to identify situations that led to over- and underreactions. At the theoretical level, we were able to provide a systematic account for these over- and underreactions in the system-neglect hypothesis. Finally, at the neurobiological level, we were able to quantify the degree to which individual subjects neglected the system parameters and use these behavioral measures to unravel the neural mechanisms that give rise to over- and underreactions to change.

The second insight is on the brain networks associated with change detection. In particular, we were able to clarify whether the neural systems involved in change detection in the reversal-learning tasks are contingent on whether rewards are involved. Since the reversal-learning tasks are about learning the reward contingencies and the change in reward contingencies, it would be challenging to infer whether the neural implementations of change detection are dissociable from reward processing. Indeed, brain regions shown to be involved in the reversal-learning tasks, the OFC, mPFC, striatum, and amygdala, were also found to be highly involved in reward-related learning and value-based decision making. In the current study, unlike in reversal-learning paradigms, regimes were not defined by rewards (e.g., high reward-probability regime vs. low reward-probability regime in reversal learning paradigm). Therefore, estimating the probability of regime shifts in our task did not require considerations for change in reward contingencies. Our findings that vmPFC and ventral striatum represent probability estimates of change and belief revision therefore suggest that these brain regions might be part of a common pathway for change detection in general where changes in the state of the world do not have to be about changes in reward contingencies.

The third insight has to do with the impact of learning on change detection. Under the reversal-learning paradigm, it has been challenging to infer whether there exists a unique change-detection mechanism that is dissociable from reinforcement learning mechanisms. The way to make such inference is through theory, such as implementing a prior for state changes (Bartolo & Averbeck, 2020; Costa et al., 2015). Unlike the reversal-learning task, participants in our task did not have to learn about the different regimes through experience. Without the confound of reinforcement learning, our results help clarify the roles of change detection on choice behavior by suggesting that independent of learning, there exists a specialized change-detection mechanism in the brain that impacts decision making. This mechanism involves the participation of the vmPFC-striatum network and the frontoparietal network, which partially overlap with the brain regions involved in reversal learning. However, it remains to be seen how learning interacts with change detection. Future investigations can address this question by combining the key features of both the reversal-learning paradigm and regime-shift paradigm.

Outside of the reversal-learning paradigm, previous fMRI studies that investigated learning and belief updating in dynamic environments where change takes place regularly had identified brain regions that represent perceived likelihood of change inferred from participants’ choice behavior. Payzan-LeNestour et al. (2013) identified that the posterior cingulate cortex, postcentral gyrus, middle temporal gyrus, hippocampus, and insula correlated with subjects’ perceived likelihood of change in a multi-arm bandit task. McGuire et al. (2014) found that subjective change-point probability was represented in a large posterior cluster including occipital, inferior temporal, and posterior parietal cortex. In addition, activity in dorsomedial frontal cortex, posterior cingulate cortex, superior frontal cortex, and anterior insula also positively correlated with change probability. Interestingly, both McGuire et al. (2014) and our results found that the ventral striatum negatively correlated with probability estimates of change. This result suggested that ventral striatum represents probability estimates of change irrespective of whether the task was based on a learning-based paradigm (McGuire et al., 2014) or non-learning paradigm where information about task-related variables was explicitly revealed to the participants. Further, both McGuire et al. and our results found the involvement of the dorsomedial prefrontal cortex (dmPFC; or dorsomedial frontal cortex in McGuire et al.) in change detection. Our results further suggest that dmPFC is specialized in weighing the strength of change evidence and represents individual subjects’ sensitivity to signal diagnosticity, both of which played important roles in contributing to the over- and underreactions to change.

How might our results relate to value-based decision making? In previous studies, vmPFC had been implicated to dynamically track financial risks that carry potential monetary gains or losses. To understand dynamic computations of risk, Schonberg et al. (2012) used a Balloon Analog Risk Task (BART) where subjects decide whether to inflate a simulated balloon through successive pumps for the potential to win larger gains or incur larger losses (if balloon explodes), or to cash out before the balloon explodes. They found that vmPFC activity decreased as subjects pumped and expanded the balloon, suggesting its involvement in estimating the risk of potential losses. Since the explosion of balloon can be regarded as a change in the state of the balloon, as the balloon expands the possibility of such a change in state (regime shift) also increases. In this view, the vmPFC result from Schonberg et al. (2012) was consistent with our finding in that vmPFC negatively correlated with probability estimates of regime shift. Together, these results add to the existing literature by suggesting that vmPFC is involved in estimating and updating the state of the world in dynamic environments where changes take place regularly.

Our results are also closely related to the literature on the neural mechanisms for evidence accumulation in decision making (Gold & Shadlen, 2007; Mante et al., 2013; Philiastides et al., 2010; Roitman & Shadlen, 2002; Yates et al., 2017). In our task, evaluating the signals (red or blue balls) and in particular, the strength of change evidence associated with the signals is central to performing the task. Normatively, such evaluation should depend on the signal diagnosticity. In a highly diagnostic environment, seeing a red ball should signal a strong possibility of being in the red regime, while seeing a blue ball should signal otherwise. By contrast, in a low diagnostic environment, a red (resp. blue) ball is not strongly indicative of a red (resp. blue) regime. Hence, the evaluation of signals should reflect the interaction between the signals and the diagnosticity of the signals.

We found that this key computation was implemented in a frontoparietal network commonly referred to as the frontoparietal control network (Buckner et al., 2013; Dosenbach et al., 2007; Seeley et al., 2007; Vincent et al., 2008; Yeo et al., 2011). This network was proposed to support adaptive control functions, including initiating control and providing flexibility to adjust the level of control through feedback (Dosenbach et al., 2007). The IPS and dlPFC, part of this network, have also been found to play a major role in the top-down control of attention (Corbetta & Shulman, 2002; Woldorff et al., 2004). In perceptual decision making, the IPS and dlPFC were also shown to represent the accumulation of sensory evidence that leads to the formation of a perceptual decision (Heekeren et al., 2004; Heekeren et al., 2006). Our findings—that activity in this network does not reflect just the sensory signals (red or blue balls) but how these signals should be interpreted through the lens of their diagnosticity—highlights the involvement of the frontoparietal control network in computing the strength of evidence through combining information about sensory signals and knowledge about the precision of those signals.

Our result on the intraparietal sulcus (IPS) being part of the brain network that represents diagnosticity-weighted sensory signals are consistent with previous studies showing that IPS is involved in accumulating sensory evidence over time (Gold & Shadlen, 2007). There are three interesting aspects of our data that add to the current literature on evidence accumulation. First, IPS representations for sensory evidence need not be in the space of actions. Unlike previous studies showing that IPS represents sensory evidence for potential motor actions, we found that IPS represents the strength of evidence in favor of or against regime shifts. This result points to a more general role of the IPS in estimating the strength of sensory evidence. In fact, our result suggests that it depends on the task goal, which in the current study is to estimate whether a change has taken place. Second, although evidence accumulation is important and necessary for a wide array of cognitive functions, it is not a central requirement for the regime-shift task. Bayesian updating—the framework in which our system-neglect model was built upon—only requires the computation of strength of change evidence associated with the signal shown in the latest period. By showing that IPS representing this quantity, this suggests that IPS is involved in evaluating the latest piece of evidence necessary for belief updating.

In the current study, the central opercular cortex—in addition to the vmPFC—is another brain region that represented the probability estimates of change. Like the vmPFC, activity in this region negatively correlated with the probability estimates of change. This finding is associated with previous findings on change detection using the oddball paradigm. Using the oddball paradigm, it was found that the central opercular cortex is involved in the detection of change, showing stronger activation in blocks containing only the standard stimulus than blocks containing both the standard and deviant stimulus (Hedge et al., 2015) and correlating with ERP P3 signals at the trial-level that reflected differences between standard and deviant stimuli (Warbrick et al., 2009). There are two implications here. First, our findings suggest that the central opercular cortex is not only involved in the detection of change—as revealed by the oddball tasks—but also is involved in the estimation of change where there is uncertainty regarding whether the state of the world had changed. Second, the central opercular cortex may be part of a common pathway for the detection of change across very different tasks such as the oddball paradigm and our regime-shift detection task.

In summary, our results suggest that an important mechanism for under- and overreactions to change has to do with neural sensitivity to system parameters that impact regime shifts. Importantly, different system parameters appear to recruit distinct brain networks according to their unique computational specializations. Given that under- and overreactions underly a wide array of human judgments, our findings indicate that network-level computational specificity and parameter selectivity are two key building blocks that give rise to human judgment biases.

Materials and Methods

The data and analysis code are available at https://osf.io/xh7dy/.

We performed three fMRI experiments (90 subjects in total, 30 subjects for each experiment). Experiment 1 was the main experiment where we investigated the neurocomputational substrates for regime shifts. Experiments 2 and 3 were control experiments. Experiment 2 was designed to rule out brain activity that correlated with probability estimates but was not specifically about regime shifts. Experiment 3 attempted to rule out brain activity that correlated with entering numbers through button presses. In the main text, we only presented the results from Experiment 1. The procedure and results of Experiments 2 and 3 were presented in Supplementary Materials.

Subjects

All subjects gave informed written consent to participate in the study. All subjects were right-handed. The study procedures were approved by the National Yang Ming Chiao Tung University Institutional Review Board. Ninety subjects participated in this study:

• Experiment 1: n = 30 subjects; 15 males; mean age: 22.9 years; age range: 20 to 29 yrs.

• Experiment 2: n = 30 subjects; 15 males; mean age: 23.3 years; age range, 20 to 30 years.

• Experiment 3: n = 30 subjects; 15 males; mean age: 23.7 years; age range: 20 to 34 years.

Subjects were paid 300 New Taiwan dollar (TWD, 1 USD = 30 TWD) for participating the behavioral session and 500 TWD for the fMRI session. Subjects received additional monetary bonus based on his or her performance on probability estimation in Experiments 1 and 2 (Experiment 1: an average of 209 and 212 TWD for the behavioral and fMRI session respectively; Experiment 2: an average of 223 and 206 TWD for the behavioral and fMRI session respectively). In Experiment 3, subjects received the bonus based on their performance for entering the correct number (an average of 243 TWD for the fMRI session).

Procedure

Overview

Experiment 1 consisted of two sessions—a behavioral session followed by an fMRI session—that took place on two consecutive days. Subjects performed the same task in both sessions. The goals of having the behavioral session were to familiarize subjects with the task and to have enough trials—along with the fMRI session—to reliably estimate the parameters of the system-neglect model. Details of Experiments 2 and 3 can be found in the Supplement.

Regime-shift detection task

In this task, the environments the subjects are described as regimes. There were two possible regimes and at any point in time the regime can shift from one to another. Subjects judged whether the regime had shifted based on three sources of information: transition probability, signal diagnosticity, and signals. Prior to each trial, the subjects were given information about the transition probability and signal diagnosticity of the upcoming trial. The transition probability described how likely it is that the regime can shift from one to another. The signal diagnosticity described how different the two regimes are. The two regimes—red and blue—were represented by a red urn and a blue urn respectively, with each urn consisting of red and blue balls. The red urn had more red balls and the blue urn had more blue balls. The two urns always shared the same ratio of the dominant ball color to the dominated ball color. That is, the ratio of the red balls to the blue balls in the red urn was same as the ratio of the blue balls to the red balls in the blue urn. Signal diagnosticity was quantitatively defined by this ratio. For example, when the ratio was 9:1, signal diagnosticity (d) was 9. Under this definition, larger signal diagnosticity indicated that the two regimes were more different from each other.

After information about the transition probability and signal diagnosticity were revealed, 10 sensory signals—red or ball balls shown on the computer screen—were sequentially presented to the subjects. Each signal was sampled with replacement from one of the two urns. We also referred to the sequential presentation of these signals as periods. By design, each trial always started at the red regime, but the regime can shift from the red to the blue regime at any point in time during a trial, including prior to the sampling of the first signal. In addition, also by design, the regime can shift only once in a trial (i.e., the blue regime was an absorbing or trapping state). That is, once the regime changed from the red to the blue regime, signals would be sampled from the blue urn until the end of the trial. Subjects provided an estimate of the probability that current regime was the blue regime at each period when a new signal was presented. Since the regime can only shift from the red to the blue, this probability estimate was equivalent to the probability estimate of regime shift.

Stimuli

For each subject and each trial separately, we generated the stimuli, i.e., the sequence of red and blue balls according to the transition probability and signal diagnosticity of the trial. Before the start of each period, we first determined whether the regime would shift from the Red regime (the starting regime) to the Blue regime by sampling from the transition probability. There were two possible outcomes: 1 indicates a change in regime, whereas 0 indicates no change. If the outcome were 1, we would sample from the Blue regime for that period and for all the remaining period(s). If the outcome were 0, we would sample from the Red regime for that period and repeat the same process described above for the next period.

Manipulations of transition probability and signal diagnosticity

We implemented a 3 × 3 factorial design where there were three possible transition probabilities, denoted as 2 where 2 = [0.01, 0.5, 0.1] and three levels of signal diagnosticity, denoted as q where q = [1.5, 3, 9]. The transition probability—the probability that the regime shifts from the red to the blue regime—was used to simulate the stability of the environment. Larger transition probabilities indicate less stable environments. The signal diagnosticity was the ratio of dominant balls to the dominated balls in the urns. For example, when d = 9, it indicates that the red balls are 9 times more than the blue balls in the red urn and that the blue balls are 9 times more than the red balls in the blue urn. In this case, the two regimes were highly different and hence the signals shown to the subjects should be highly diagnostic of the regime the subjects were currently in.

Session 1: behavioral session

A pre-test block followed by the main task (8 blocks of trials) were implemented in the behavioral session. We implemented the pre-test block to make sure the subjects understood the experimental instructions and to train them to become familiar with the task, especially the time limit in entering probability estimates (see Trial sequence below for details on the pre-test block). A 3 (transition probability) by 3 (signal diagnosticity) factorial design was implemented, resulting in a total of 9 experimental conditions. Each block consisted of 9 trials (one for each condition, randomized in order) and each trial consisted of 10 periods or equivalently, 10 sequentially presented signals. After the pre-test block, subjects performed the main task that consisted of 8 blocks of trials. The behavioral session took approximately 70 minutes to complete.

Trial sequence

At the beginning of each trial, the subjects were shown information about the transition probability and signal diagnosticity (3s) (Fig. 1A). This was followed by the sequential presentation of 10 signals. At each period, a signal—a red or blue dot sampled from the current regime was shown. Subjects’ task was to estimate the probability that the current regime was the blue regime within 4s. The subjects indicated the probability estimate through two button presses. During the experiment, the subjects placed his or her 10 fingers on a 10-button keypad. Each finger was associated with a single button that corresponded to a unique integer value from 0 to 9, with the left little finger for 1, left ring finger for 2, left middle finger for 3, left index finger for 4, left thumb for 5, right thumb for 6, right index finger for 7, right middle finger for 8, right ring finger for 9, and right little finger for 0. To enter the probability estimate, the subjects had to first press the number corresponding to the tens and second the number corresponding to the ones. For example, if the probability estimate was 95%, the subjects first had to press the “9” button and second the “5” button. Once the subjects entered the probability estimate, she or he was not allowed to change it. After providing the probability estimate, they were given a brief feedback (0.5s) on the number they just entered. If they failed to indicate probability estimate within the time limit (4s), a “too slow” message would be shown. At the end of each trial, the subjects received feedback (2s) on the amount of monetary bonus earned in the trial (2s) and information about whether the regime shifted during the trial. If the regime was shifted, the signal that was drawn right after the shift took place would be highlighted in white. This was followed by a variable inter-trial interval (ITI, 1s to 5s in steps of 1s drawn from a discrete uniform distribution).

We implemented a pre-test block at the beginning of the behavioral so as to train the subjects to enter the probability estimate within the time limit (4s). In the pre-test block, we started with a lenient time limit and gradually decreased it. The time limit for the first three trials was 20s, 10s from trial number 4 to 6, and 4s from trial number 7 to 9. Recall that in each trial, the subjects encountered 10 sequentially presented signals (red and blue balls) and had to provide probability estimate at each period. The subjects therefore entered 30 probability estimates under each of these three time limits—from 20s, 10s, to 4s. After the pre-test block, all the subjects were able to enter the probability estimates within 4s.

Session 2: fMRI session

The fMRI session consisted of 3 blocks (9 trials in each block, with each trial consisting of 10 sequentially presented signals). The task was identical to the behavioral session (except with varying inter-stimulus interval for the purpose of fMRI analysis) and took approximately 60 minutes to complete. The subjects indicated the probability estimate through two button presses. During the experiment, the subjects placed his or her 10 fingers on a 10-button keypad. Each finger was associated with a single button that corresponded to a unique integer value from 0 to 9 (starting from the left pinkie for 1, left ring finger for 2, left middle finger for 3, and etc., to right ring finger for 9, and finally right pinkie for 0). The trial sequence was identical to the behavioral session with a few exceptions. First, each new signal was presented for 4s regardless of when subjects make a response. Second, we added a variable inter-stimulus interval (ISI, 1s to 5s in steps of 1 drawn from a uniform distribution) between two successive signals. We also made the range of ITI to be slightly larger (1s to 7s drawn from a discrete uniform distribution between in steps of 1s) than the behavioral session. The design of variable ISIs and ITIs was to allow better dissociations between events, i.e., between different sensory signals presented during a trial, and between trials for fMRI analysis.

Monetary bonus

To incentivize subjects to perform well in this experiment, they received monetary bonus based on his or her probability estimates. The bonus rule was designed so that the subjects who gave more accurate estimates would earn more bonus. The bonus structure used a quadratic payoff:

where P_t is the probability estimate that the current regime was blue at the t-th period in a trial and B_t is the regime at t (B_t = 1 for the blue regime and B_t = 0 for the red regime). For each probability estimate, the bonus therefore ranged from winning $3 to losing $3 TWD. For example, if the subject gave a 99% probability estimate that the current regime was blue and the current regime was indeed the blue regime, she would receive a bonus close to 3 TWD. By contrast, if the subjects gave a 1% probability estimate that the current regime was blue but the current regime was the red regime, she would receive a penalty close to 3 TWD. With 10 probability estimates given in each trial, the subjects can therefore receive a bonus up to 30 TWD or a penalty up to 30 TWD in a trial. The subjects did not receive feedback on bonus after each probability estimate. Instead, at the end of each trial, the subjects received information about the total amount won or lost in that trial. The final total bonus was realized by randomly selecting 10 trials at the end of the experiment.

Computational models for regime shift

We examined two computational models for regime shift: the Bayesian model and the system-neglect model (Massey & Wu, 2005). The Bayesian model was parameter-free and was used to compare with subjects’ probability estimates of regime shift. The system-neglect model is a quasi-Bayesian model or parameterized version of the Bayesian model that was fit to the subjects’ probability-estimate data. The parameter estimates of the system-neglect model were further used in the fMRI analysis so as to identify neural representations for over- and underreactions to change.

Bayesian model

Here we describe the Bayesian posterior odds of shifting to the blue regime given the period history H_t (Edwards, 1968; Massey & Wu, 2005):

is the posterior probability that the regime has shifted to the blue regime at the t-th period, H_t denotes the sequence of history from r₁ to r_t. Here r_t donates the t-th period, where r_t = 1 when the signal at t is red, and r_t = 0 when the signal at t is blue. The transition probability and signal diagnosticity are denoted by q and d respectively.

The posterior odds are the product of the prior odds and the likelihood ratio. The prior odds,, indicates that given the transition probability q, the probability that the regime has shifted at time t, 1 − (1 − q)^t, relative to the probability of no change in regime, (1 − q)^t. The likelihood ratio, indicates the probability of observing the history of signals H_t given that the regime has shifted relative to the probability given that the regime has not shifted. This requires considering all the possibilities on the timing of shift, i.e., the likelihood ratio that the regime was shifted at weighted by its odds .Since these possibilities are disjoint events, the likelihood ratio that the regime has shifted is simply the weighted sum of all the likelihood ratios associated with these disjoint possibilities.

Index of overreaction to change

To quantify under- and overreactions to change, we derived an Index of Overreaction (78). In short, this index reflects the degree to which the subjects overreacted or underreacted to change such that an index value greater than 0 indicates overreaction and an index value smaller than 0 suggests underreaction. To compute IO, we compared the change in probability estimates between two adjacent periods (ΔP_t = P_t − P_t−1) with the normative change in probability estimates according to the Bayesian model . Here we use P_t to denote the subject’s probability estimate at the t-th period and to denote the Bayesian probability estimate. We computed separately for each subject and for each condition or combination of transition probability and signal diagnosticity. Overreaction is defined when the actual change in probability estimates is greater than the normative change in probability estimates . By contrast, underreaction is defined when the actual change is smaller than the normative change in probability estimate . When , there neither overreaction nor underreaction to change.

System-neglect model

Following Massey and Wu (2005), we fit the system-neglect model—a quasi- Bayesian model—to the subjects’ probability estimates. In this model, we aimed to capture system-neglect—that people respond primarily to the signals and secondarily to the system generating the signals. Responding secondarily to the system indicates that people with system-neglect would be less sensitive to the changes in the system parameters compared with the normative Bayesian. This is captured by adding a weighting parameter to each system parameter level. Hence, two weighting parameters, α and β, were added to Eq. (1) to transition probability and signal diagnosticity respectively such that

where P_t is the probability estimate that regime has changed to the blue regime at period t. We separately estimated α for each level of transition probability and β for each level of signal diagnosticity. This was implemented by setting dummy variables for each level of transition probability and signal diagnosticity in Eq. (2)

Where Q_i is the dummy variable for transition probability q_i and D_j is the dummy variable for diagnosticity d_j. α_i and β_j therefore respectively reflect the sensitivity to different levels of transition probability and signal diagnosticity. For each subject separately, we performed nonlinear regression (using the fitnlm function in MATLAB) to fit the model to the subject’s probability estimates and estimated the parameters of interest.

Parameter recovery analysis

To examine whether the fitting procedure gave reasonable parameter estimates of the system-neglect model, we performed a parameter recovery analysis (Wilson & Collins, 2019). The analysis proceeded in the following steps. First, we simulated each subject’s probability estimation data based on the system-neglect model by using the parameter estimates obtained for that subject after fitting the system-neglect model to the subject’ probability estimation data. Second, we fit the system-neglect model to the simulated data. Third, as a measure of parameter recovery, we computed the correlation across subjects between the estimated parameters and the parameter values we used to simulate data, where larger correlation indicates better recovery. Fourth, we repeated the above steps by adding independent white noise to the simulated data, from a Gaussian distribution with mean 0 and variance σ². We implemented 3 levels of noise with σ ∈ {0.01,0.5,1} and examined the impact of noise on parameter recovery for each level of noise.

Model-free sensitivity analysis

This analysis was used for Fig. 6. In this analysis, we estimated how much subjects’ probability estimates changed as a function of transition probability and signal diagnosticity, separately for the blue (signals consistent with change) and red (signals inconsistent with change) signals. To do so, for each subject, we conducted a multiple regression analysis with probability estimates as data. We implemented a parametric regressor for transition probability and a parametric regressor for signal diagnosticity separately for the blue and red signals. As a result, four parametric regressors were included in the model. Since no computational model was implemented, the coefficients of these regressors provided a model-free measure of sensitivity to the system parameters and were referred to as the model-free behavioral slopes in Fig. 6. To contrast with the model-free behavioral slopes, we referred to the behavioral slopes in Fig. 5 also as the model-based behavioral slopes, since they were obtained based on the fitting the system-neglect model. In Fig. 6AB, we computed the Pearson correlation coefficient between the model-free behavioral slopes and model-based behavior slopes. In Fig. 6CD, we performed identical analyses as in Fig. 6AB except using simulated probability estimates as data. That is, we simulated probability estimation data based on the system-neglect model. For each subject separately, we used his or her : and ; parameter estimates to simulate probability estimates. Using simulated data, we estimated the model-based behavioral slopes. The reason for performing the same analysis on simulated data was to strengthen the evidence shown in Fig. 6AB. That the : parameters in the system-neglect model are driven by both blue and red signals, but that the ; parameters are driven only by the blue signals.

fMRI data acquisition

All the imaging parameters, including the EPI sequence and MPRAGE sequence, remained the same throughout the three experiments. For Experiments 1 and 2, the subjects completed the task in a 3T Siemens MRI scanner (MAGNETOM Trio) equipped with a 32-channel head array coil. Experiment 3 was collected at a later date after the same scanner went through a major upgrade (from Trio to the Prisma system) where a 64-channel head array coil was used. Each subject completed three functional runs. Before each run, a localizer scan was implemented for slice positioning. For each run, T2*-weighted functional images were collected using an EPI sequence (TR=2000ms, TE=30ms, 33 oblique slices acquired in ascending interleaved order, 3.4×3.4×3.4 mm isotropic voxel, 64×64 matrix in 220 mm field of view, flip angle 90°). To reduce signal loss in the ventromedial prefrontal cortex and orbitofrontal cortex, the sagittal axis was tilted clockwise up to 30°. Each run consisted of 9 trials and a total of 374 images. After the functional scans, T1-weighted structural images were collected (MPRAGE sequence with TR=2530ms, TE=3.03ms, flip angle=7°, 192 sagittal slices, 1×1×1 mm isotropic voxel, 224×256 matrix in a 256-mm field of view). For each subject, a field map image was also acquired for the purpose of estimating and partially compensating for geometric distortion of the EPI image so as to improve registration performance with the T1-weighted images.

fMRI preprocessing

The imaging data were preprocessed with FMRIB’s software Library (FSL version 6.0). First, for motion correction, MCFLIRT was used to remove the effect of head motion during each run. Second, FUGUE (FMRIB’s Utility for Geometric Unwarping of EPIs) was used to estimate and partially compensate for geometric distortion of the EPI images using field map images collected for the subject. Third, spatial smoothing was applied with a Gaussian kernel with FWHM=6mm. Fourth, a high-pass temporal filtering was applied using Gaussian-weighted least square straight-line fitting with σ = 50s. Fifth, registration was performed in a two-step procedure, with the field map used to improve the performance of registration. First, EPI images were registered to the high-resolution brain T1-weighted structural image (non-brain structures were removed via FSL’s BET (Brain Extraction Tool). Second, the transformation matrix (12-parameter affine transformation) from the T1-weighted image to the Montreal Neurological Institute (MNI) template brain was estimated using FLIRT (FMRIB’s Linear Image Registration Tool), followed by nonlinear registration using FNIRT (FMRIB’s Non-linear Image Registration Tool) with a 10mm warp resolution. This two-step procedure allowed for transforming the EPI images to the standard MNI template brain.

General Linear Models of BOLD signals

All GLM analyses were carried out in the following steps (Beckmann et al., 2003). First, BOLD time series were pre-whitened with local autocorrelation correction. A first-level FEAT analysis was carried out for each run of each subject. Second, a second-level (subject-level) fixed-effect (FE) analysis was carried out for each subject that combined the first-level FEAT results from different runs using the summary statistics approach. Finally, a third-level (group-level) mixed-effect (ME) analysis using FSL’s FLAME module (FMRIB’s Local Analysis of Mixed Effects) was carried out across subjects by taking the FE results from the previous level and treating subjects as a random effect (Woolrich et al., 2004). All reported whole-brain results were corrected for multiple comparisons. We first identified clusters of activation by defining a cluster-forming threshold of the z statistic (z > 3.1 or equivalently p < .001) (Eklund et al., 2016; Woo et al., 2014). Then, a family-wise error corrected p-value of each cluster based on its size was estimated using Gaussian random field theory (Worsley et al., 1992). In addition, we performed nonparametric permutation test using the randomize function in FSL (threshold-free cluster enhancement or TFCE option; (Smith & Nichols, 2009) on all the contrasts reported.

GLM-1

This model was used for Fig 3. The model served two purposes. First, we used it to examine neural representations for probability estimates (P_t) and belief revision (ΔP_t) (Fig. 3B). Second, we used this model to compare results between the main experiment (Experiment 1) and the control experiments (Experiments 2 and 3) (Fig. 3CD). We implemented the following regressors. At the time of each signal presentation, we implemented the following regressors: (R1) an indicator regressor with length equal to the subject’s RT, (R2) R1 multiplied by the subject’s probability estimate, P_t, that the signal came from the Blue regime, (R3) R1 multiplied by difference in the subject’s probability estimate between two successive periods, ΔP_t, which captured the updating of belief about change, (R4) R1 multiplied by the degree of certainty in probability estimate |P_t − 0.5|, (R5) R1 multiplied by the period number (from 1 to 10). Both positive and negative contrasts of R2 (P_t) and R3 (ΔP_t) were set up to identify activity that either positively or negatively correlate with these regressors. At the end of each trial (after subjects saw all 10 signals), we provided the monetary bonus the subject earned in that trial. We implemented an indicator regressor (R6) and a parametric regressor for the subject’s winning (R7). We implemented an indicator regressor (length equal to 4s) for the no-response periods (R8), which corresponded to the period(s) where the subject did not indicate the probability estimate within the time limit (4s). Finally, to directly address the motor confound issue, we implemented an action-handedness regressor, (R9), which was R1 multiplied by action-handedness. This regressor served to address the motor confounds of probability estimates P_t, as higher probability estimates preferentially involved right-handed responses for entering higher digits and lower estimates involved left-handed responses. Therefore, at the time of each signal presentation, the action-handedness regressor coded -1 if both finger presses to enter P_t involved left hand, 0 if using one left finger and one right finger, and 1 if both finger presses involved right hand. For Experiment 1, we also performed a GLM analysis that was identical to GLM-1 except that it did not include the action-handedness regressor and the results on both P_t and ΔP_t were largely identical (Supplementary Table S10). Note that for both Experiment 1 (the main regime-shift experiment) and Experiment 2 (control experiment), P_t indicates the probability estimate that the signal came from the Blue regime. However, for Experiment 3 (control experiment) subjects were instructed to press a two-digit number shown on the screen at each period. Hence, the P_t regressor for Experiment 3 was effectively the instructed number, and the ΔP_t regressor was effectively the difference in instructed number between adjacent periods.

GLM-2

This model was implemented to examine the effects of component variables critically contributing to regime-shift probability estimation. In particular, we used this model to identify the effects on the strength of change evidence (based on R11 below) and the intertemporal prior (based on R9 below) (Fig. 4 in main text). The model was identical to GLM-1 from (R1) to (R8) with the addition of the following regressors: (R9) R1 multiplied by the intertemporal prior, defined as ln , where 2 is the transition probability and, is the period, (R10) R1 multiplied by the natural logarithm of signal diagnosticity, ln(d), (R11) R1 multiplied by the current signal (1 for blue ball, -1 for red ball), and (R12) the interaction R9 × R10 (interaction between signal diagnosticity and signal). We note that since these component variables (R9 to R12) contributed to regime-shift probability estimation, they were correlated with P_t and ΔP_t. Having both with P_t and ΔP_t in the same model as these component variables would therefore introduce collinearity and reduce the reliability of regression coefficients. While we were aware of this issue, we also recognized that having these correlated regressors in the same model have the advantage of statistically stating that neural activity attributed to a particular variable cannot be otherwise attributed to another variable that correlated with it. We also performed another GLM analysis that was identical to GLM-2 except that it did not include P_t and Δ P_t in the model and the results on strength of change evidence and intertemporal prior shown in Fig. 4 were largely identical (Supplementary Table S11).

GLM-3

This model was the basis of results shown in Fig. 5. We set up two sets of regressors, one set for when the blue signal (signal for potential change) appeared and the other for when the red signal (signal for no change) appeared. For each set, at the time of signal presentation, we included 9 indicator regressors, one for every combination of 3 transition probability levels and 3 signal diagnosticity levels. Based on these 9 regressors, we set up a linear-increasing contrast separately for signal diagnosticity and transition probability. At the subject level (second level), the parameter estimate of these contrasts reflects individual subjects’ sensitivity to signal diagnosticity and transition probability were extracted from a given ROI and was used to correlate with individual subjects’ behavioral sensitivity to signal diagnosticity and transition probability. The ROIs used were vmPFC, ventral striatum, dmPFC, bilateral IFG, and bilateral IPS. These ROIs were identified by results from GLM 1 and 2 and were constructed in a statistically independent manner using leave-one-subject-out method. See Independent region-of-interest (ROI) analysis in the section below. Same as in the previous two GLMs, at the end of each trial when feedback on the current-trial monetary bonus was revealed, we implemented an indicator regressor and a parametric regressor for the monetary bonus. Finally, we implemented an indicator regressor (length equal to 4s) for no-response periods. These corresponded to the period(s) where the subject did not indicate the probability estimate within the time limit (4s).

Independent regions-of-interest (ROIs) analysis

We performed two kinds of independent ROI analysis—leave-one-subject-out (LOSO) method and functional/structural masks based on previous meta-analysis paper (Bartra et al. (2013) or existing structural atlases. Specifically, we used the functional mask in vmPFC from Bartra et al. (2013) and structural mask for the ventral striatum based on the Harvard-Oxford cortical and subcortical atlases in FSL. For LOSO, based on results from the whole-brain analysis, we created independent and unbiased ROIs using the leave-one-subject-out (LOSO) method (Litt et al., 2011; Ting et al., 2015). The LOSO method was used to analyze probability estimates and representations for transition probability and signal diagnosticity described above. We performed the analysis for each subject separately in the following steps. First, we identified the significant cluster in a brain region (e.g., dmPFC) that correlated with a contrast of interest (e.g., probability estimates) using all other subjects’ data. We referred to this as the LOSO ROI. Second, we extracted the mean beta value (regression coefficient) within the LOSO ROI from the subject and used it for further statistical analysis. Note that the LOSO ROI tended to be different spatially between subjects. To give an idea of these differences and the spatial distribution of these ROIs, in Figs. 4 and 5 where the LOSO analysis was performed, we showed both the voxels that were part of the LOSO ROI of all the subjects in one color and the voxels that were part of at least one LOSO ROI in another color. Finally, we note that since the LOSO procedure involves, for each subject separately, performing the leave-one-subject-out inference at the group level and identifying the significant clusters of activation. Therefore, it is possible that some subject(s), after the LOSO inference, did not have significant cluster in some brain region. For the analysis shown in Fig. 5C, for the left IFG ROI, we had 3 subjects that did not have a significant cluster in this brain region. All other ROIs had data from all subjects.

Acknowledgements

This work was supported by the National Science and Technology Council (NSTC) in Taiwan (Grants 108-2410-H-010-012-MY3, 110-2410-H-A49A-504 -MY3 to S.- W.W.) and by the Brain Research Center, National Yang Ming Chiao Tung University from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan. We acknowledge magnetic resonance imaging support from National Yang Ming Chiao Tung University, Taiwan, which is in part supported by the Ministry of Education plan for the top University.

Additional files

supplemental file