In what follows, we derive a computational model of the metacognitive control of decisions or MCD. In brief, we assume that the amount of cognitive resources that is deployed during a decision is controlled by an effort-confidence tradeoff. Critically, this tradeoff relies on a prospective anticipation of how these resources will perturb the internal representations of subjective values. As we will see, the MCD model eventually predicts how cognitive effort expenditure depends upon prior estimates of decision difficulty, and what impact this will have on post-choice value representations.

Let $z$ be the amount of cognitive (e.g., executive, mnemonic, or attentional) resources that serve to process value-relevant information. Allocating these resources will be associated with both a benefit $B\left(z\right)$, and a cost $C\left(z\right)$. As we will see, both are increasing functions of $z$: $B\left(z\right)$ derives from the refinement of internal representations of subjective values of alternative options or actions that compose the choice set, and $C\left(z\right)$ quantifies how aversive engaging cognitive resources are (mental effort). In line with the framework of expected value of control or EVC (Musslick et al., 2015; Shenhav et al., 2013), we assume that the brain chooses to allocate the amount of resources $\widehat{z}$ that optimizes the following cost–benefit trade-off:

where the expectation accounts for predictable stochastic influences that ensue from allocating resources (this will be clarified below). Note that the benefit term $B\left(z\right)$ is the (weighted) choice confidence $P_c\left(z\right)$:

where the weight $R$ is analogous to a reward and quantifies the importance of making a confident decision (see below). As we will see, $P_c\left(z\right)$ plays a pivotal role in the model, in that it captures the efficacy of allocating resources for processing value-relevant information. So, how do we define choice confidence?

We assume that decision makers may be unsure about how much they like/want the alternative options that compose the choice set. In other words, the internal representations of values $V_i$ of alternative options are probabilistic. Such a probabilistic representation of value can be understood in terms of, for example, an uncertain prediction regarding the to-be-experienced value of a given option. Without loss of generality, the probabilistic representation of option value takes the form of Gaussian probability density functions, as follows:

where ${\mu }_i$ and ${\sigma }_i$ are the mode and the variance of the probabilistic value representation, respectively (and $i$ indexes alternative options in the choice set).

This allows us to define choice confidence $P_c$ as the probability that the (predicted) experienced value of the (to be) chosen item is higher than that of the (to be) unchosen item:

where $s\left(x\right)=1/1+e^{-x}$ is the standard sigmoid mapping. Here the second line derives from assuming that the choice follows the sign of the preference $\Delta \mu ={\mu }_1-{\mu }_2$, and the last line derives from a moment-matching approximation to the Gaussian cumulative density function (Daunizeau, 2017a).

As stated in the Introduction section, we assume that the brain valuation system automatically generates uncertain estimates of options' value (Lebreton et al., 2009; Lebreton et al., 2015), before cognitive effort is invested in decision-making. In what follows, ${\mu }_i^0$ and ${\sigma }_i^0$ are the mode and variance of the ensuing prior value representations (we treat them as inputs to the MCD model). We also assume that these prior representations neglect existing value-relevant information that would require cognitive effort to be retrieved and processed (Lopez-Persem et al., 2016).

Now, how does the system anticipate the benefit of allocating resources to the decision process? Recall that the purpose of allocating resources is to process (yet unavailable) value-relevant information. The critical issue is thus to predict how both the uncertainty ${\sigma }_i$ and the modes ${\mu }_i$ of value representations will eventually change, before having actually allocated the resources (i.e., without having processed the information). In brief, allocating resources essentially has two impacts: (i) it decreases the uncertainty ${\sigma }_i$, and (ii) it perturbs the modes ${\mu }_i$ in a stochastic manner.

The former impact derives from assuming that the amount of information that will be processed increases with the amount of allocated resources. Here, this implies that the variance of a given probabilistic value representation decreases in proportion to the amount of allocated effort, that is:

where ${\sigma }_i^0$ is the prior variance of the representation (before any effort has been allocated), and $\beta$ controls the efficacy with which resources increase the precision of the value representation. Formally speaking, Equation 5 has the form of a Bayesian update of the belief's precision in a Gaussian-likelihood model, where the precision of the likelihood term is $\beta z$. More precisely, $\beta$ is the precision increase that follows from allocating a unitary amount of resources $z$. In what follows, we will refer to $\beta$ as the ‘type #1 effort efficacy’.

The latter impact follows from acknowledging the fact that the system cannot know how processing more value-relevant information will affect its preference before having allocated the corresponding resources. Let ${\delta }_i\left(z\right)$ be the change in the position of the mode of the ith value representation, having allocated an amount $z$ of resources. The direction of the mode's perturbation ${\delta }_i\left(z\right)$ cannot be predicted because it is tied to the information that would be processed. However, a tenable assumption is to consider that the magnitude of the perturbation increases with the amount of information that will be processed. This reduces to stating that the variance of ${\delta }_i\left(z\right)$ increases in proportion to $z$, that is:

where ${\mu }_i^0$ is the mode of the value representation before any effort has been allocated, and $\gamma$ controls the relationship between the amount of allocated resources and the variance of the perturbation term $\delta$. The higher the parameter $\gamma$, the greater the expected perturbation of the mode for a given amount of allocated resources. In what follows, we will refer to $\gamma$ as the ‘type #2 effort efficacy’. Note that Equation 6 treats the impact of future information processing as a non-specific random perturbation on the mode of the prior value representation. Our justification for this assumption is twofold: (i) it is simple, and (ii) and it captures the idea that the MCD controller does not know how the allocated resources will be used (here, by the value-based decision system downstream). We will see that, in spite of this, the MCD controller can still make quantitative predictions regarding the expected benefit of allocating resources.

Taken together, Equations 5 and 6 imply that predicting the net effect of allocating resources onto choice confidence is not trivial. On one hand, allocating effort will increase the precision of value representations (cf. Equation 5), which mechanically increases choice confidence, all other things being equal. On the other hand, allocating effort can either increase or decrease the absolute difference $\left|\Delta \mu \left(z\right)\right|$ between the modes. This, in fact, depends upon the sign of the perturbation terms $\delta$, which are not known in advance. Having said this, it is possible to derive the expected absolute difference between the modes that would follow from allocating an amount $z$ of resources:

where we have used the expression for the first-order moment of the so-called 'folded normal distribution', and the second term in the right-hand side of Equation 7 derives from the same moment-matching approximation to the Gaussian cumulative density function as above. The expected absolute means' difference $E\left[\left|\Delta \mu \right||z\right]$ depends upon both the absolute prior mean difference $\left|\Delta {\mu }_{}^0\right|$ and the amount of allocated resources $z$. This is depicted in Figure 2.

Figure 2

Download asset Open asset

One can see that ${\displaystyle E\left[|\mathrm{\Delta }\mu ||z\right]-|\mathrm{\Delta }{\mu }^0|}$ is always greater than 0 and increases with $z$ (and if $z=0$, then $E\left[\left|\Delta \mu \right||z\right]=\left|\Delta {\mu }_{}^0\right|$). In other words, allocating resources is expected to increase the value difference, despite the fact that the impact of the perturbation term can go either way. In addition, the expected gain in value difference afforded by allocating resources decreases with the absolute prior means' difference.

Similarly, the variance $V\left[\left|\Delta \mu \right||z\right]$ of the absolute means' difference is derived from the expression of the second-order moment of the corresponding folded normal distribution:

One can see in Figure 2 that $V\left[\left|\Delta \mu \right||z\right]$ increases with the amount $z$ of allocated resources (but if $z=0$, then $V\left[\left|\Delta \mu \right||z\right]=0$).

Knowing the moments of the distribution of $\left|\Delta \mu \right|$ now enables us to derive the expected confidence level ${\overline{P}}_c\left(z\right)$ that would result from allocating the amount of resource $z$:

where we have assumed, for the sake of conciseness, that both prior value representations are similarly uncertain (i.e., ${\sigma }_1^0\approx {\sigma }_2^0\triangleq {\sigma }_{}^0$). It turns out that the expected choice confidence ${\overline{P}}_c\left(z\right)$ always increase with $z$, irrespective of the efficacy parameters, as long as $\beta \ne 0$ or $\gamma \ne 0$. These, however, control the magnitude of the confidence gain that can be expected from allocating an amount $z$ of resources. Equation 9 is important, because it quantifies the expected benefit of resource allocation, before having processed the ensuing value-relevant information. More details regarding the accuracy of Equation 9 can be found in section 1 of the Appendix. In addition, section 2 of the Appendix summarizes the dependence of MCD-optimal choice confidence on $\left|\Delta {\mu }_{}^0\right|$ and ${\sigma }_{}^0$.

To complete the cost–benefit model, we simply assume that the cost of allocating resources to the decision process linearly scales with the amount of resources, that is:

where $\alpha$ determines the effort cost of allocating a unitary amount of resources $z$. In what follows, we will refer to $\alpha$ as the ‘effort unitary cost’. We note that weak nonlinearities in the cost function (e.g., quadratic terms) would not qualitatively change the model predictions.

In brief, the MCD-optimal resource allocation $\widehat{z}\triangleq \widehat{z}\left(\alpha ,\beta ,\gamma \right)$ is simply given by:

which does not have any closed-form analytic solution. Nevertheless, it can easily be identified numerically, having replaced Equations 7–9 into Equation 11. We refer the readers interested in the impact of model parameters $\left\{\alpha ,\beta ,\gamma \right\}$ on the MCD-optimal control to section 2 of the Appendix.

At this point, we do not specify how Equation 11 is solved by neural networks in the brain. Many alternatives are possible, from gradient ascent (Seung, 2003) to winner-take-all competition of candidate solutions (Mao and Massaquoi, 2007). We will also comment on the specific issue of prospective (offline) versus reactive (online) MCD processes in the Discussion section.

Note: implicit in the above model derivation is the assumption that the allocation of resources is similar for both alternative options in the choice set (i.e. $z_1^{}\approx z_2^{}\triangleq z$). This simplifying assumption is justified by eye-tracking data (cf. section 8 of the Appendix).

In the previous section, we derived the MCD-optimal resource allocation $\widehat{z}$, which effectively best balances the expected choice confidence with the expected effort costs, given the predictable impact of stochastic perturbations that arise from processing value-relevant information. This quantitative prediction is effectively shown in Figures 5 and 6 of the Results section below, as a function of (empirical proxies for) the prior absolute difference between modes $\left|\Delta {\mu }^0\right|$ and the prior certainty $1/{\sigma }_{}^0$ of value representations. But, this resource allocation mechanism has a few interesting corollary implications.

To begin with, note that knowing $\widehat{z}$ enables us to predict what confidence level the system should eventually reach. In fact, one can define the MCD-optimal confidence level as the expected confidence evaluated at the MCD-optimal amount of allocated resources, that is, ${\overline{P}}_c\left(\widehat{z}\right)$. This is important, because it implies that the model can predict both the effort the system will invest and its associated confidence, on a decision-by-decision basis. The impact of the efficacy parameters on this quantitative prediction is detailed in section 2 of the Appendix.

Additionally, $\widehat{z}$ determines the expected improvement in the certainty of value representations (hereafter: the ‘certainty gain’), which trivially relates to type #2 efficacy, that is: $1/\sigma \left(\widehat{z}\right)-1/{\sigma }_{}^0=\beta \widehat{z}$. This also means that, under the MCD model, no choice-induced value certainty gain can be expected when $\beta =0$.

Similarly, one can predict the MCD-optimal probability of changing one's mind. Recall that the probability $Q\left(z\right)$ of changing one's mind depends on the amount of allocated resources $z$, that is:

One can see that the MCD-optimal probability of changing one's mind $Q\left(\widehat{z}\right)$ is a simple monotonic function of the allocated effort $\widehat{z}$. Importantly, $Q\left(z\right)=0$ when $\gamma =0$. This implies that MCD agents do not change their minds when effort cannot change the relative position of the modes of the options’ value representations (irrespective of type #1 effort efficacy). In retrospect, this is critical, because there should be no incentive to invest resources in deliberation if it would not be possible to change one’s pre-deliberation preference.

Lastly, we can predict the magnitude of choice-induced preference change, that is, how value representations are supposed to spread apart during the decision. Such an effect is typically measured in terms of the so-called ‘spreading of alternatives’ or SoA, which is defined as follows:

where $\Delta \delta \left(z\right)\sim N\left(0,2\gamma z\right)$ is the cumulative perturbation term of the modes' difference. Taking the expectation of the right-hand term of Equation 13 under the distribution of $\Delta \delta \left(z\right)$ and evaluating it at $z=\widehat{z}$ now yields the MCD-optimal spreading of alternatives $\overline{SOA}\left(\widehat{z}\right)$:

where the last line derives from the expression of the first-order moment of the truncated Gaussian distribution. Note that the expected preference change also increases monotonically with the allocated effort $\widehat{z}$. Here again, under the MCD model, no preference change can be expected when $\gamma =0$.

We note that all of these corollary predictions essentially capture choice-induced modifications of value representations. This is why we will refer to choice confidence, value certainty gain, change of mind, and spreading of alternatives as ‘decision-related’ variables.

In summary, the MCD model predicts cognitive effort (or, more properly, the amount of allocated resources) and decision-related variables, given the prior absolute difference between modes $\left|\Delta {\mu }^0\right|$ and the prior certainty $1/{\sigma }_{}^0$ of value representations. In other words, the inputs to the MCD model are the prior moments of value representations, whose trial-by-trial variations determine variations in model predictions. Here, we simply assume that pre-choice value and value certainty ratings provide us with an approximation of these prior moments. More precisely, we use ΔVR⁰ and VCR⁰ (cf. section 3.3 below) as empirical proxies for $\Delta {\mu }^0$ and $1/{\sigma }_{}^0$, respectively. Accordingly, we consider post-choice value and value certainty ratings as empirical proxies for the posterior mean $\mu \left(\widehat{z}\right)$ and precision $1/\sigma \left(\widehat{z}\right)$ of value representations, at the time when the decision was triggered (i.e., after having invested the effort $\widehat{z}$). Similarly, we match expected choice confidence ${\overline{P}}_c\left(z\right)$ (i.e., after having invested the effort $\widehat{z}$) with empirical choice confidence.

Note that the MCD model does not specify what the allocated resources are. In principle, both mnesic and attentional resources may be engaged when processing value-relevant information. Nevertheless, what really matters is assessing the magnitude $z$ of decision-related effort. We think of $z$ as the cumulative engagement of neurocognitive resources, which varies both in terms of duration and intensity. Empirically, we relate $\widehat{z}$ to two different ‘effort-related’ empirical measures, namely subjective feeling of effort and response time. The former relies on the subjective cost incurred when deploying neurocognitive resources, which would be signaled by experiencing mental effort. The latter makes sense if one thinks of response time in terms of effort duration. Although it is a more objective measurement than subjective rating of effort, response time only approximates $\widehat{z}$ if effort intensity shows relatively small variations. We will comment on this in the Discussion section.

Finally, the MCD model is also agnostic about the definition of ‘decision importance’, that is, the weight $R$ in Equation 2. In this work, we simply investigate the effect of decision importance by comparing subjective effort and response time in ‘neutral’ versus ‘consequential’ decisions (cf. section 'Task conditions' below). We will also comment on this in the Discussion section.

Participants for our study were recruited from the RISC (Relais d’Information sur les Sciences de la Cognition) subject pool through the ICM (Institut du Cerveau et de la Moelle – Paris Brain Institute). All participants were native French speakers, with no reported history of psychiatric or neurological illness. A total of 41 people (28 female; age: mean = 28, SD = 5, min = 20, max = 40) participated in this study. The experiment lasted approximately 2 hr, and participants were paid a flat rate of 20€ as compensation for their time, plus a bonus, which was given to participants to compensate for potential financial losses in the ‘penalized’ trials (see below). More precisely, in ‘penalized’ trials, participants lost 0.20€ (out of a 5€ bonus) for each second that they took to make their choice. This resulted in an average 4€ bonus (across participants). One group of 11 participants was excluded from the cross-condition analysis only (see below) due to technical issues.

Written instructions provided detailed information about the sequence of tasks within the experiment, the mechanics of how participants would perform the tasks, and images illustrating what a typical screen within each task section would look like. The experiment was developed using Matlab and PsychToolbox, and was conducted entirely in French. The stimuli for this experiment were 148 digital images, each representing a distinct food item (50 fruits, 50 vegetables, and 48 various snack items including nuts, meats, and cheeses). Food items were selected such that most items would be well known to most participants.

Eye gaze position and pupil size were continuously recorded throughout the duration of the experiment using The Eye Tribe eye-tracking devices. Participants’ head positions were fixed using stationary chinrests. In case of incidental movements, we corrected the pupil size data for distance to screen, separately for each eye.

Prior to commencing the testing session of the experiment, participants underwent a brief training session. The training tasks were identical to the experimental tasks, although different stimuli were used (beverages). The experiment itself began with an initial section where all individual items were displayed in a random sequence for 1.5 s each, in order to familiarize the participants with the set of options they would later be considering and form an impression of the range of subjective value for the set. The main experiment was divided into three sections, following the classic Free-Choice Paradigm protocol (e.g., Izuma and Murayama, 2013): pre-choice item ratings, choice, and post-choice item ratings. There was no time limit for the overall experiment, nor for the different sections, nor for the individual trials. The item rating and choice sections are described below (see Figure 3).

Figure 3

Download asset Open asset

We partitioned the task trials into three conditions, which were designed to test the following two predictions of the MCD model: all else equal, effort should increase with decision importance and decrease with related costs. We aimed to check the former prediction by asking participants to make some decisions where they knew that the choice would be real, that is, they would actually have to eat the chosen food item at the end of the experiment. We refer to these trials as ‘consequential’ decisions. To check the latter prediction, we imposed a financial penalty that increased with response time. More precisely, participants were instructed that they would lose 0.20€ (out of a 5€ bonus) for each second that they would take to make their choice. The choice section of the experiment was composed of 60 ‘neutral’ trials, 7 ‘consequential’ trials, and 7 ‘penalized’ trials, which were randomly intermixed. Instructions for both ‘consequential’ and ‘penalized’ decisions were repeated at each relevant trial, immediately prior to the presentation of the choice items.

The MCD model predicts trial-by-trial variations in the probability of changing one’s mind, choice confidence, spreading of alternatives, certainty gain, response time, and subjective effort ratings (MCD outputs) from trial-by-trial variations in value rating difference ΔVR⁰ and mean value certainty rating VCR⁰ (MCD inputs). Together, three unknown parameters control the quantitative relationship between MCD inputs and outputs: the effort unitary cost $\alpha$, type #1 effort efficacy $\beta$, and type #2 effort efficacy $\gamma$. However, additional parameters are required to capture variations induced by experimental conditions. Recall that we expect ‘consequential’ decisions to be more important than ‘neutral’ ones, and ‘penalized’ decisions effectively include an extraneous cost-of-time term. One can model the former condition effect by making $R$ (cf. Equation 2) sensitive to whether the decision is consequential or not. We proxy the latter condition effect by making the effort unitary cost $\alpha$ a function of whether the decision is penalized (where effort induces both intrinsic and extrinsic costs) or not (intrinsic effort cost only). This means that condition effects require one additional parameter each.

In principle, all of these parameters may vary across people, thereby capturing idiosyncrasies in people’s (meta-)cognitive apparatus. However, in addition to estimating these five parameters, fitting the MCD model to each participant’s data also requires a rescaling of the model’s output variables. This is because there is no reason to expect the empirical measure of these variables to match their theoretical scale. We thus inserted two additional nuisance parameters per output MCD variable, which operate a linear rescaling (affine transformation, with a positive constraint on slope parameters). Importantly, these nuisance parameters cannot change the relationship between MCD inputs and outputs. In other terms, the MCD model really has only five degrees of freedom.

For each subject, we fit all MCD dependent variables concurrently with a single set of MCD parameters. Within-subject probabilistic parameter estimation was performed using the variational Laplace approach (Daunizeau, 2017b; Friston et al., 2007), which is made available from the VBA toolbox (Daunizeau et al., 2014). We refer the reader interested in the mathematical details of within-subject MCD parameter estimation to the section 3 of the Appendix (this also includes a parameter recovery analysis). In what follows, we compare empirical data to MCD-fitted dependent variables (when binned according to ΔVR⁰ and VCR⁰). We refer to the latter as ‘postdictions’, in the sense that they derive from a posterior predictive density that is conditional on the corresponding data.

We also fit the MCD model on reduced subsets of dependent variables (e.g., only ‘effort-related’ variables), and report proper out-of-sample predictions of data that were not used for parameter estimation (e.g., ‘decision-related’ variables). We note that this is a strong test of the model, since it does not rely on any train/test partition of the predicted variable (see next section below).

To capture idiosyncrasies in participants’ metacognitive control of decisions, the MCD model was fitted to subject-specific trial-by-trial data, where all MCD outputs (namely change of mind, choice confidence, spreading of alternatives, value certainty gain, response time, and subjective effort ratings) were considered together. In the next section, we present summary statistics at the group level, which validate the predictions that can be derived from the MCD model, when fitted to all dependent variables. But can we provide even stronger evidence that the MCD model is capable of predicting all dependent variables at once? In particular, can the model make out-of-sample predictions regarding effort-related variables (i.e., RT and subjective effort ratings) given decision-related variables (i.e., choice confidence, change of mind, spreading of alternatives, and certainty gain), and vice versa?

To address this question, we performed two partial model fits: (i) with decision-related variables only, and (ii) with effort-related variables only. In both cases, out-of-sample predictions for the remaining dependent variables were obtained directly from within-subject parameter estimates. For each subject, we then estimated the cross-trial correlation between each pair of observed and predicted variables. Figure 4 reports the ensuing group-average correlations, for each dependent variable and each model fit. In this context, the predictions derived when fitting the full dataset only serve as a reference point for evaluating the accuracy of out-of-sample predictions. For completeness, we also show chance-level prediction accuracy (i.e. the 95% percentile of group average correlations between observed and predicted variables under the null).

Figure 4

Download asset Open asset

In what follows, we refer to model predictions on dependent variables that were actually fitted by the model as ‘postdictions’ (full data fits: all dependent variables, partial model fits: variables included in the fit). As one would expect, the accuracy of postdictions is typically higher than that of out-of-sample predictions. Slightly more interesting, perhaps, is the fact that the accuracy of model predictions/postdictions depends upon which output variable is considered. For example, choice confidence is always better predicted/postdicted than spreading of alternatives. This is most likely because the latter data has lower reliability.

But the main result of this analysis is the fact that out-of-sample predictions of dependent variables perform systematically better than chance. In fact, all across-trial correlations between observed and predicted (out-of-sample) data were statistically significant at the group-level (all p<10⁻³). In particular, this implies that the MCD model makes accurate out-of-sample predictions regarding effort-related variables given decision-related variables, and reciprocally.

In what follows, we inspect the three-way relationships between pre-choice value and value certainty ratings and each effort-related variable: namely, RT and subjective effort rating. The former can be thought of as a proxy for the duration of resource allocation, whereas the latter is a metacognitive readout of resource allocation cost. Unless stated otherwise, we will focus on both the absolute difference between pre-choice value ratings (hereafter: |ΔVR⁰|) and the mean pre-choice value certainty rating across paired choice items (hereafter: VCR⁰). Under the MCD model, increasing |ΔVR⁰| and/or VCR⁰ will decrease the demand for effort, which should result in smaller expected RT and subjective effort rating. We will now summarize the empirical data and highlight the corresponding quantitative MCD model postdictions and out-of-sample predictions (here: predictions are derived from model fits on decision-related variables only, that is, all dependent variables except RT and subjective effort rating).

First, we checked how RT relates to pre-choice value and value certainty ratings. For each subject, we regressed (log-) RT data against |ΔVR⁰| and VCR⁰, and then performed a group-level random-effect analysis on regression weights. The results of this model-free analysis provide a qualitative summary of the impact of trial-by-trial variations in pre-choice value representations on RT. We also compare RT data with both MCD model postdictions (full data fit) and out-of-sample predictions. In addition to summarizing the results of the model-free analysis, Figure 5 shows the empirical, predicted, and postdicted RT data, when median-split (within subjects) according to both |ΔVR⁰| and VCR⁰.

Figure 5

Download asset Open asset

One can see that RT data behave as expected under the MCD model, that is, RT decreases when |ΔVR⁰|and/or VCR⁰ increases. The random effect analysis shows that both variables have a significant negative effect at the group level (|ΔVR⁰|: mean standardized regression weight = −0.16, s.e.m. = 0.02, p<10⁻³; VCR⁰: mean standardized regression weight = −0.08, s.e.m. = 0.02, p<10⁻³; one-sided t-tests). Moreover, MCD postdictions are remarkably accurate at capturing the effect of both |ΔVR⁰|and VCR⁰ variables in a quantitative manner. Although MCD out-of-sample predictions are also very accurate, they tend to slightly underestimate the quantitative effect of |ΔVR⁰|. This is because this effect is typically less pronounced in decision-related variables than in effort-related variables (see below), which then yield MCD parameter estimates that eventually attenuate the impact of |ΔVR⁰| on effort.

Second, we checked how subjective effort ratings relate to pre-choice value and value certainty ratings. We performed the same analysis as above, the results of which are summarized in Figure 6.

Figure 6

Download asset Open asset

Here as well, subjective effort rating data behave as expected under the MCD model, that is, subjective effort decreases when |ΔVR⁰| and/or VCR⁰ increases. The random effect analysis shows that both variables have a significant negative effect at the group level (|ΔVR⁰|: mean standardized regression weight = −0.21, s.e.m. = 0.03, p<10⁻³; VCR⁰: mean regression weight = −0.05, s.e.m. = 0.02, p=0.027; one-sided t-tests). One can see that MCD postdictions and out-of-sample predictions accurately capture the effect of both |ΔVR⁰|and VCR⁰ variables. More quantitatively, we note that MCD postdictions slightly overestimate the effect VCR⁰, whereas out-of-sample predictions also tend to underestimate the effect of |ΔVR⁰|.

At this point, we note that the MCD model makes two additional predictions regarding effort-related variables, which relate to our task conditions. In brief, all else equal, effort should increase in ‘consequential’ trials, while it should decrease in ‘penalized’ trials. To test these predictions, we modified the model-free regression analysis of RT and subjective effort ratings by including two additional subject-level regressors, encoding consequential and penalized trials, respectively. Figure 7 shows the ensuing augmented set of standardized regression weights for both RT and subjective effort ratings.

Figure 7

Download asset Open asset

First, we note that accounting for task conditions does not modify the statistical significance of the impact of |ΔVR⁰| and VCR⁰ on effort-related variables, except for the effect of VCR⁰ on subjective effort ratings (p=0.09, one-sided t-test). Second, one can see that the impact of ‘consequential’ and ‘penalized’ conditions on effort-related variables globally conforms to MCD predictions. More precisely, both RT and subjective effort ratings were significantly higher for ‘consequential’ decisions than for ‘neutral’ decisions (log-RT: mean standardized regression weight = 0.07, s.e.m. = 0.03, p=0.036; effort ratings: mean standardized regression weight = 0.12, s.e.m. = 0.03, p<10⁻³; one-sided t-tests). In addition, response times are significantly faster for ‘penalized’ than for ‘neutral’ decisions (mean standardized regression weight = −0.26, s.e.m. = 0.03, p<10⁻³; one-sided t-test). However, the difference in subjective effort ratings between ‘neutral’ and ‘penalized’ decisions does not reach statistical significance (mean effort difference = 0.012, s.e.m. = 0.024, p=0.66; two-sided t-test). We will comment on this in the Discussion section.

Under the MCD model, ‘decision-related’ dependent variables (i.e., choice confidence, change of mind, spreading of alternatives, and value certainty gain) are determined by the amount of resources allocated to the decision. However, their relationship to features of prior value representation is not trivial (see section 2 of the Appendix for the specific case of choice confidence). For this reason, we will recapitulate the qualitative MCD prediction that can be made about each of them, prior to summarizing the empirical data and its corresponding postdictions and out-of-sample predictions. Note that here, the latter are obtained from a model fit on effort-related variables only.

First, we checked how choice confidence relates to |ΔVR⁰| and VCR⁰. Under the MCD model, choice confidence reflects the discriminability of the options’ value representations after optimal resource allocation. Recall that more resources are allocated to the decision when either |ΔVR⁰| or VCR⁰ decreases. However, under moderate effort efficacies, this does not overcompensate decision difficulty, and thus choice confidence should decrease. As with effort-related variables, we regressed trial-by-trial confidence data against |ΔVR⁰| and VCR⁰, and then performed a group-level random-effect analysis on regression weights. The results of this analysis, as well as the comparison between empirical, predicted, and postdicted confidence data are shown in Figure 8.

Figure 8

Download asset Open asset

The results of the group-level random effect analysis confirm our qualitative predictions. In brief, both |ΔVR⁰| (mean standardized regression weight = 0.25, s.e.m. = 0.02, p<10⁻³; one-sided t-test) and VCR⁰ (mean standardized regression weight = 0.16, s.e.m. = 0.03, p<10⁻³; one-sided t-test) have a significant positive impact on choice confidence. Here again, MCD postdictions and out-of-sample predictions are remarkably accurate at capturing the effect of both |ΔVR⁰|and VCR⁰ variables (though predictions slightly underestimate the effect of |ΔVR⁰|).

Second, we checked how change of mind relates to |ΔVR⁰| and VCR⁰. Note that we define a change of mind according to two criteria: (i) the choice is incongruent with the prior preference inferred from the pre-choice value ratings, and (ii) the choice is congruent with the posterior preference inferred from post-choice value ratings. The latter criterion distinguishes a change of mind from a mere ‘error’, which may arise from attentional and/or motor lapses. Under the MCD model, we expect no change of mind unless type #2 efficacy $\gamma \ne 0$. In addition, the rate of change of mind should decrease when either |ΔVR⁰| or VCR⁰ increases. This is because increasing |ΔVR⁰| and/or VCR⁰ will decrease the demand for effort, which implies that the probability of reversing the prior preference will be smaller. Figure 9 shows the corresponding model predictions/postdictions and summarizes the corresponding empirical data.

Figure 9

Download asset Open asset

Note that, on average, the rate of change of mind reaches about 14.5% (s.e.m. = 0.008, p<10⁻³, one-sided t-test), which is significantly higher than the rate of ‘error’ (mean rate difference = 2.3%, s.e.m. = 0.01, p=0.032; two-sided t-test). The results of the group-level random effect analysis confirm our qualitative MCD predictions. In brief, both |ΔVR⁰| (mean standardized regression weight = −0.17, s.e.m. = 0.02, p<10⁻³; one-sided t-test) and VCR⁰ (mean standardized regression weight = −0.08, s.e.m. = 0.03, p<10⁻³; one-sided t-test) have a significant negative impact on the rate of change of mind. Again, MCD postdictions and out-of-sample predictions are remarkably accurate at capturing the effect of both |ΔVR⁰|and VCR⁰ variables (though predictions slightly underestimate the effect of |ΔVR⁰|).

Third, we checked how spreading of alternatives relates to |ΔVR⁰| and VCR⁰. Recall that spreading of alternatives measures the magnitude of choice-induced preference change. Under the MCD model, the reported value of alternative options cannot spread apart unless type #2 efficacy $\gamma \ne 0$. In addition, and as with change of mind, spreading of alternatives should globally follow the optimal effort allocation, that is, it should decrease when |ΔVR⁰| and/or VCR⁰ increase. Figure 10 shows the corresponding model predictions/postdictions and summarizes the corresponding empirical data.

Figure 10

Download asset Open asset

One can see that there is a significant positive spreading of alternatives (mean = 0.04 A.U., s.e.m. = 0.004, p<10⁻³, one-sided t-test). This is reassuring, because it dismisses the possibility that $\gamma =0$ (which would mean that effort does not perturb the mode of value representations). In addition, the results of the group-level random effect analysis confirm that both |ΔVR⁰| (mean standardized regression weight = −0.09, s.e.m. = 0.03, p=0.001; one-sided t-test) and VCR⁰ (mean standardized regression weight = −0.04, s.e.m. = 0.02, p=0.03; one-sided t-test) have a significant negative impact on spreading of alternatives. Note that this replicates previous findings on choice-induced preference change (Lee and Coricelli, 2020; Lee and Daunizeau, 2020). Finally, MCD postdictions and out-of-sample predictions accurately capture the effect of both |ΔVR⁰| and VCR⁰ variables in a quantitative manner (though predictions slightly underestimate the effect of |ΔVR⁰|).

Fourth, we checked how |ΔVR⁰| and VCR⁰ impact value certainty gain. Under the MCD model, the certainty of value representations cannot improve unless type #1 efficacy $\beta \ne 0$. In addition, value certainty gain should globally follow the optimal effort allocation, i.e., it should decrease when |ΔVR⁰| and/or VCR⁰ increase. Figure 11 shows the corresponding model predictions/postdictions and summarizes the corresponding empirical data.

Figure 11

Download asset Open asset

Importantly, there is a small but significantly positive certainty gain (mean = 0.11 A.U., s.e.m. = 0.06, p=0.027, one-sided t-test). This is reassuring, because it dismisses the possibility that $\beta =0$ (which would mean that effort does not increase the precision of value representation). This time, the results of the group-level random effect analysis only partially confirm our qualitative MCD predictions. In brief, although VCR⁰ has a very strong negative impact on certainty gain (mean standardized regression weight = −0.61, s.e.m. = 0.04, p<10⁻³; one-sided t-test), the effect of |ΔVR⁰| does not reach statistical significance (mean standardized regression weight = −0.009, s.e.m. = 0.01, p=0.35; one-sided t-test). We note that a simple regression-to-the-mean artifact (Stigler, 1997) likely inflates the observed negative correlation between VCR⁰ and certainty gain, beyond what would be predicted under the MCD model. Accordingly, both MCD postdictions and out-of-sample predictions clearly underestimate the effect of VCR⁰ (and overestimate the effect of |ΔVR⁰|).

The computer code and algorithms that support the findings of this study will soon be made available from the open academic freeware VBA (http://mbb-team.github.io/VBA-toolbox/). Until then, they are available from the corresponding author upon reasonable request.

This study complies with all relevant ethical regulations and received formal approval from the INSERM Ethics Committee (CEEI-IRB00003888, decision no 16–333). In particular, in accordance with the Helsinki declaration, all participants gave written informed consent prior to commencing the experiment, which included consent to disseminate the results of the study via publication.

The MCD model relies on the system's ability to anticipate the benefit of allocating resources to the decision process. Given the mathematical expression of choice confidence (Equation 4 in the main text), this reduces to finding an analytical approximation to the following expression:

where $x\to s\left(x\right)=1/1+{\mathrm{e}}^{-x}$ is the sigmoid mapping, $\lambda$ is an arbitrary constant, and the expectation is taken under the Gaussian distribution of $x\sim N\left(\mu ,{\sigma }^2\right)$, whose mean and variance are µ and ${\sigma }^2$, respectively.

Note that the absolute value mapping $x\to \left|x\right|$ follows a folded normal distribution, whose first two moments $E\left[\left|x\right|\right]$ and $V\left[\left|x\right|\right]$ have known expressions:

where the first line relies on a moment-matching approximation to the cumulative normal distribution function (Daunizeau, 2017a). This allows us to derive the following analytical approximation to Equation A1:

where setting $a\approx 3/{\pi }^2$ makes this approximation tight (Daunizeau, 2017a).

The quality of this approximation can be evaluated by drawing samples of $x\sim N\left(\mu ,{\sigma }^2\right)$, and comparing the Monte-Carlo average of $s\left(\lambda \left|x\right|\right)$ with the expression given in Equation A3. This is summarized in Appendix 1—figure 1, where the range of variation for the moments of $x$ was set as follows: $\mu \in \left[-4,4\right]$ and ${\sigma }^2\in \left[0,4\right]$.

Appendix 1—figure 1

Download asset Open asset

One can see that the error rarely exceeds 5%, across the whole range of moments $\left\{\mu ,{\sigma }^2\right\}$ of the parent distribution. This is how tight the analytic approximation of the expected confidence gain (Equation 9 in the main text) is.

To begin with, note that the properties of the metacognitive control of decisions (in terms of effort allocation and/or confidence) actually depend on the demand for resources, which is itself determined by prior value representations (or, more properly, by the prior uncertainty ${\sigma }_{}^0$ and the absolute means' difference $\left|\Delta {\mu }_{}^0\right|$). Now, the way the MCD-optimal control responds to the resource demand is determined by effort efficacy and unitary cost parameters. In addition, MCD-optimal confidence may not trivially follow resource allocation, because it may be overcompensated by choice difficulty.

First, recall that the amount $\widehat{z}$ of allocated resources maximizes the EVC:

where ${\overline{P}}_c\left(z\right)$ is given in Equation 9 in the main text. According to the implicit function theorem, the derivatives of $\widehat{z}$ w.r.t. ${\sigma }_{}^0$ and $\left|\Delta {\mu }_{}^0\right|$ are given by Gould et al., 2016:

The double derivatives in Equations A5 are not trivial to obtain.

First, the gradient $\partial {\overline{P}}_c\left(z\right)/\partial \left|\Delta {\mu }^0\right|$ of choice confidence w.r.t. $\left|\Delta {\mu }_{}^0\right|$ writes:

where $K\left(z\right)\ge 0$ is given by:

Note that the gradient $\partial E\left[\left|\Delta \mu \right||z\right]/\partial \left|\Delta {\mu }^0\right|\ge 0$ in Equation A6 can be obtained analytically from Equation 7 in the main text. However, we refrain from doing this, because it is clear that deriving the right-hand term of Equation A6 w.r.t. both ${\sigma }^0$ and $z$ will not bring any simple insight regarding the impact of $\left|\Delta {\mu }_{}^0\right|$ onto $\widehat{z}$.

Also, although the gradient $\partial {\overline{P}}_c\left(\widehat{z}\right)/\partial {\sigma }^0$ of choice confidence wr.t. ${\sigma }^0$ takes a much more concise form:

it still remains tedious to derive its expression with respect to both ${\sigma }^0$ and $z$. This is why we opt for separating the respective effects of type #1 and type #2 efficacies.

First, let us ask what would be the MCD-optimal effort $\widehat{z}$ and confidence ${\overline{P}}_c\left(\widehat{z}\right)$ when $\gamma =0$, that is, if the only effect of allocating resources is to increase the precision of value representations. We call this the 'β-effect'. In this case, $E\left[\left|\Delta \mu \right||z\right]=\left|\Delta {\mu }^0\right|$ and $V\left[\left|\Delta \mu \right||z\right]=0$ irrespective of $z$. This greatly simplifies Equations A6–A8:

Inserting Equation A9 back into Equation A5 now yields:

Now the sign of the gradients of $\widehat{z}$ w.r.t. ${\sigma }_{}^0$ and $\left|\Delta {\mu }_{}^0\right|$ are driven by the numerators of Equation A10 because all partial derivatives of $K\left(z\right)$ have unambiguous signs:

Replacing the expression for $\partial K\left(z\right)/\partial z$ in Equation A11 into Equation A10 now yields:

At the limit $\left|\Delta {\mu }^0\right|\to 0$, then: $\partial \widehat{z}/\partial \left|\Delta {\mu }^0\right|\ge 0$ and $\partial \widehat{z}/\partial {\sigma }^0\ge 0$. However, one can see from Equation A12 that there may be a critical value for $\left|\Delta {\mu }^0\right|$, above which the gradient $\partial \widehat{z}/\partial \left|\Delta {\mu }^0\right|$ will eventually become negative. This means that the amount of allocated resources will behave as a bell-shaped function of $\left|\Delta {\mu }^0\right|$. This may not be the case along the ${\sigma }_{}^0$ direction though, because ${\sigma }_{}^0\ge \sigma \left(z\right)$ and the last term in the brackets shrinks as ${\sigma }_{}^0$ increases.

Similar derivations eventually yield expressions for the gradients of MCD-optimal confidence:

Equation A13 implies that, under moderate type #1 efficacy ($\beta \approx 0$), MCD-optimal confidence decreases when $\left|\Delta {\mu }_{}^0\right|$ decreases and/or when ${\sigma }_{}^0$ increases, irrespective of the amount $\widehat{z}$ of allocated resources. In other terms, variations in choice confidence are dominated by variations in the discriminability of prior value representations.

This analysis is exemplified in Appendix 1—figure 2, which summarizes the β-effect, in terms of how MCD-optimal resource allocation and choice confidence depend upon $\left|\Delta {\mu }_{}^0\right|$ and ${\sigma }_{}^0$.

Appendix 1—figure 2

Download asset Open asset

One can see that, overall, increasing the prior variance ${\sigma }_{}^0$ increases the resource demand, which eventually increases the MCD-optimal allocated effort $\widehat{z}$. This, however, does not overcompensate for the loss of confidence incurred when increasing the prior variance. This is why the MCD-optimal confidence ${\overline{P}}_c\left(\widehat{z}\right)$ decreases with the prior variance ${\sigma }_{}^0$. Note that, for the same reason, the MCD-optimal confidence increases with the absolute prior means' difference $\left|\Delta {\mu }_{}^0\right|$.

Now the impact of the absolute prior means' difference $\left|\Delta {\mu }_{}^0\right|$ on $\widehat{z}$ is less trivial. In brief, when $\left|\Delta {\mu }_{}^0\right|$ is high, the MCD-optimal allocated effort $\widehat{z}$ decreases when $\left|\Delta {\mu }_{}^0\right|$ increases. This is due to the fact that the resource demand decreases with $\left|\Delta {\mu }_{}^0\right|$. However, there is a critical value for $\left|\Delta {\mu }_{}^0\right|$, below which the MCD-optimal allocated effort $\widehat{z}$increases with $\left|\Delta {\mu }_{}^0\right|$. This is because, although the resource demand still increases when $\left|\Delta {\mu }_{}^0\right|$ decreases, the cost of allocating resources overcompensates the gain in confidence. For such difficult decisions, the system does not follow the demand anymore, and progressively de-motivates the allocation of resources as $\left|\Delta {\mu }_{}^0\right|$ continues to decrease. In brief, the amount $\widehat{z}$ of allocated resources decreases away from a 'sweet spot', which is the absolute prior means' difference that yields the maximal confidence gain per effort unit. Critically, the position of this sweet spot along the $\left|\Delta {\mu }_{}^0\right|$ dimension decreases with $\beta$ and increases with $\alpha$. This is because confidence gain increases, by definition, with effort efficacy, whereas it becomes more costly when $\alpha$ increases.

Second, let us ask what would be the MCD-optimal effort $\widehat{z}$ and confidence ${\overline{P}}_c\left(\widehat{z}\right)$ when $\beta =0$, that is, if the only effect of allocating resources is to perturb the value difference. The ensuing 'γ -effect' is depicted in Appendix 1—figure 3.

Appendix 1—figure 3

Download asset Open asset

In brief, the overall picture is reversed, with a few minor differences. One can see that increasing the absolute prior means' difference $\left|\Delta {\mu }_{}^0\right|$ decreases the resource demand, which eventually decreases the MCD-optimal allocated effort $\widehat{z}$. This can decrease confidence, if $\gamma$ is high enough to overcompensate the effect of variations in $\left|\Delta {\mu }_{}^0\right|$. When no effort is allocated, however, confidence is driven by $\left|\Delta {\mu }_{}^0\right|$, that is, it becomes an increasing function of $\left|\Delta {\mu }_{}^0\right|$. In contrast, variations in the prior variance ${\sigma }_{}^0$ always overcompensate the ensuing changes in effort, which is why confidence always decreases with ${\sigma }_{}^0$. In addition, the amount $\widehat{z}$ of allocated resources decreases away from a sweet prior variance spot, which is the prior variance ${\sigma }_{}^0$ that yields the maximal confidence gain per effort unit. Critically, the position of this sweet spot increases with $\gamma$ and decreases with $\alpha$, for reasons similar to the β-effect.

Now one can ask what happens in the presence of both the β-effect and the γ-effect. If the effort unitary cost $\alpha$ is high enough, the MCD-optimal effort allocation is essentially the superposition of both effects. This means that there are two 'sweet spots': one around some value of $\left|\Delta {\mu }_{}^0\right|$ at high ${\sigma }_{}^0$ (β-effect) and one around some value of ${\sigma }_{}^0$ at high $\left|\Delta {\mu }_{}^0\right|$ (γ-effect). If the effort unitary cost $\alpha$ decreases, then the position of the β-sweet spot increases and that of the β-sweet spot decreases, until they effectively merge together. This is exemplified in Appendix 1—figure 4.

Appendix 1—figure 4

Download asset Open asset

One can see that, somewhat paradoxically, the effort response is now much simpler. In brief, the MCD-optimal effort allocation $\widehat{z}$ increases with the prior variance ${\sigma }_{}^0$ and decreases with the absolute prior means' difference $\left|\Delta {\mu }_{}^0\right|$. The landscape of the ensuing MCD-optimal confidence level ${\overline{P}}_c\left(\widehat{z}\right)$ is slightly less trivial, but globally, it can be thought of as increasing with $\left|\Delta {\mu }_{}^0\right|$ and decreasing with ${\sigma }_{}^0$. Here again, this is because variations in $\left|\Delta {\mu }_{}^0\right|$ and/or ${\sigma }_{}^0$ almost always overcompensate the ensuing effects of changes in allocated effort.

Let $y_t$ be a 6 × 1 vector composed of measured choice confidence, spreading of alternatives, value certainty gain, change of mind, response time, and subjective effort rating on trial $t$. Let $u_t$ be a 4 × 1 vector, whose two first entries are composed of pre-choice value difference (ΔVR⁰) and average value certainty (VCR⁰) ratings, and whose two last entries encode consequential and penalized trials. Finally, let $\phi$ be the set of unknown MCD parameters (i.e., intrinsic effort cost $\alpha$ and effort efficacies $\beta$ and $\gamma$), augmented with condition-effect parameters and affine transform parameters (see below). From a statistical perspective, the MCD model then reduces to the following observation equation:

where $\overline{y}$ denotes data that have been z-scored across trials, ${\epsilon }_t$ are model residuals, and the observation mapping $g\left(\phi ,u_t\right)$ is given by:

where $E\left[\left|\Delta \mu \right||\widehat{z}\right]$ and $V\left[\left|\Delta \mu \right||\widehat{z}\right]$ depend upon $\gamma$ (see Equations 7 and 8 in the main text). In Equation A15, $a_{1:6}$ and $b_{1:6}$ are the unknown offset and slope parameters of the (nuisance) affine transform on MCD outputs. Note that when fitting the MCD model to empirical data, theoretical pre-choice value difference and value certainty ratings are replaced by their empirical proxies, that is, $\Delta {\mu }^0\approx \Delta {\text{VR}}^0$ and $1/{\sigma }_{}^0\approx {\text{VCR}}^0$. In turn, given MCD parameters, Equations A14 and A15 predict trial-by-trial variations in choice confidence, spreading of alternatives, value certainty gain, change of mind, response time, and subjective effort rating from variations in prior moments of value representations. We note that Equation A15 does not yet include condition-specific effects. As we will see, it will be easier to complete the definition of model parameters $\phi$ once we have explained the variational Laplace scheme for parameter estimation.

Recall that the variational Laplace scheme is an iterative algorithm that indirectly optimizes an approximation to both the model evidence $p\left(y|m,u\right)$ and the posterior density $p\left(\phi |y,m,u\right)$, where $m$ is the so-called generative model (i.e., the set of assumptions that are required for inference). The key trick is to decompose the log model evidence into:

where $q\left(\phi \right)$ is any arbitrary density over the model parameters, $D_{KL}$ is the Kullback-Leibler divergence and the so-called free energy $F\left(q\right)$, defined as:

where $S\left(q\right)$ is the Shannon entropy of $q$ and the expectation ${⟨·⟩}_q$ is taken under $q$.

From Equation A16, maximizing the functional $F\left(q\right)$ w.r.t. $q$ indirectly minimizes the Kullback-Leibler divergence between $q\left(\phi \right)$ and the exact posterior $p\left(\phi |y,m\right)$. This decomposition is complete in the sense that if $q\left(\phi \right)=p\left(\phi |y,m\right)$, then $F\left(q\right)=\ln p\left(y|m\right)$.

The variational Laplace algorithm iteratively maximizes the free energy $F\left(q\right)$ under simplifying assumptions (see below) about the functional form of $q$, rendering $q$ an approximate posterior density over model parameters and $F\left(q\right)$ an approximate log model evidence (Daunizeau, 2017a; Friston et al., 2007). The free energy optimization is then made with respect to the sufficient statistics of $q$, which makes the algorithm generic, quick, and efficient.

Under normal i.i.d. model residuals (i.e., ${\epsilon }_t\sim N\left(0,1/\lambda \right)$), the likelihood function writes:

where $\lambda$ is the residuals' precision or inverse variance hyperparameter and the observation mapping $g\left(\phi ,u_t\right)$ is given in Equation A15.

We also use Gaussian priors $p\left(\phi |m\right)=N\left({\eta }_0,{\Sigma }_0\right)$ for model parameters and gamma priors for precision hyperparameters $p\left(\lambda |m\right)=Ga\left({\varpi }_0,{\kappa }_0\right)$.

In what follows, we derive the variational Laplace algorithm under a 'mean-field' separability assumption between parameters and hyperparameters, that is: $q\left(\phi ,\lambda \right)=q\left(\phi \right)q\left(\lambda \right)$. We will see that this eventually yields a Gaussian posterior density $q\left(\phi \right)\approx N\left(\eta ,\Sigma \right)$ on model parameters, and a Gamma posterior density $q\left(\lambda \right)=Ga\left(\varpi ,\kappa \right)$ on the precision hyperparameter.

First, let us note that, under the Laplace approximation, the free energy bound on the log-model evidence can be written as:

where $n_{\phi }$ is the number of parameters, $\Gamma (·)$ is the gamma function, $\psi (·)$ is the digamma function, and $I\left(\phi \right)$ is defined as:

Given the Gamma posterior $q\left(\lambda \right)$ on the precision hyperparameter, $I\left(\phi \right)$ can be simply expressed as follows:

where we have ignored the terms that do not depend upon $\phi$, and $⟨\lambda ⟩=E\left[\lambda |y,m\right]=\varpi /\kappa$ is the posterior mean of the data precision hyperparameter $\lambda$.

The variational Laplace update rule for the approximate posterior density $q\left(\phi \right)$ on model parameters now simply reduces to an update rule for its sufficient statistics:

In Equation A22, the first-order moment $\eta$ of $q\left(\phi \right)$ is obtained from the following Gauss-Newton iterative gradient ascent scheme:

where the gradient and Hessians of $I\left(\phi \right)$ are given by:

At convergence of the above gradient ascent, the approximate posterior density $q\left(\phi \right)$ on the precision hyperparameter is updated as follows:

where $n_t$ is the number of trials.

The variational Laplace scheme alternates between Equations A22 and A25 iteratively until convergence of the free energy.

Now, let us complete the definition of the model parameter vector $\phi ={\phi }_{1:17}$.

First, note that effort efficiency parameters are necessarily positive. Enforcing this constraint can be done using the following simple change of variable in Equation A15: $\beta =\exp \left({\phi }_1\right)$ and $\gamma =\exp \left({\phi }_2\right)$. In other words, ${\phi }_{1:2}$ effectively measure efficiency parameters in log-space. Second, recall that we want to insert condition-specific effects in the model. More precisely, we expect ‘consequential’ decisions to be more important than ‘neutral’ ones, and ‘penalized’ decisions effectively include an extraneous cost-of-time term. One can model the former condition effect by making $R$ (Equation 2 in the main text) sensitive to whether the decision is consequential ($u_{}^{\left(c\right)}=1$) or not ($u_{}^{\left(c\right)}=0$), that is: $R_t=\exp \left({\phi }_3{u}_t^{\left(c\right)}\right)$, where $t$ indexes trials, and ${\phi }_3$ is the unknown weight of consequential choices on decision importance. This parameterization makes decision importance necessarily positive, and forces non-consequential trials to act as reference choices (in the sense that their decision importance is set to 1). We proxy the latter condition effect by making the effort unitary cost a function of whether the decision is penalized ($u_{}^{\left(p\right)}=1$) or not ($u_{}^{\left(p\right)}=0$), that is: ${\alpha }_t=\exp \left({\phi }_4+{\phi }_5{u}_t^{\left(p\right)}\right)$, where ${\phi }_4$ is the unknown intrinsic effort cost (in log-space), and ${\phi }_5$ is the unknown weight of penalized choices on effort cost. The remaining parameters ${\phi }_{6:17}$ lump the offsets ($a_{1:6}$) and log-slopes ($\log b_{1:6}$: this enforces a positivity constraint on slope parameters) of the affine transform.

Finally, we set the prior probability density functions on model parameters and hyperparameters as follows:

• $p\left({\phi }_i|m\right)=N\left(0,{10}^2\right)\forall i$, that is, the prior mean of model parameters is ${\eta }_0=0$ and their prior variance is ${\Sigma }_0={10}^2\times I$.

• $p\left(\lambda |m\right)=Ga\left(1,1\right)$. Since the data has been z-scored prior to model inversion, this ensures that the prior and likelihood components of $I\left(\phi \right)$ are balanced when the variational Laplace algorithm starts.

This completes the description of the variational Laplace approach to MCD inversion. For more details, we refer the interested reader to the existing literature on variational approaches to approximate Bayesian inference (Beal, 2003; Daunizeau, 2017b; Friston et al., 2007). We note that the above variational Laplace approach is implemented in the opensource VBA toolbox (Daunizeau et al., 2014).

In what follows, we use Monte-Carlo numerical simulations to evaluate the ability of this approach to recover MCD parameters. Our parameter recovery analyses proceed as follows. First, we sample a set of model parameters $\phi$ under a standard i.i.d. normal distribution. Here, we refer to ${\phi }_{ij}$ as i^th element of $\phi$ at the j^th Monte-Carlo simulation. Second, for each of these parameter set ${\phi }_{·j}$, we simulate a series of N=100 decision trials according to Equation A14 and A15 above (under random prior moments of value representations). Note that we set the variance of model residuals ($\epsilon$ in Equation A14) to match the average correlation between MCD predictions and empirical data (about 20%, see Figure 4 in the main text). We also used the same rate of neutral, consequential, and penalized choices as in our experiment. Third, we fit the model to the resulting simulated data (after z-scoring) and extract parameter estimates ${\eta }_{·j}$ (at convergence of the variational Laplace approach). We repeat these three steps 1000 times, yielding a series of 1000 simulated parameter sets, and their corresponding 1000 estimated parameters sets. Should ${\eta }_{·j}^{}\approx {\phi }_{·j}\forall j$, then parameter recovery would be perfect. Appendix 1—figure 5 compares simulated and estimated parameters to each other across Monte-Carlo simulations. Note that we only report recovery results for ${\phi }_{1:5}$, since we do not care about nuisance affine transform parameters.

We also quantify pairwise non-identifiability issues, which arise when the estimation method confuses two parameters with each other. We do this using the so-called ‘recovery matrices’, which summarize whether variations (across the 1000 Monte-Carlo simulations) in estimated parameters faithfully capture variations in simulated parameters. We first z-score simulated and estimated parameters across Monte-Carlo simulations. We then regress each estimated parameter against all simulated parameters through the following multiple linear regression model:

where ${\theta }_{ii\text{'}}^{}$ are regression weights, and ${\varsigma }_{ij}^{}$ are regression residuals. Here, regression weights are partial correlation coefficients between simulated and estimated parameters (across Monte-Carlo simulations). More precisely, ${\theta }_{ii\text{'}}^{}$ quantifies the impact that variations of the simulated parameter ${\phi }_{i\text{'}·}^{}$ have on variations of the estimated parameter ${\eta }_{i·}^{}$, conditional on all other simulated parameters. Would parameters be perfectly identifiable, then ${\theta }_{ii}^{}\approx 1$ and ${\theta }_{ii\text{'}}^{}\approx 0\forall i\text{'}\ne i$. Pairwise non-identifiability issues arise when ${\theta }_{ii\text{'}}^{}\ne 0$. In other words, the regression model in Equation A26 effectively decomposes the observed variability in the series of estimated parameter ${\eta }_{i·}^{}$ into 'correct variations' that are induced by variations in the corresponding simulated parameter ${\phi }_{i·}^{}$, and 'incorrect variations' that are induced by the remaining simulated parameters ${\phi }_{i\text{'}·}^{}$ (with $i\text{'}\ne i$). This analysis is then summarized in terms of 'recovery matrices', which simply report the squared regression weights ${\theta }_{ii\text{'}}^2$ for each simulated parameter (see right panel of Appendix 1—figure 5).