Human exploration strategically balances approaching and avoiding uncertainty

Prior distributions are given in Stan syntax. All predictors used in models were centered and scaled prior to fitting, so that the same priors can apply to all parameters.

The model can be summarized with the following R syntax formula: $accuracy\sim 0.5+0.5\ast inv\mathrm{\_}logit(1+final>uncertainty+$ $(1+final>uncertainty|participant))$, where $inv\mathrm{\_}logit$ is the inverse logit function. This functional form limits predicted accuracy between 0.5 and 1.0, since guessing-level accuracy on a two-alternative forced-choice test is 0.5. Since accuracy is a binary variable, this regression was fit with a Bernoulli likelihood.

The model can be summarized with the following R syntax formula: $confidence\sim 0+finaluncertainty\ast accuracy+$ $(1+finaluncertainty\ast accuracy|PID)$. This model was fit as an ordered-logistic regression, with four threshold variables since confidence was rated on a 5-point Likert scale (Bürkner and Vuorre, 2019).

The model can be summarized with the following R syntax formula: $tableonrightchosen\sim 1+\mathrm{\Delta }uncertainty+$ $(1+\mathrm{\Delta }uncertainty|participant)$. This model was fit as a logistic regression.

The model can be summarized with the following R syntax formula: $tableonrightchosen\sim 1+\mathrm{\Delta }EIG+$ $(1+\mathrm{\Delta }EIG|participant)$. This model was fit as a logistic regression.

The model can be summarized with the following R syntax formula: $tableonrightchosen\sim 1+\mathrm{\Delta }exposure+$ $(1+\mathrm{\Delta }exposure|participant)$. This model was fit as a logistic regression.

This model can be summarized with the following formula: $logit(P(tableonrightchosen))=Intercept$ $+b_1\ast \mathrm{\Delta }uncertainty$ $+b_2\ast (overalluncertainty-\omega )$ $\ast step(overalluncertainty-\omega )\ast \mathrm{\Delta }uncertainty$, where step is the step function, $\omega =-2ln(0.5)\ast inv\mathrm{\_}logit(\alpha )$. The intercept and parameters b₁, b₂, and α all vary by participant. This model was fit as a logistic regression.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+approachtendency$. For the tendency to approach uncertainty, we computed the mean posterior approach parameter for each participant in the model described in Appendix 3—table 6. The model described here was fit as a logistic regression with binomial likelihood.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+approachtendency$. For the tendency to avoid uncertainty when overall uncertainty is high, we computed for each participant the area under the curve of the uncertainty approach/avoid graph, averaging across the posterior of the model described in Appendix 3—table 6. The model described here was fit as a logistic regression with binomial likelihood.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+avoidtendency+approachtendency$. For the tendency to avoid uncertainty when overall uncertainty is high, we computed for each participant the area under the curve of the uncertainty approach/avoid graph, averaging across the posterior of the model described in Appendix 3—table 6. The model described here was fit as a logistic regression with binomial likelihood.

We used a drift-diffusion model to formalize the dependence of RTs and choice on evidence. A drift-diffusion model is one variant in the sequential sampling family of models. The model posits that samples of momentary evidence are integrated over time. The expectation of the momentary evidence distribution is termed the drift rate µ, and its standard deviation is termed the diffusion coefficient. The decision is made when integrated evidence reaches an upper or lower bound (±B), whose sign determines the choice. Processes external to decision making are modeled by t_ND, a constant added to the RT. In this model, µ is allowed to depend linearly on Δ-uncertainty, $\mu ={\mu }_0+\kappa \cdot \mathrm{\Delta }-uncertainty$, such that $\kappa$ captures the dependence of drift rate on Δ-uncertainty, and µ₀ is a general bias to make rightward or leftward choices.

Prior to fitting the model to the data, we excluded trials for which overall uncertainty was above the threshold estimated for each participant by the model described in Appendix 3—table 6. As we find qualitatively different choice behavior above the threshold, we couldn’t justify modeling these trials together with the majority of trials. Fitting a piecewise regression DDM model was beyond the capabilities of current software.

This model can be summarized with the following R syntax formula: $testaccuracy\sim 1+finaluncertainty+$ $B+\kappa +(1+finaluncertainty|participant)$. This model was fit as a logistic regression. As B and $\kappa$ are parameters estimated from the model described in Appendix 3—table 10, we took into account our error in measuring them when using them as predictors in this model. Thus, the posterior distribution for each participant’s B and $\kappa$ parameters was summarized as a mean and standard deviation. These summary statistics were used to approximate the posterior as a normal distribution from which a latent variable was drawn during the estimation of this model. This method propagates the uncertainty in the values of B and $\kappa$ into the estimates reported here. Prior to using this method, we inspected the posteriors from the model summarized in Appendix 3—table 10, and made sure the normal distribution is an adequate approximation for these posteriors.

This model can be summarized with the following formula: $logit(P(tableonrightchosen))=Intercept+$ $b1\cdot \mathrm{\Delta }-uncertainty+b2\cdot (overalluncertainty-\omega )$ $\cdot step(overalluncertainty-\omega )\cdot \mathrm{\Delta }-uncertainty+b3\cdot repeatonright$, where step is the step function, $\omega =-2ln(0.5)\cdot inv\mathrm{\_}logit(\alpha )$. The intercept and parameters b1, b2, b3, and α all vary by participant, and their correlations across participants are modeled. This model was fit as a logistic regression.

Comparing the three models described in Appendix 3—tables 3–5, and the same models with the addition of a term coding for whether the repeat choice was presented on the right. Values represent the difference in expected log predictive density (ELPD) from the best fitting model. For both samples, adding the choice repetition term improves model fit substantially, while not changing the order of the three strategies.

We refit the DDM described in Appendix 3—table 10, allowing both B and $\kappa$ to vary by whether the choice was a repeat or switch choice (see main text for definition).

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+tendencytorepeat$. For the tendency to repeat, we computed the mean posterior parameter for each participant in the model described in Appendix 3—tables 1 and 2 This model was fit as a logistic regression with binomial likelihood.

The model can be summarized with the following R syntax formula: $logRT\sim 1+memorylag\ast repeatonright+$ $(1+memorylag\cdot repeatonright|participant)$. This model was fit as a lognormal regression.

The model can be summarized with the following R syntax formula: $tableonrightchosen\sim 1+\mathrm{\Delta }-uncertainty$ $\cdot memorylag+memorylag:repeatonright+$ $(1+\mathrm{\Delta }-uncertainty\cdot memorylag+$ $memorylag:repeatonright|participant)$. This model was fit as a logistic regression. For brevity, the correlations in participant-wise variability are omitted from this table.

We refit the piecewise regression model described in Appendix 3—table 6, accounting for a possible interaction between Δ-uncertainty and trial number. We find no significant interaction in the pre-registered sample, nor the preliminary sample. All other terms in the model remained practically the same.

The model can be summarized with the following formula: $logit(P(tableonrightchosen))=Intercept+$ $b1\cdot \mathrm{\Delta }-uncertainty+b2\cdot (overalluncertainty-\omega )$ $\cdot step(overalluncertainty-\omega )\cdot \mathrm{\Delta }$ $-uncertainty+b4\cdot trial\mathrm{\#}\cdot \mathrm{\Delta }-uncertainty$, where step is the step function, $\omega =-2ln(0.5)\cdot inv\mathrm{\_}logit(\alpha )$. The intercept and parameters b1, b2, b4, and α all vary by participant. This model was fit as a logistic regression.

The model reported in Appendix 3—table 14 captures the tendency to repeat previous choices by allowing both the dependence of RT on Δ-uncertainty and the bound height parameters to vary by whether the choice was a repeat or switch choice (last row in this table). Here, it is compared against the simpler models nested within it. For both samples, the full model is favored over the partial models, as is indicated by lower deviance information criterion (DIC) values. DIC values are derived from the likelihood of the data given estimated parameters, and the effective number of parameters in the model. Absolute values of DIC depend on sample size and the attributes of the noise distribution. Accordingly, DIC values should only be compared between models fit to the same dataset.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+uncertainty$ , where uncertainty is the mean posterior parameter for each participant in the model described in Appendix 3—table 3. The model described here was fit as a logistic regression with binomial likelihood.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+EIG$, where EIG is the mean posterior parameter for each participant in the model described in Appendix 3—table 4. The model described here was fit as a logistic regression with binomial likelihood.

The model can be summarized with the following R syntax formula: $testaccuracy\sim 1+exposure$, where exposure is the mean posterior parameter for each participant in the model described in Appendix 3—table 5. The model described here was fit as a logistic regression with binomial likelihood.

Comparing the three models described in Appendix 3—tables 20–22. Values represent the difference in expected log predictive density (ELPD) from the best fitting model. For both samples, individual differences in the uncertainty coefficient bet predict test accuracy.