Novelty and uncertainty differentially drive exploration across development
- New York University, United States
- University of Maryland, United States
- Temple University, United States
- Leiden University, Netherlands
- Geisinger Health System, United States
- Caltech, United States
Figures
Figure 1
Exploration task.
Participants completed 10 blocks of 15 choice trials in which they selected between two of three ‘hiding spots’ to find gold coins. Within each block, two hiding spots had been previously encountered and one was completely novel. Each block took place within a different ‘territory’ in which a new creature hid coins. Each creature had different preferred hiding spots, such that the reward probabilities associated with each option were reset at the beginning of each block.
Figure 2
Exploration task performance.
(A) Participants’ (n = 122) proportion of optimal choices as a function of age and block difficulty. (B) Reward participants earned in easy and hard blocks of the task across age. In both plots, points represent participant averages in each block condition, lines show the best-fitting linear regression modeling the effect of age, and the shaded regions around them represent 95% confidence intervals. The dotted lines indicate chance-level performance.
Figure 3
Influence of expected value, uncertainty, and novelty on choice behavior across age.
(A) The proportion of all trials in which participants (n = 122) chose the left, more novel, and more uncertain choice option as a function of the expected value difference between the options. Participants were more likely to choose options with greater expected value, higher novelty, and lower uncertainty (ps < 0.001). The influence of novelty did not vary across age, whereas uncertainty was more aversive in older participants (p < 0.001). Points indicate age group means and error bars show standard errors. (B) The proportion of similar-expected-value trials (difference between the two options <0.05) in which participants chose the more novel and more uncertain option, plotted as a function of continuous age. The lines show the best-fitting linear regression lines and the shaded regions around them represent 95% confidence intervals.
Figure 4
Influence of expected value, uncertainty, and novelty on choice response times across age.
(A) Participants (n = 122) were faster to select options with higher expected values, (B) slower to select options with greater uncertainty, and (C) slower to select options with higher novelty. Individual points in panels A–C show individual log-transformed response times on each trial. The lines in panels A–C show the best-fitting linear regression lines and the shaded regions around them represent 95% confidence intervals. (D) The influence of expected value on response times did not vary across age, whereas younger participants demonstrated (E) a weaker influence of uncertainty on response times (p = 0.005) and (F) a stronger influence of novelty on response times (p = 0.043). The lines in panels D–E show predictions from a linear mixed-effects model and the shaded regions around them represent 95% confidence intervals.
Figure 5
Model simulations.
The average proportion of similar-expected-value trials (with expected value magnitude differences <0.05) in which both real (n = 122) and simulated (n = 122) participants chose the more novel and more uncertain option. The shaded regions show the empirical data and best-fitting model for each age group. Error bars represented the standard error across participant means. The novelty bias and familiarity-gated uncertainty model each had three free parameters, while the baseline model had two, and the novelty + uncertainty bias model had four.
Figure 6
Model recovery results.
(A) Confusion matrices showing the probability of each fitted model given a simulated model. Across simulated datasets, the most frequent, best-fitting model usually matched the model that was used to generate the data. (B) Inversion matrices showing the probability of each simulated model given a fitted model. Together, these results indicate that the familiarity-gated uncertainty model could not be distinguished from the familiarity-gated uncertainty model with an additional novelty parameter.
Appendix 1—figure 1
Influence of expected value, uncertainty, and novelty on choice behavior across age.
The proportion of all trials in which participants (n = 122) chose the left, more novel, and more uncertain choice option as a function of the expected value difference between the options. Participants were more likely to choose options with greater expected value, higher novelty, and lower uncertainty (ps < 0.001). The influence of novelty did not vary across linear and quadratic age, whereas uncertainty was more aversive in older participants (p = 0.027). Uncertainty aversion did not vary as a function of quadratic age (p = 0.076). Points indicate age group means and error bars show standard errors.
Appendix 1—figure 2
Model simulations of optimal choices and reward earned.
The average proportion of trials in which both real and simulated participants (A) selected the optimal choice option and (B) earned reward. In both sets of plots, points represent participant averages in each block condition, lines show the best-fitting linear regression modeling the effect of age, and the shaded regions around them represent 95% confidence intervals. The dotted lines in panel A indicate chance-level performance.
Appendix 1—figure 3
Influence of expected value, uncertainty, and novelty on choice behavior across age and model simulations.
The proportion of all trials in which the real and simulated participants chose the left, more novel, and more uncertain choice option as a function of the expected value difference between the options. Points indicate age group means and error bars show standard errors. The thick black outlines indicate the best-fitting model for each age group.
Appendix 1—figure 4
Influence of uncertainty, and novelty on choice behavior across age and model simulations for trials in which the choice options had similar-expected-values.
The proportion of similar-expected-value trials (difference between the two options <0.05) in which participants chose the more novel and more uncertain option, plotted as a function of continuous age. The lines show the best-fitting linear regression lines and the shaded regions around them represent 95% confidence intervals.
Tables
Appendix 1—table 1
Influences on exploratory choice including within-block trial number.
| Odds ratio | 95% confidence interval | Χ2 | p | |
|---|---|---|---|---|
| Intercept | 1.00 | [.95, 1.05] | ||
| Expected Value | 3.39 | [2.98, 3.86] | 164.5 | <0.001 |
| Uncertainty | 0.87 | [0.81, 0.93] | 15.6 | <0.001 |
| Novelty | 1.38 | [1.31, 1.46] | 107.1 | <0.001 |
| Expected Value × Trial | 0.82 | [0.78, 0.86] | 72.5 | <0.001 |
| Uncertainty × Trial | 1.00 | [0.96, 1.05] | 0.03 | 0.858 |
| Novelty × Trial | 1.03 | [1.00, 1.07] | 4.2 | 0.041 |
| Expected Value × Age | 1.23 | [1.08, 1.40] | 9.8 | 0.002 |
| Uncertainty × Age | 0.88 | [0.83, 0.94] | 14.1 | <0.001 |
| Novelty × Age | 1.04 | [0.98, 1.09] | 1.8 | 0.185 |
| EV × Trial × Age | 1.00 | [0.96, 1.05] | 0.0 | 0.906 |
| Uncertainty × Trial × Age | 1.02 | [0.98, 1.06] | 0.7 | 0.409 |
| Novelty × Trial × Age | 1.02 | [0.99, 1.06] | 2.1 | 0.145 |
Appendix 1—table 2
Influences on exploratory choice including WASI scores.
| Odds ratio | 95% confidence interval | Χ2 | p | |
|---|---|---|---|---|
| Intercept | 1.00 | [0.95, 1.05] | ||
| Expected Value | 3.31 | [2.92, 3.75] | 167.4 | <0.001 |
| Uncertainty | 0.87 | [0.81, 0.93] | 16.0 | <0.001 |
| Novelty | 1.35 | [1.29, 1.42] | 104.0 | <0.001 |
| Expected Value × Age | 1.29 | [1.14, 1.46] | 15.8 | <0.001 |
| Uncertainty × Age | 0.89 | [0.83, 0.95] | 12.6 | <0.001 |
| Novelty × Age | 1.04 | [0.99, 1.09] | 2.1 | 0.145 |
| Expected Value × WASI | 1.33 | [1.17, 1.50] | 19.1 | <0.001 |
| Uncertainty × WASI | 0.96 | [0.90, 1.02] | 1.5 | 0.217 |
| Novelty × WASI | 1.06 | [1.01, 1.11] | 5.0 | 0.025 |
| EV × Age × WASI | 1.06 | [0.94, 1.19] | 1.1 | 0.288 |
| Uncertainty × Age × WASI | 0.96 | [0.90, 1.02] | 2.0 | 0.158 |
| Novelty × Age × WASI | 1.01 | [0.97, 1.06] | 0.4 | 0.532 |
Appendix 1—table 3
Influences on exploratory choice including block number.
| Odds ratio | 95% confidence interval | Χ2 | p | |
|---|---|---|---|---|
| Intercept | 1.00 | [0.96, 1.05] | ||
| Expected Value | 3.22 | [2.82, 3.67] | 155.6 | <0.001 |
| Uncertainty | 0.88 | [0.83, 0.94] | 12.3 | <0.001 |
| Novelty | 1.35 | [1.29, 1.41] | 105.1 | <0.001 |
| Expected Value × Age | 1.22 | [1.07, 1.39] | 9.0 | 0.003 |
| Uncertainty × Age | 0.89 | [0.83, 0.95] | 11.4 | <0.001 |
| Novelty × Age | 1.02 | [0.98, 1.07] | 1.0 | 0.330 |
| Expected Value × Block | 0.99 | [0.95, 1.03] | 0.2 | 0.666 |
| Uncertainty × Block | 1.02 | [0.98, 1.06] | 0.7 | 0.398 |
| Novelty × Block | 0.98 | [0.95, 1.02] | 1.0 | 0.310 |
| EV × Age × Block | 1.00 | [0.96, 1.04] | 0.0 | 0.913 |
| Uncertainty × Age × Block | 1.01 | [0.97, 1.05] | 0.3 | 0.580 |
| Novelty × Age × Block | 1.00 | [0.96, 1.03] | 0.0 | 0.924 |
Appendix 1—table 4
Influences on exploratory choice including non-reset expected value.
| Odds ratio | 95% CI | Χ2 | p | |
|---|---|---|---|---|
| Intercept | 0.997 | [0.948, 1.05] | ||
| Expected Value | 3.04 | [2.65, 3.49] | 141.0 | <0.001 |
| Non-reset Expected Value | 1.09 | [1.04, 1.15] | 10.5 | 0.001 |
| Uncertainty | 0.902 | [0.842, 0.967] | 8.0 | 0.005 |
| Novelty | 1.37 | [1.31, 1.45] | 100.0 | <0.001 |
| Expected Value × Age | 1.22 | [1.06, 1.46] | 7.8 | 0.005 |
| Non-reset Expected Value × Age | 0.981 | [0.930, 1.04] | 0.5 | 0.490 |
| Uncertainty × Age | 0.884 | [0.825, 0.948] | 11.7 | <0.001 |
| Novelty × Age | 1.02 | [0.972, 1.08] | 0.7 | 0.393 |
Appendix 1—table 5
Model-derived effects (and standard errors) of expected value, novelty, and uncertainty on log-transformed reaction times across age groups.
| Expected value coefficient | Novelty coefficient | Uncertainty coefficient | |
|---|---|---|---|
| Children | −0.024 (0.01) | 0.074 (0.01) | 0.043 (0.01) |
| Adolescents | −0.023 (0.01) | 0.060 (0.01) | 0.068 (0.01) |
| Adults | −0.041 (0.01) | 0.053 (0.01) | 0.085 (0.01) |
Appendix 1—table 6
Influences on exploratory choice including quadratic age and WASI scores.
| Odds ratio | 95% confidence interval | Χ2 | p | |
|---|---|---|---|---|
| Intercept | 1.00 | [0.95, 1.05] | ||
| Expected Value | 3.30 | [2.92, 3.74] | 168.0 | <0.001 |
| Uncertainty | 0.87 | [0.82, 0.93] | 16.3 | <0.001 |
| Novelty | 1.36 | [1.30, 1.42] | 111.1 | <0.001 |
| Expected Value × Age | 1.82 | [0.76, 4.33] | 1.8 | 0.183 |
| Uncertainty × Age | 0.60 | [0.38, 0.94] | 4.9 | 0.027 |
| Novelty × Age | 0.71 | [0.51, 0.98] | 4.2 | 0.040 |
| Expected Value × Age2 | 0.70 | [0.29, 1.68] | 0.6 | 0.429 |
| Uncertainty × Age2 | 1.51 | [0.96, 2.37] | 3.2 | 0.076 |
| Novelty × Age2 | 1.45 | [1.05, 2.01] | 4.9 | 0.026 |
| Expected Value × WASI | 1.33 | [1.18, 1.50] | 19.2 | <0.001 |
| Uncertainty × WASI | 0.96 | [0.89, 1.02] | 1.9 | 0.167 |
| Novelty × WASI | 1.06 | [1.01, 1.11] | 6.6 | 0.010 |
| EV × Age × WASI | 1.37 | [0.59, 3.17] | 0.5 | 0.463 |
| Uncertainty × Age × WASI | 0.71 | [0.46, 1.10] | 2.3 | 0.128 |
| Novelty × Age × WASI | 1.48 | [1.08, 2.02] | 5.9 | 0.016 |
| EV × Age2 × WASI | 0.77 | [0.34, 1.74] | 0.4 | 0.534 |
| Uncertainty × Age2 × WASI | 1.35 | [0.89, 2.06] | 2.0 | 0.162 |
| Novelty × Age2 × WASI | 0.70 | [0.52, 0.94] | 5.2 | 0.022 |
Appendix 1—table 7
Relation between parameter estimates from novelty bias model and memory accuracy in children.
| Estimate | SE | t | p | |
|---|---|---|---|---|
| Intercept | 0.25 | 0.03 | ||
| Age | −0.07 | 0.03 | −2.51 | 0.020 |
| Inverse Temperature | −0.03 | 0.04 | −0.71 | 0.484 |
| Learning Rate | −0.03 | 0.03 | −1.00 | 0.330 |
| Novelty Bias | −0.01 | 0.03 | −0.30 | 0.768 |
| Inverse Temperature × Age | 0.03 | 0.04 | 0.78 | 0.443 |
| Learning Rate × Age | 0.00 | 0.03 | 0.05 | 0.961 |
| Novelty Bias × Age | −0.01 | 0.04 | −0.27 | 0.793 |
Appendix 1—table 8
Relation between parameter estimates from familiarity-gated uncertainty model and memory accuracy in adolescents and adults.
| Estimate | SE | t | p | |
|---|---|---|---|---|
| Intercept | 0.25 | 0.01 | ||
| Age | −0.01 | 0.01 | −0.71 | 0.483 |
| Inverse Temperature | −0.01 | 0.01 | −0.76 | 0.450 |
| Learning Rate | −0.02 | 0.01 | −1.14 | 0.259 |
| Uncertainty Bias | −0.01 | 0.01 | 0.41 | 0.685 |
| Inverse Temperature × Age | 0.02 | 0.02 | 1.53 | 0.131 |
| Learning Rate × Age | 0.01 | 0.01 | −0.39 | 0.695 |
| Uncertainty Bias × Age | −0.01 | 0.01 | −0.70 | 0.489 |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Novelty and uncertainty differentially drive exploration across development
eLife 12:e84260.
https://doi.org/10.7554/eLife.84260
