Two time scales of adaptation in human learning rates
- Department of Experimental Psychology, Ghent University, Belgium
- Institute of Psychology, Universität Hamburg, Germany
Figures
Figure 1
Experimental design.
(A) Participants went fishing for crabs on six different locations around an island which differed in terms of optimal initial learning rate. At the beginning of each block, µs was sampled from the prior distribution µs ~ N(µp, ), truncated between µp ± 1.65 * σp, with µp = the centre of the screen. Subsequently, on each trial, once a cage was dropped, five crabs appeared and spread out from one location in the sand sampled from the sampling distribution S ~ N(µs, ), truncated between µs ± 1.65 * σs, each of which was either caught by the cage or ran away. (B) Overview of optimal learning rates for performing the task for 1 block of trials according to the Kalman filter assuming that measurement uncertainty = and estimate uncertainty = on trial 1 (see Model estimation and selection for details). (C) Overview of the trial procedure (see also Video 1). At the beginning of each block, participants were taken to one of six locations around the island. On each trial, participants positioned the cage somewhere along the x-axis of the screen and dropped it. As the cage sank, five crabs appeared out of one point in the sand and spread out. When the cage reached the ocean floor, crabs caught by the cage remained there while the other crabs ran away. At the start of the next trial, the cage was again at the top of the screen, but at the same x-coordinate where it was dropped in the last trial, and a little heap of sand was left where the five crabs had appeared out of the sand on the last trial.
Figure 2
Behavioural results Experiment 1.
(A) Group-level mean of each participant’s median learning rate for each trial in each environment (see Behavioural data analyses for details). Error bars represent standard errors of the means. (B) Detailed overview of all participants’ median initial learning rates. Evolution of (group-level mean) learning rates over trials (within blocks) for the first half (C) and the second half (D) of the task separately. (E) Moving-window analysis of trial 2 learning rate across blocks.
Figure 3
Bai model estimation results Experiment 1.
The density plots on the left side of each subfigure show the full posterior densities over the means of the group-level distributions of the relevant parameters. The scatter plots on the right side of each subfigure show the means of all individual-level posterior distributions of the relevant parameters.
Figure 4
Behavioural results Experiment 2.
(A) Group-level mean of each participant’s median learning rate for each trial in each environment (see Behavioural data analyses for details). Error bars represent standard errors of the means. (B) Detailed overview of all participants’ median initial learning rates. One participant’s median initial learning rate of −0.674 in the high noise environment is not visible on the plot. (C–F) Evolution of (group-level mean) learning rates over trials (within blocks) for each quarter of the task separately. (G) Moving-window analysis of second-trial learning rate.
Figure 5
Bai model estimation results Experiment 2.
The density plots on the left side of each subfigure show the full posterior densities over the means of the group-level distributions of the relevant parameters. The scatter plots on the right side of each subfigure show the means of all individual-level posterior distributions of the relevant parameters.
Figure 6
Representational similarity analysis (RSA) of the fMRI data.
(A) Spatial location representational dissimilarity matrix (RDM). (B) Learning rate RDM. (C) Brain map of significant t-values resulting from the whole-brain searchlight RSA of fMRI data acquired while participants had just been transported to the next location around the island (correlation with learning rate RDM). Interaction effect between time (first vs. second half of task) and RDM (spatial location vs. learning rate RDM) in the occipital cortex, defined as the cluster of significant voxels found in the aforementioned whole-brain searchlight RSA (D); the central orbitofrontal cortex (OFC) as defined by Kahnt et al., 2012, based on connections to other brain regions (E); and the ventral striatum, defined as the left and right nucleus accumbens according to the AAL atlas (F). Grey dots represent individual-level Kendall’s tau-values, while black dots and error bars represent group-level means and SEs of the means, respectively.
Figure 7
Results of the analyses of the effect of prediction error on the fMRI data.
Brain map of significant t-values resulting from the whole-brain (univariate) tests of voxel activity being (parametrically) modulated by prediction error on trial 1 (A) and trial 2 (B). Interaction effect between time (first vs. second half of task) and environment (low vs. medium vs. high measurement noise) on the modulating effect of prediction error on trial 1 (C) and trial 2 (D) on ventral striatum activity. This region of interest (ROI) was defined as the left and right nucleus accumbens according to the AAL atlas. Grey dots represent individual-level general linear model (GLM) beta-values, while black dots and error bars represent group-level means and SEs of the means, respectively.
Author response image 1
Author response image 2
Videos
Video 1
Experimental paradigm.
Tables
Table 1
Model comparison.
| Model | LOOIC | SE | ∆LOOIC | ∆SE |
|---|---|---|---|---|
| Environment-specific Bai model | 28,735 | 404 | 0 | 0 |
| Non-environment-specific Bai model | 28,582 | 441 | 153 | 65 |
| Environment-specific Rescorla–Wagner model | 28,436 | 398 | 299 | 67 |
| Non-environment-specific Rescorla–Wagner model | 28,172 | 440 | 563 | 112 |
| Environment-specific Kalman filter | 27,853 | 493 | 882 | 211 |
| Non-environment-specific Kalman filter | 27,698 | 521 | 1037 | 211 |
-
Note. Models are ranked in descending order according to how well they fit the data. LOOIC refers to a model’s approximated expected log pointwise predictive density. Higher values indicate higher out-of-sample predictive fit. SE refers to the standard error of a model’s LOOIC. ∆LOOIC refers to the difference between a model’s LOOIC and the top ranked model’s LOOIC. ∆SE refers to the standard error of the difference between a model’s LOOIC and the top ranked model’s LOOIC.
Table 2
Model comparison.
| Model | LOOIC | SE | ∆LOOIC | ∆SE |
|---|---|---|---|---|
| Environment-specific Bai model | 34,725 | 142 | 0 | 0 |
| Non-environment-specific Bai model | 34,682 | 158 | 43 | 34 |
| Environment-specific Kalman filter | 34,592 | 160 | 133 | 42 |
| Environment-specific Rescorla–Wagner model | 34,567 | 136 | 158 | 32 |
| Non-environment-specific Kalman filter | 34,523 | 174 | 202 | 49 |
| Non-environment-specific Rescorla–Wagner model | 34,434 | 154 | 291 | 51 |
-
Note. Models are ranked in descending order according to how well they fit the data. LOOIC refers to a model’s approximated expected log pointwise predictive density. Higher values indicate higher out-of-sample predictive fit. SE refers to the standard error of a model’s LOOIC. ∆LOOIC refers to the difference between a model’s LOOIC and the top ranked model’s LOOIC. ∆SE refers to the standard error of the difference between a model’s LOOIC and the top ranked model’s LOOIC.
Author response table 1
| Model | LOOIC | SE | DLOOIC | △SE |
|---|---|---|---|---|
| Environment-specific Bai model | 28735 | 404 | 0 | 0 |
| Non-environment-specific Bai model | 28582 | 441 | 153 | 65 |
| Environment-specific Rescorla–Wagner model | 28436 | 398 | 299 | 67 |
| Non-environment-specific Rescorla–Wagner model | 28172 | 440 | 563 | 112 |
| Environment-specific Kalman filter | 27853 | 493 | 882 | 211 |
| Non-environment-specific Kalman filter | 27698 | 521 | 1037 | 211 |
Author response table 2
| Model | LOOIC | SE | /_\LOOIC | DeltaSE |
|---|---|---|---|---|
| Environment-specific Bai model | 34725 | 142 | 0 | 0 |
| Non-environment-specific Bai model | 34682 | 158 | 43 | 34 |
| Environment-specific Kalman filter | 34592 | 160 | 133 | 42 |
| Environment-specific Rescorla–Wagner model | 34567 | 136 | 158 | 32 |
| Non-environment-specific Kalman filter | 34523 | 174 | 202 | 49 |
| Non-environment-specific Rescorla–Wagner model | 34434 | 154 | 291 | 51 |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Two time scales of adaptation in human learning rates
eLife 14:RP108223.
https://doi.org/10.7554/eLife.108223.3
