Experimental design.

A: Participants went fishing for crabs on six different locations around an island which differed in terms of optimal initial learning rate. At the beginning of each block, µs was sampled from the prior distribution µs ∼ N(µp, ,), truncated between µp ± 1.65 * σp, with µp = the centre of the screen. Subsequently, on each trial, once a cage was dropped, five crabs appeared and spread out from one location in the sand sampled from the sampling distribution S ∼ N(µs, ), truncated between µs ± 1.65 * σs, each of which was either caught by the cage or ran away. B: Overview of optimal learning rates for performing the task for 1 block of trials according to the Kalman filter assuming that measurement uncertainty = and estimate uncertainty = on trial 1 (See Model estimation and selection for details). C: Overview of the trial procedure. At the beginning of each block, participants were taken to one of six locations around the island. On each trial, participants positioned the cage somewhere along the x-axis of the screen and dropped it. As the cage sank, five crabs appeared out of one point in the sand and spread out. When the cage reached the ocean floor, crabs caught by the cage remained there while the other crabs ran away. At the start of the next trial, the cage was again at the top of the screen, but at the same x-coordinate where it was dropped in the last trial, and a little heap of sand was left where the five crabs had appeared out of the sand on the last trial.

Behavioural results Experiment 1.

A: Group-level mean of each participant’s median learning rate for each trial in each environment (See Behavioural data analyses for details). Error bars represent standard errors of the means. B: Detailed overview of all participants’ median initial learning rates. C-D: Evolution of (group-level mean) learning rates over trials (within blocks) for the first half (C) and the second half (D) of the task separately.

Bai model estimation results Experiment 1.

The density plots on the left side of each subfigure show the full posterior densities over the means of the group-level distributions of the relevant parameters. The scatter plots on the right side of each subfigure show the means of all individual-level posterior distributions of the relevant parameters.

Behavioural results Experiment 2.

A: Group-level mean of each participant’s median learning rate for each trial in each environment (See Behavioural data analyses for details). Error bars represent standard errors of the means. B: Detailed overview of all participants’ median initial learning rates. One participant’s median initial learning rate of -0.674 in the high noise environment is not visible on the plot. C-E: Evolution of (group-level mean) learning rates over trials (within blocks) for each quarter of the task separately.

Bai model estimation results Experiment 2.

The density plots on the left side of each subfigure show the full posterior densities over the means of the group-level distributions of the relevant parameters. The scatter plots on the right side of each subfigure show the means of all individual-level posterior distributions of the relevant parameters.

RSA analsyis of the fMRI data.

A: spatial location RDM. B: Learning rate RDM. C: Brain map of significant t-values resulting from the whole-brain searchlight RSA of fMRI data acquired while participants had just been transported to the next location around the island (correlation with learning rate RDM). The map is centred around coordinate X = -21.5 Y = 37.5 Z = -18.5 where the largest cluster of significant voxels in the OFC reached its peak t-value. D-F: Interaction effect between time (first vs. second half of task) and RDM (spatial location vs. learning rate RDM) in the occipital cortex, defined as the cluster of significant voxels found in the aforementioned whole-brain searchlight RSA (D); the central OFC as defined by (Kahnt et al., 2012), based on connections to other brain regions (E); and the ventral striatum, defined as the left and right nucleus accumbens according to the AAL atlas (F). Grey dots represent individual-level Kendall’s tau-values, while black dots and error bars represent group-level means and SEs of the means, respectively.

Results of the analyses of the effect of prediction error on the fMRI data.

A-B: Brain map of significant t-values resulting from the whole-brain (univariate) tests of voxel activity being (parametrically) modulated by prediction error on trial 1 (A) and trial 2 (B). Each map is centred around the coordinate where it reaches its peak (negative) t-value. C-D: Interaction effect between time (first vs. second half of task) and environment (low vs. medium vs. high measurement noise) on the modulating effect of prediction error on trial 1 (C) and trial 2 (D) on ventral striatum activity. This ROI was defined as the left and right nucleus accumbens accordingto the AAL atlas . Grey dots represent individual-level GLM beta-values, while black dots and error bars represent group-level means and SEs of the means, respectively.