The ability to form and revise uncertain beliefs through information sampling is a hallmark of human cognition. This inference process has been extensively studied using two main classes of decision tasks (Bartolo and Averbeck, 2021; Wyart and Koechlin, 2016). In passive sampling tasks, participants are observers who sample information over which they have no control (Murphy et al., 2016; van den Berg et al., 2016; Zylberberg et al., 2018). This is the case in most perceptual decision tasks, in which the experimenter controls the sensory information provided to participants (for reviews, see Gold and Shadlen, 2007; Hanks and Summerfield, 2017). Outside the laboratory, for example, one might see an ad for a new movie at a bus stop. By contrast, active sampling tasks let participants choose which source of information to sample from Charpentier et al., 2018; Gureckis and Markant, 2012; Markant and Gureckis, 2014. In these situations, the information provided to participants corresponds to the outcome of their own choices. Using the same example, one can alternatively browse the web for information about new movies. Critically, unlike passive sampling, active sampling provides participants with control over which source of information to sample from, even if the information itself (about the new movie) can be exactly the same in both cases.

In both passive and active sampling tasks, recent work has assigned a key role for confidence in the formation and revision of uncertain beliefs (Meyniel et al., 2015; Nassar et al., 2010; Rouault et al., 2019; Sarafyazd and Jazayeri, 2019). Low confidence in current beliefs has been shown to predict changes-of-mind (Balsdon et al., 2020; Folke et al., 2016) and allows for the flexible adaptation of behavioral strategies even when external feedback is unavailable (Desender et al., 2018; Desender et al., 2019; Fleming et al., 2018; Rollwage et al., 2020). But besides this pervasive role of confidence in belief updating, the control conferred by active sampling allows one to seek information about alternative strategies, a cognitive process that is by definition not possible in the absence of control over information sampling. Information seeking has been mostly studied using ‘exploration-exploitation’ dilemmas (Costa et al., 2019; Daw et al., 2006), where it is confounded with changes-of-mind about the reliability of the current behavioral strategy. In these paradigms, participants evolve in controllable environments and usually sample one among several options to maximize reward. Therefore, they have to either exploit a currently rewarding option or sacrifice rewards to explore alternative options and seek information about their possible rewards (Rich and Gureckis, 2018; Wilson et al., 2014). This trade-off means that in these paradigms exploration differs from exploitation not only in terms of information seeking, but also in terms of other co-occurring cognitive events, including overt response switches and covert changes-of-mind.

These different families of paradigms developed for studying information seeking vary on several dimensions, particularly the sources of uncertainty (Fleming et al., 2018), stimuli used (Gesiarz et al., 2019), desirability of the information to be sought (Hertwig and Engel, 2021), and degree of control over information sampled (Desender et al., 2018). These differences have made direct comparisons between paradigms extremely challenging. To date, no study has directly manipulated control over evidence sampling in otherwise aligned experimental conditions. At the neurophysiological level, exploration is known to be associated with larger pupil-linked arousal (Jepma and Nieuwenhuis, 2011) and increased activity in lateral prefrontal regions in electroencephalographic (EEG) and blood oxygen-level dependent (BOLD) activity (Donoso et al., 2014; Tzovara et al., 2012). However, due to confounds between information seeking and other cognitive events during exploration in these studies, it remains unclear whether information seeking is associated with specific neurophysiological signatures.

Here, to isolate the specific signatures of information seeking, we contrasted changes-of-mind occurring with and without control over information sampling. We employed an adaptive decision-making task that allows comparing the formation and revision of uncertain beliefs between controllable and uncontrollable conditions (Figure 1). We previously used these experimental conditions to compare how participants integrate evidence when it is a cue (uncontrollable condition) vs. when it is an outcome (controllable condition) (Weiss et al., 2021). Here, we focus on the comparison of repeat and switch decisions between these conditions so as to isolate the behavioral signatures and neural basis of information seeking, while replicating most of the previously observed effects in Weiss et al. on behavioral choices and their computational modeling (Weiss et al., 2021). We found that participants need more evidence against their current beliefs to change their behavior in the controllable condition, and that they do so with decreased confidence. Nevertheless, participants are more likely to probe again their new behavioral strategy in the next decision, even in the absence of objective evidence supporting this new strategy – a form of active ‘hypothesis testing’ (Markant and Gureckis, 2014). Using computational modeling (Glaze et al., 2015; Weiss et al., 2021), we show that controllability increases the stability of participants’ beliefs, a mechanism that explains the observed behavioral correlates of information seeking. At the physiological level, changes-of-mind occurring in the controllable condition are preceded by stronger suppression of neuromagnetic alpha-band activity in the dorsal attention network, and followed by increased pupil-linked arousal during the presentation of decision outcomes. Taken together, these features suggest that information seeking increases the saliency of information that is perceived as the direct consequence of one’s own actions.

Figure 1

Download asset Open asset

Accurate decisions in uncertain environments require forming and updating beliefs through efficient information sampling. Previous studies of human decisions and confidence have often relied on tasks in which participants have no control over the sampling of information (Fleming et al., 2018; Murphy et al., 2016; van den Berg et al., 2016). However, outside the laboratory, we usually have some control over information sampling: which way to look, which food to try, which person to listen to. We do not passively receive information but rather actively seek information for decision-making (Gureckis and Markant, 2012; Sharot and Sunstein, 2020). Here, we reasoned that control is necessary for information seeking, which we used to dissociate information seeking from changes in beliefs and behavior. Across several variants of the same task tested with and without control over information sampling, we obtained converging evidence that information seeking is associated with decreased confidence and drives active hypothesis testing. At the physiological level, information seeking is associated with stronger and longer-lasting correlates of attention and arousal.

In previous work, information seeking has often been studied and theorized in the context of ‘exploration-exploitation’ dilemmas (Costa et al., 2019; Daw et al., 2006). In these tasks, participants are typically immersed in controllable environments and asked to sample one among several options to maximize reward. Therefore, they have to choose between exploiting a currently rewarding option and foregoing rewards to explore alternative options to seek information about their associated rewards (Rich and Gureckis, 2018; Wilson et al., 2014). This cognitive trade-off raises the issue that exploration and exploitation not only differ in terms of information seeking, but also in terms of other cognitive and behavioral events. In particular, three main events co-occur during an exploratory decision: (1) a covert change-of-mind about the expected reward of the current option, (2) an overt change in behavior (e.g., a response switch), and (3) information seeking, that is, an active search for information about the new option being considered. The latter is only possible in the controllable condition, allowing us to isolate information seeking by comparing response switches between the two conditions.

We fitted a Bayesian inference model to participants’ choices and extended our model to predict confidence reports. Simulations with best-fitting parameters of individual participants confirmed that each parameter captures a specific behavioral gradient matching participants’ psychometric signatures (Figure 4—figure supplements 3 and 4). First, we replicate our previous finding of a lower perceived hazard rate in the controllable condition (Figure 5B), corresponding to participants perceiving contingencies as more stable in controllable environments (Weiss et al., 2021). This lower perceived hazard rate contributes to explaining a larger loss of confidence on changes-of-mind in the controllable condition. In the model, the lower perceived hazard rate creates stronger prior beliefs, such that more inconsistent evidence is required for them to be reversed. This mechanism predicts lower confidence during changes-of-mind despite stronger evidence in favor of a change-of-mind in the current trial. This increase in the magnitude of prior beliefs also explains why participants confirm more often their changes-of-mind in the controllable condition, despite lower confidence in these changes-of-mind.

In both conditions, our model predicts that the quantity of inconsistent evidence required for participants to change their minds (reflected in choice-PSE estimates) should be smaller following a switch than a repeat decision (Figure 5A). This prediction was verified in the uncontrollable condition. By contrast, in the controllable condition, participants violated this prediction by showing similar choice-PSEs following repeat and switch decisions (Figure 5A). This deviation between model and data in this condition where changes-of-mind are associated with information seeking could reflect at least two cognitive effects. First, participants may believe a new reversal in task contingencies to be less likely just after a perceived reversal. Second, participants may engage in an active form of hypothesis testing; that is, testing that the new action draws from the target category. Such hypothesis testing is by definition only possible in the controllable condition, where participants can actively sample the environment to confirm or discard their new hypothesis (Collins and Frank, 2013; Markant and Gureckis, 2014). Hypothesis testing is known to be particularly valuable in environments with many choice options (Gureckis and Markant, 2012), but the fact that we observe it in a condition with only two options suggests that it is a constitutive feature of information seeking.

The notion of controllability bears a partial resemblance with the distinction between learning under selection vs. reception (Gureckis and Markant, 2012), and between free vs. forced choices (Chambon et al., 2020; Ligneul, 2021; Sidarus et al., 2019; Wilson et al., 2014). Performance benefits have been identified in selection contexts and free choice contexts, where participants can maximize the informativeness of their choices (Freeman et al., 2014; Voss et al., 2010; Xu and Tenenbaum, 2007). However, in these previous studies, sequences from selection and reception (and, likewise, from free and forced) choices vastly differ, which make it difficult to separate the specific effects of controllability from its consequences on the evidence available for subsequent choices. Instead, the current paradigm carefully matched the amount of evidence provided in both controllability conditions. Furthermore, we sought to validate controllability as the true cause of differences between conditions. In Experiment 3, we examined whether a distinction in temporal focus (prospective instead of retrospective inference in an uncontrollable context) would account for the differences between the original conditions (Figure 2—figure supplement 1). Although the pattern of choices was markedly different, for confidence it remains possible that a lingering effect of temporality affected the original conditions, even if it cannot account for the results overall. In Experiment 2B, when the instructions changed unpredictably across trials instead of blocks, participants still perceived the controllable condition as more stable, providing evidence that the specificity of this condition arises from the controllability of information sampling experienced (Figure 2—figure supplement 2).

Despite stimuli carrying no explicitly affective or rewarding value, it remains possible that the mere presence of a target in the C+ condition makes the target category desirable and may produce a form of confirmation bias (Talluri et al., 2018). However, at the behavioral level, a confirmation bias would predict that in the controllable condition the sensitivity to evidence should be degraded when the sequence of evidence is inconsistent with participants’ previous choice, a prediction that is absent from human data (Figure 2). At the neural level, a confirmation bias would decrease the weighting of belief-inconsistent evidence. Instead, in our previous study, we found an absence of difference between the neural coding of belief-consistent and belief-inconsistent evidence (Weiss et al., 2021). In addition, the findings of Experiment 2B, where the target category changes from trial to trial, also mean that the differences between conditions are unlikely to reflect a bias in direction of the target category (Gesiarz et al., 2019). Indeed, the direction of this bias would change from one trial to the next, and should therefore decrease in Experiment 2B – which we did not observe. Finally, based on theoretical considerations and empirical results, we also found evidence that our controllability manipulation did not create differences in working memory or executive demands between experimental conditions. We designed our conditions so that they were strictly matched in terms of number of tasks to do, sources of uncertainty, and motor actions to perform. At the behavioral level, the lack of a difference in choice accuracy, sensitivity to the objective evidence, and inference noise parameter between conditions makes it unlikely that the controllable condition was more demanding. At the physiological level, an increased load should have triggered changes in attention and arousal signals across all trials, unlike our observations that the neurophysiological measures only differed on switch decisions, whereas no difference was observed on repeat decisions. Together, these considerations make it unlikely that observed differences between conditions are due to an increased executive or working memory load in the controllable condition. At the neurophysiological level, we observed a stronger suppression of neuromagnetic alpha-band activity in the dorsal attention networks during the last seconds preceding and following response switches, in both conditions (Figure 6). This observation suggests that uncertainty at the time of a change-of-mind is associated with participants being more strongly oriented toward the presented evidence before the response switch occurs. Critically, this finding does not merely reflect greater attention or engagement in the controllable condition, which would have suggested general differences in task processing between conditions across repeat and switch decisions. Previous work using fMRI showed that task switching recruits a similar dorsal frontoparietal network involved in attentional control (Corbetta and Shulman, 2002), whereas response switching recruits a frontal network involved in executive control (Daw et al., 2006; Donoso et al., 2014; Findling et al., 2019). However, the use of MEG here allowed us to reveal largely anticipatory activations preceding response switches. Our results are also generally consistent with medial frontal activations during hypothesis testing (for a review, see Monosov and Rushworth, 2022) and in situations where the gathered information affords to predict but does not influence future outcomes (White et al., 2019). The temporal dissociation between switch effects observed in the occipital and frontal clusters suggests that changes-of-mind are preceded by a state of increased attention toward external (here, visual) evidence. This first stage is followed by a covert change-of-mind and ends with an overt response switch reflected in strong alpha-band suppression overlying frontal cortex (Figure 6, Figure 6—figure supplement 1).

In contrast to MEG dynamics, phasic pupil dilation revealed noradrenergic responses starting after the response probe onset and remained during presentation of the outcomes (Figure 7). This pattern of findings suggests a functional dissociation between alpha-band correlates of attention and pupil-linked arousal during changes-of-mind. The pupil-linked arousal associated with response switches is consistent with previous evidence for an association between pupil response and belief updating in uncertain environments (Filipowicz et al., 2020). Our results are also consistent with a previous study providing evidence that pupil-linked arousal following decision uncertainty increased participants’ tendency to switch their choice on the next trial (Braun et al., 2017). The fact that the pupil effect extends well into the next trial suggests that participants pay attention to the consequences of their changes-of-mind well into the next trial, particularly so in the controllable condition where they are in control of the upcoming sequence of evidence. This again stands in contrast to alpha suppression that was mostly anticipatory, a distinction between pupil dilation and alpha-band suppression previously observed in relation to attention (Whitmarsh et al., 2021). Even when accounting for the slower dynamics of pupillary responses (Hoeks and Levelt, 1993), the 4 s difference between the onsets of the effects observed in pupil dilation and alpha-band suppression makes it highly unlikely that the two effects arise from the same source. An additional difference between switch and repeat trials was further observed in the cardiac IBI, with similar dynamics as the alpha-band suppression. This increased slowing of heartbeats preceding response switches could partly reflect, but are unlikely to entirely reduce to, changes in respiration (Park et al., 2014), for example, participants holding their breath before response switches in the controllable condition. Together, these findings indicate that information seeking is characterized by a specific temporal succession of neurophysiological mechanisms in cortical, subcortical, and peripheral nervous systems.

By comparing changes-of-mind occurring in controllable and uncontrollable environments, we identified cognitive and neurophysiological signatures of information seeking that had otherwise proved difficult to isolate. We found that in controllable environments humans require more evidence to change their mind and do so with reduced confidence but are nevertheless more likely to stick to their decision on the next trial, suggesting a form of hypothesis testing. With computational modeling indicating that participants perceive controllable environments as more stable, we were not only able to explain why information seeking is associated with a higher degree of perceived uncertainty, but also to identify stronger and longer-lasting effects of changes-of-mind on cognition and behavior. Alterations in confidence (Rouault et al., 2018), information seeking (Hauser et al., 2017), and perceived controllability (Voss et al., 2017) are associated with various psychiatric symptoms. For instance, participants with obsessive-compulsive disorder typically need to gather more evidence before committing to a decision, while participants with schizophrenia present an inflated sense of agency associated with an inability to accurately update these representations (Metcalfe et al., 2014). By clarifying the effects of controllability on inference and confidence, our study lays the groundwork for explaining how the interplay between perceived control, information seeking, and confidence may go awry in psychiatric conditions.

Data and code availability. Participants' data and MATLAB code for behavioral and psychometric analyses of Experiments 1 and 2A, and code for fitting the computational model are available at https://www.github.com/marionrouault/actobscom/, (copy archived at swh:1:rev:43bfb417394bd243aa4ccf2ee2b2f50850d4cea8). Because tested participants did not provide written consent regarding the posting of their anonymized data on public repositories, the MEG, pupillometric and ECG datasets are available from the corresponding authors upon request.

1. 1. Wyart V
   2. Weiss A
   3. Chambon V
   4. Lee JK
   5. Drugowitsch J

   (2020) figshare

   Weiss_2020_NatComm_data_behavior.

   https://doi.org/10.6084/m9.figshare.13200128.v1

1. Conference

   1. Acerbi L
   2. Ma WJ

   (2017)

   Practical Bayesian optimization for model fitting with Bayesian adaptive direct search

   Advances in Neural Information Processing Systems. pp. 1836–1846.

   • Google Scholar
2. 1. Balsdon T
   2. Wyart V
   3. Mamassian P

   (2020) Confidence controls perceptual evidence accumulation

   Nature Communications 11:1–11.

   https://doi.org/10.1038/s41467-020-15561-w

   • PubMed
   • Google Scholar
3. 1. Bartolo R
   2. Averbeck BB

   (2021) Inference as a fundamental process in behavior

   Current Opinion in Behavioral Sciences 38:8–13.

   https://doi.org/10.1016/j.cobeha.2020.06.005

   • PubMed
   • Google Scholar
4. 1. Braun A
   2. Urai AE
   3. Donner TH

   (2017) Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias

   Nature Communications 8:1–11.

   https://doi.org/10.1038/ncomms14637

   • PubMed
   • Google Scholar
5. 1. Chambon V
   2. Théro H
   3. Vidal M
   4. Vandendriessche H
   5. Haggard P
   6. Palminteri S

   (2020) Information about action outcomes differentially affects learning from self-determined versus imposed choices

   Nature Human Behaviour 4:1067–1079.

   https://doi.org/10.1038/s41562-020-0919-5

   • PubMed
   • Google Scholar
6. 1. Charpentier CJ
   2. Bromberg-Martin ES
   3. Sharot T

   (2018) Valuation of knowledge and ignorance in mesolimbic reward circuitry

   PNAS 115:E7255–E7264.

   https://doi.org/10.1073/pnas.1800547115

   • PubMed
   • Google Scholar
7. 1. Collins AGE
   2. Frank MJ

   (2013) Cognitive control over learning: creating, clustering, and generalizing task-set structure

   Psychological Review 120:190–229.

   https://doi.org/10.1037/a0030852

   • PubMed
   • Google Scholar
8. 1. Corbetta M
   2. Shulman GL

   (2002) Control of goal-directed and stimulus-driven attention in the brain

   Nature Reviews. Neuroscience 3:201–215.

   https://doi.org/10.1038/nrn755

   • PubMed
   • Google Scholar
9. 1. Costa VD
   2. Mitz AR
   3. Averbeck BB

   (2019) Subcortical substrates of explore-exploit decisions in primates

   Neuron 103:533–545.

   https://doi.org/10.1016/j.neuron.2019.05.017

   • PubMed
   • Google Scholar
10. 1. Daw ND
    2. O’Doherty JP
    3. Dayan P
    4. Seymour B
    5. Dolan RJ

    (2006) Cortical substrates for exploratory decisions in humans

    Nature 441:876–879.

    https://doi.org/10.1038/nature04766

    • PubMed
    • Google Scholar
11. 1. Desender K
    2. Boldt A
    3. Yeung N

    (2018) Subjective confidence predicts information seeking in decision making

    Psychological Science 29:761–778.

    https://doi.org/10.1177/0956797617744771

    • PubMed
    • Google Scholar
12. 1. Desender K
    2. Murphy P
    3. Boldt A
    4. Verguts T
    5. Yeung N

    (2019) A postdecisional neural marker of confidence predicts information-seeking in decision-making

    The Journal of Neuroscience 39:3309–3319.

    https://doi.org/10.1523/JNEUROSCI.2620-18.2019

    • PubMed
    • Google Scholar
13. 1. Donoso M
    2. Collins AGE
    3. Koechlin E

    (2014) Human cognition: Foundations of human reasoning in the prefrontal cortex

    Science 344:1481–1486.

    https://doi.org/10.1126/science.1252254

    • PubMed
    • Google Scholar
14. 1. Drugowitsch J
    2. Wyart V
    3. Devauchelle AD
    4. Koechlin E

    (2016) Computational precision of mental inference as critical source of human choice suboptimality

    Neuron 92:1398–1411.

    https://doi.org/10.1016/j.neuron.2016.11.005

    • PubMed
    • Google Scholar
15. 1. Filipowicz AL
    2. Glaze CM
    3. Kable JW
    4. Gold JI

    (2020) Pupil diameter encodes the idiosyncratic, cognitive complexity of belief updating

    eLife 9:e57872.

    https://doi.org/10.7554/eLife.57872

    • PubMed
    • Google Scholar
16. 1. Findling C
    2. Skvortsova V
    3. Dromnelle R
    4. Palminteri S
    5. Wyart V

    (2019) Computational noise in reward-guided learning drives behavioral variability in volatile environments

    Nature Neuroscience 22:2066–2077.

    https://doi.org/10.1038/s41593-019-0518-9

    • PubMed
    • Google Scholar
17. 1. Fleming SM
    2. van der Putten EJ
    3. Daw ND

    (2018) Neural mediators of changes of mind about perceptual decisions

    Nature Neuroscience 21:617–624.

    https://doi.org/10.1038/s41593-018-0104-6

    • Google Scholar
18. 1. Folke T
    2. Jacobsen C
    3. Fleming SM
    4. De Martino B

    (2016) Explicit representation of confidence informs future value-based decisions

    Nature Human Behaviour 1:105–108.

    https://doi.org/10.1038/s41562-016-0002

    • Google Scholar
19. 1. Freeman S
    2. Eddy SL
    3. McDonough M
    4. Smith MK
    5. Okoroafor N
    6. Jordt H
    7. Wenderoth MP

    (2014) Active learning increases student performance in science, engineering, and mathematics

    PNAS 111:8410–8415.

    https://doi.org/10.1073/pnas.1319030111

    • Google Scholar
20. 1. Gesiarz F
    2. Cahill D
    3. Sharot T

    (2019) Evidence accumulation is biased by motivation: A computational account

    PLOS Computational Biology 15:e1007089.

    https://doi.org/10.1371/journal.pcbi.1007089

    • PubMed
    • Google Scholar
21. 1. Glaze CM
    2. Kable JW
    3. Gold JI

    (2015) Normative evidence accumulation in unpredictable environments

    eLife 4:e08825.

    https://doi.org/10.7554/eLife.08825

    • PubMed
    • Google Scholar
22. 1. Gold JI
    2. Shadlen MN

    (2007) The neural basis of decision making

    Annual Review of Neuroscience 30:535–574.

    https://doi.org/10.1146/annurev.neuro.29.051605.113038

    • PubMed
    • Google Scholar
23. 1. Gureckis TM
    2. Markant DB

    (2012) Self-directed learning: A cognitive and computational perspective

    Perspectives on Psychological Science 7:464–481.

    https://doi.org/10.1177/1745691612454304

    • PubMed
    • Google Scholar
24. 1. Hanks TD
    2. Summerfield C

    (2017) Perceptual decision making in rodents, monkeys, and humans

    Neuron 93:15–31.

    https://doi.org/10.1016/j.neuron.2016.12.003

    • PubMed
    • Google Scholar
25. 1. Hauser TU
    2. Moutoussis M
    3. Iannaccone R
    4. Brem S
    5. Walitza S
    6. Drechsler R
    7. Dayan P
    8. Dolan RJ

    (2017) Increased decision thresholds enhance information gathering performance in juvenile Obsessive-Compulsive Disorder (OCD)

    PLOS Computational Biology 13:e1005440.

    https://doi.org/10.1371/journal.pcbi.1005440

    • PubMed
    • Google Scholar
26. Book

    1. Hertwig R
    2. Engel C

    (2021) Age Differences in Deliberate Ignorance

    Psychology and Aging.

    https://doi.org/10.7551/mitpress/13757.001.0001

    • Google Scholar
27. 1. Hoeks B
    2. Levelt WJM

    (1993) Pupillary dilation as a measure of attention: A quantitative system analysis

    Behavior Research Methods, Instruments, & Computers 25:16–26.

    https://doi.org/10.3758/BF03204445

    • Google Scholar
28. 1. Jepma M
    2. Nieuwenhuis S

    (2011) Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory

    Journal of Cognitive Neuroscience 23:1587–1596.

    https://doi.org/10.1162/jocn.2010.21548

    • PubMed
    • Google Scholar
29. 1. Kiesel A
    2. Miller J
    3. Jolicoeur P
    4. Brisson B

    (2008) Measurement of ERP latency differences: A comparison of single-participant and jackknife-based scoring methods

    Psychophysiology 45:250–274.

    https://doi.org/10.1111/j.1469-8986.2007.00618.x

    • PubMed
    • Google Scholar
30. 1. Ligneul R

    (2021) Prediction or causation? Towards a redefinition of task controllability

    Trends in Cognitive Sciences 25:431–433.

    https://doi.org/10.1016/j.tics.2021.02.009

    • PubMed
    • Google Scholar
31. 1. Maniscalco B
    2. Lau H

    (2012) A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings

    Consciousness and Cognition 21:422–430.

    https://doi.org/10.1016/j.concog.2011.09.021

    • PubMed
    • Google Scholar
32. 1. Maris E
    2. Oostenveld R

    (2007) Nonparametric statistical testing of EEG- and MEG-data

    Journal of Neuroscience Methods 164:177–190.

    https://doi.org/10.1016/j.jneumeth.2007.03.024

    • PubMed
    • Google Scholar
33. 1. Markant DB
    2. Gureckis TM

    (2014) Is it better to select or to receive? Learning via active and passive hypothesis testing

    Journal of Experimental Psychology. General 143:94–122.

    https://doi.org/10.1037/a0032108

    • PubMed
    • Google Scholar
34. Book

    1. Metcalfe J
    2. Van Snellenberg JX
    3. DeRosse P
    4. Balsam P
    5. Malhotra AK

    (2014) Judgments of agency in schizophrenia: an impairment in autonoetic metacognition

    In: Metcalfe JF, editors. In: The Cognitive Neuroscience of Metacognition. Springer. pp. 367–387.

    https://doi.org/10.1007/978-3-642-45190-4

    • Google Scholar
35. 1. Meyniel F
    2. Schlunegger D
    3. Dehaene S

    (2015) The sense of confidence during probabilistic learning: A normative account

    PLOS Computational Biology 11:e1004305.

    https://doi.org/10.1371/journal.pcbi.1004305

    • PubMed
    • Google Scholar
36. 1. Monosov IE
    2. Rushworth MFS

    (2022) Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change

    Neuropsychopharmacology 47:196–210.

    https://doi.org/10.1038/s41386-021-01079-2

    • PubMed
    • Google Scholar
37. 1. Murphy PR
    2. Robertson IH
    3. Harty S
    4. OConnell RG

    (2016) Neural evidence accumulation persists after choice to inform metacognitive judgments

    eLife 1:e23.

    https://doi.org/10.7554/eLife.11946.001

    • Google Scholar
38. 1. Nassar MR
    2. Wilson RC
    3. Heasly B
    4. Gold JI

    (2010) An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment

    The Journal of Neuroscience 30:12366–12378.

    https://doi.org/10.1523/JNEUROSCI.0822-10.2010

    • PubMed
    • Google Scholar
39. 1. Oostenveld R
    2. Fries P
    3. Maris E
    4. Schoffelen JM

    (2011) FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data

    Computational Intelligence and Neuroscience 2011:156869.

    https://doi.org/10.1155/2011/156869

    • PubMed
    • Google Scholar
40. 1. Palminteri S
    2. Wyart V
    3. Koechlin E

    (2017) The importance of falsification in computational cognitive modeling

    Trends in Cognitive Sciences 21:425–433.

    https://doi.org/10.1016/j.tics.2017.03.011

    • PubMed
    • Google Scholar
41. 1. Park HD
    2. Correia S
    3. Ducorps A
    4. Tallon-Baudry C

    (2014) Spontaneous fluctuations in neural responses to heartbeats predict visual detection

    Nature Neuroscience 17:612–618.

    https://doi.org/10.1038/nn.3671

    • PubMed
    • Google Scholar
42. 1. Pouget A
    2. Drugowitsch J
    3. Kepecs A

    (2016) Confidence and certainty: distinct probabilistic quantities for different goals

    Nature Neuroscience 19:366–374.

    https://doi.org/10.1038/nn.4240

    • PubMed
    • Google Scholar
43. 1. Rich AS
    2. Gureckis TM

    (2018) Exploratory choice reflects the future value of information

    Decision 5:177–192.

    https://doi.org/10.1037/dec0000074

    • Google Scholar
44. 1. Rollwage M
    2. Loosen A
    3. Hauser TU
    4. Moran R
    5. Dolan RJ
    6. Fleming SM

    (2020) Confidence drives a neural confirmation bias

    Nature Communications 11:1–11.

    https://doi.org/10.1038/s41467-020-16278-6

    • PubMed
    • Google Scholar
45. 1. Rouault M
    2. Seow T
    3. Gillan CM
    4. Fleming SM

    (2018) Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance

    Biological Psychiatry 84:443–451.

    https://doi.org/10.1016/j.biopsych.2017.12.017

    • PubMed
    • Google Scholar
46. 1. Rouault M
    2. Dayan P
    3. Fleming SM

    (2019) Forming global estimates of self-performance from local confidence

    Nature Communications 10:1141.

    https://doi.org/10.1038/s41467-019-09075-3

    • PubMed
    • Google Scholar
47. 1. Sarafyazd M
    2. Jazayeri M

    (2019) Hierarchical reasoning by neural circuits in the frontal cortex

    Science 364:eaav8911.

    https://doi.org/10.1126/science.aav8911

    • PubMed
    • Google Scholar
48. 1. Sharot T
    2. Sunstein CR

    (2020) How people decide what they want to know

    Nature Human Behaviour 4:14–19.

    https://doi.org/10.1038/s41562-019-0793-1

    • PubMed
    • Google Scholar
49. 1. Sidarus N
    2. Palminteri S
    3. Chambon V

    (2019) Cost-benefit trade-offs in decision-making and learning

    PLOS Computational Biology 15:e1007326.

    https://doi.org/10.1371/journal.pcbi.1007326

    • PubMed
    • Google Scholar
50. 1. Talluri BC
    2. Urai AE
    3. Tsetsos K
    4. Usher M
    5. Donner TH

    (2018) Confirmation bias through selective overweighting of choice-consistent evidence

    Current Biology 28:3128–3135.

    https://doi.org/10.1016/j.cub.2018.07.052

    • PubMed
    • Google Scholar
51. 1. Tzovara A
    2. Murray MM
    3. Bourdaud N
    4. Chavarriaga R
    5. Millán JDR
    6. De Lucia M

    (2012) The timing of exploratory decision-making revealed by single-trial topographic EEGanalyses

    NeuroImage 60:1959–1969.

    https://doi.org/10.1016/j.neuroimage.2012.01.136

    • PubMed
    • Google Scholar
52. 1. van den Berg R
    2. Anandalingam K
    3. Zylberberg A
    4. Kiani R
    5. Shadlen MN
    6. Wolpert DM

    (2016) A common mechanism underlies changes of mind about decisions and confidence

    eLife 5:e12192.

    https://doi.org/10.7554/eLife.12192

    • PubMed
    • Google Scholar
53. 1. Voss JL
    2. Gonsalves BD
    3. Federmeier KD
    4. Tranel D
    5. Cohen NJ

    (2010) Hippocampal brain-network coordination during volitional exploratory behavior enhances learning

    Nature Neuroscience 14:115–120.

    https://doi.org/10.1038/nn.2693

    • PubMed
    • Google Scholar
54. 1. Voss M
    2. Chambon V
    3. Wenke D
    4. Kühn S
    5. Haggard P

    (2017) In and out of control: brain mechanisms linking fluency of action selection to self-agency in patients with schizophrenia

    Brain 140:2226–2239.

    https://doi.org/10.1093/brain/awx136

    • PubMed
    • Google Scholar
55. 1. Weiss A
    2. Chambon V
    3. Lee JK
    4. Drugowitsch J
    5. Wyart V

    (2021) Interacting with volatile environments stabilizes hidden-state inference and its brain signatures

    Nature Communications 12:2228.

    https://doi.org/10.1038/s41467-021-22396-6

    • PubMed
    • Google Scholar
56. 1. White JK
    2. Bromberg-Martin ES
    3. Heilbronner SR
    4. Zhang K
    5. Pai J
    6. Haber SN
    7. Monosov IE

    (2019) A neural network for information seeking

    Nature Communications 10:1–19.

    https://doi.org/10.1038/s41467-019-13135-z

    • Google Scholar
57. 1. Whitmarsh S
    2. Gitton C
    3. Jousmäki V
    4. Sackur J
    5. Tallon-Baudry C

    (2021) Neuronal correlates of the subjective experience of attention

    The European Journal of Neuroscience 55:3465–3482.

    https://doi.org/10.1111/ejn.15395

    • PubMed
    • Google Scholar
58. 1. Wilson RC
    2. Geana A
    3. White JM
    4. Ludvig EA
    5. Cohen JD

    (2014) Humans use directed and random exploration to solve the explore-exploit dilemma

    Journal of Experimental Psychology. General 143:2074–2081.

    https://doi.org/10.1037/a0038199

    • PubMed
    • Google Scholar
59. 1. Wilson RC
    2. Collins AG

    (2019) Ten simple rules for the computational modeling of behavioral data

    eLife 8:e49547.

    https://doi.org/10.7554/eLife.49547

    • PubMed
    • Google Scholar
60. 1. Wyart V
    2. Koechlin E

    (2016) Choice variability and suboptimality in uncertain environments

    Current Opinion in Behavioral Sciences 11:109–115.

    https://doi.org/10.1016/j.cobeha.2016.07.003

    • Google Scholar
61. 1. Xu F
    2. Tenenbaum JB

    (2007) Sensitivity to sampling in bayesian word learning

    Developmental Science 10:288–297.

    https://doi.org/10.1111/j.1467-7687.2007.00590.x

    • PubMed
    • Google Scholar
62. 1. Zylberberg A
    2. Wolpert DM
    3. Shadlen MN

    (2018) Counterfactual reasoning underlies the learning of priors in decision making

    Neuron 99:1083–1097.

    https://doi.org/10.1016/j.neuron.2018.07.035

    • PubMed
    • Google Scholar

Decision letter after peer review:

Thank you for submitting your article "Controllability reveals defining features of information seeking" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Michael Frank as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Clare Press (Reviewer #2).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission. The reviewers felt that the conclusions of of paper are mostly well supported by data, but some aspects of the effects reported, the model and psychological interpretation need to be clarified and extended.

Essential revisions:

The reviewers enjoyed this paper, which they found to be well written and with excellent figures, but felt that the number of different effects, psychological constructs, and data types presented at times impeded their understanding of a clear take-home message that captures how this paper furthers our understanding of decision-making. Any changes that could help focus the manuscript better in this regard would be welcome. Additionally, there are several re-analyses and changes to the interpretations that are required as detailed below, as well as some further details relating to the analyses. In principle the reviewers think that it should be possible to address these concerns without further data, but if it is more straightforward to conduct further experiments to bolster their conclusions then the authors should feel free to do that.

Key re-analyses required

1) The Ob trials require selection of an action, monitoring the action-outcome relationship, and a judgement about the stimulus. The Cb trials, in contrast, simply require monitoring the stimulus (it was unclear whether participants perform an action to start each trial, but regardless, they will not need to keep track of action-outcome mappings). Therefore, the fact that participants are slower to notice the shift in Ob trials and that their judgements are associated with lower confidence could easily be driven by the fact that they have three tasks (or at least two) rather than one. Relatedly, can the fact that α suppression and pupil dilation effects are increased with changes-of-mind in the Ob condition also be explained by the fact participants have more tasks? If there are greater executive/working memory demands, this will reduce α and increase dilation (as already discussed in the manuscript). Participants may detect stimulus changes less readily because there is a greater executive/working memory load in this condition, and this has nothing to do with controllability/action or stability of beliefs per se.

The authors claim that the participants were "equally sensitive" to the available objective evidence across conditions, which would indeed be reassuring, but at the moment it is not clear how solid this inference is. It would be good for the authors to clarify what they mean here – did they test for a significant slope/precision effect in the data shown in Figure 3A? Is it possible to say participants are equally sensitive to the evidence (other than on switch trials), when the insight into their sensitivity is given by a response that only requires them to process the evidence on switch trials? More generally, it remains to be convincingly demonstrated that the Ob-Cb differences are generated by something other than number of tasks and generic working memory/executive differences. For example, it would be important to demonstrate that performance accuracy (i.e. how often participants make the correct choice), and choice reaction times are equivalent between conditions, which could be achieved using a Bayesian approach. (This analysis would need to exclude the switch trials). If there are differences in performance accuracy or reaction times across conditions, then this could undermine the authors' conclusions.

2) In experiment 3, which aimed to test whether the temporal direction of the inference (prospective vs retrospective) could in fact explain the observed differences between condition, the effects on change of mind are not shown. Instead, only the effects on confidence are displayed – why is that? Additionally, the confidence data presented in Figure S1 still shows some apparent differences: lower overall confidence, higher confidence time-constant, and higher confidence-PSE in the prospective vs retrospective condition. Those differences are reported but the authors then nonetheless conclude that these results constitute evidence that the observed differences between Ob and Cb conditions in the main experiment are only due to controllability and not to the temporal orientation. At present, this conclusion is not supported by the data. One could conclude that temporality has no effect only if no difference in choice or confidence were observed. Again, a Bayesian approach to demonstrate equivalence could be of use here.

Introduction

3) Please unify, or at least operationalize more clearly, the terms that are used to describe the main effect of interest. Is information seeking (which indeed is often studied in bandit-type tasks) the same as changes of mind (which are more often studied in within-trial evidence accumulation paradigms)? How does this relate to belief updating, hypothesis testing, exploration, and the observers' stickiness tendency? While it is laudable that the authors reach across subfields of neuroscience that have their own terminology, in places the text was confusing about what's a general psychological process vs. an effect in the behavioral data vs. a model-derived computational latent variable.

4) Related to the above point, some of the terminology or logical formulation of conclusions was a bit confusing. The authors use the term "information-seeking" specifically to refer to the controllable condition. In the uncontrollable condition, there is no "seeking" of information, only passive sampling. If that understanding is correct, some sentences are redundant. For example, shouldn't the title read "Controllability reveals defining features of information-sampling" or even (and more consistent with the findings) "Controllability over information sampling reveals defining features of changes of mind"? At times "controllability" and "information-seeking" seem to be used interchangeably, while at other times they seem to have different meanings. It would be important to specify those terms better.

5) It would greatly help the reader to include a short paragraph in the introduction listing the ways in which this work replicates vs. extends the previous paper by Weiss et al.

6) Overall the introduction is a little thin and would benefit from being expanded. First, the statement that "information-seeking has been mostly studied under 'exploration-exploitation' dilemma is not true – there are many recent studies that have studied information-seeking in humans using other paradigms. Second, and most importantly, the introduction currently lacks the rationale for the proposed work – in addition to dissociating information-seeking from changes-of-mind, which is more of a methodological aim, what are the key questions that the authors trying to address, and what are the hypotheses given the current literature? Finally, the use of MEG and other physiological measures (pupillometric, cardiac patterns) is not motivated at all in the introduction. The introduction should set the stage as to why collecting these data is needed given the question of interest. Similarly, the hypotheses should also set the stage for the specific analysis choices performed later, i.e. in what way does each presented analysis answer the question?

Methods

7) Was the model fitted to all trials together, or separately for each condition?

8) The model validation is rigorous, but would the same patterns also be observed out-of-sample? Since only one model is tested, there is always a risk of overfitting, which could be addressed with out-of-sample validation.

9) Please provides the statistics for the validation data presented on p12 (Figure S2 and S3)

10) How recoverable are the parameters if the model is re-fit to simulated data?

11) Are parameters correlated with each other? The authors should provide a correlation table of the parameters in the supplement.

Results

12) It would help to preview the specific diagnostic effects that isolate the process of interest, and then to repeat those in the model fit. Currently, we see four effects in Figure 2, four effects in Figure 3, and three effects in Figure 4. When we get to Figure 5D, slightly different effects are presented in comparison to the model. This was quite confusing and more clarity is required about the specific behavioral pattern of interest. This may just be a matter of slight reordering: the supplement has clear model-data matches, and placing those in the main figure (to highlight that most of the basic effects in Figures2-4 are indeed explained by the model) would allow a better appreciation of where the model fails to capture human behavior.

13) (see also (2) above) The data in Figure S1 are a crucial control, but it is not yet clear that there is no substantial difference in pro- vs. retro-spective decisions, which might (partially) explain the Cb vs. Ob differences. If controllability was the only important factor for behavioral effects, wouldn't we expect no difference between conditions in Figure S1? This would require a change in the authors' strength of interpretation as specifically pertaining to control.

14) Do the analyses in Figure 4 control for the difference in objective evidence (logL per trial)?

15) It is frequently stated that the Ob condition bestows instrumental control to participants over the sampling of stimuli. These statements should be modified, given that participants are told which category of stimulus they must generate. They do not have complete control over which stimuli they sample, unless they are willing to upset the experimenter!

16) It is not clear what action participants perform to start the trial in the Ob condition. This needs including. In outlining this it would also be great to include consideration of potential overlaps between the responses at the start and end of the trial in Ob, and whether this might have played into the effects.

17) Please add the change-of-mind data (i.e. equivalent of Figure 2A-B and 3A-B) for experiment 3, so it is possible to dissociate which effects are unique to the main experiment and which effects may be due to the temporal direction of the decisions. It is important to clarify this distinction as it can actually help narrow down the unique effect of controllability over temporality.

18) It would be useful to provide follow-up analyses to determine how much of the behavioral results can be explained by the difference in prior belief reported on p19, as well as discuss how much of these findings are then more consistent with a role of controllability per se, rather than a role for motivated beliefs/confirmation bias due to the presence of a target.

Discussion

19) There is some concern about the interpretation of the results – in particular, whether the observed differences between the cue-based and outcome-based conditions could be better explained by the presence of a target-induced confirmation bias in the outcome-based condition, which would induce a strong motivation to confirm that the chosen action does lead to the target. In other words, it is possible that having a target (e.g. "draw orange") may bias subjects towards believing they are drawing from that target even when they are not, and as a result needing less evidence to reach that conclusion (similar to the effect shown in Gesiarz, Cahill and Sharot, 2019, Plos Computational Biology). This could in turn lead to the observed patterns of results, i.e. needing more evidence against orange to switch, being less confident when switching etc. Additionally, the result that prior beliefs are stronger in the Ob condition (p18-19) is consistent with this idea, since confirmation bias is usually associated with stronger prior beliefs.

20) The authors claim that there is a causal role for confidence in controlling changes-of-mind (p11). While this is an interesting idea, it is not clear that it can be fully evidenced in this task without an experimentally-controlled manipulation of confidence. The reason is that there could be a common cause to both high confidence and high propensity to confirm switch decisions without the two processes actually being causally related. One such common cause could be the strength of evidence. Therefore, this conclusion needs to be tempered and this issue mentioned in the discussion.

21) There is no discussion of alternative ways in which these data might have turned out. The discussion entirely reports claims and findings with which the present data are consistent. Are there no data or theories that could have led to alternative predictions? Currently the manuscript portrays the impression of such high consensus that there was little point running the studies, suggesting that the empirical patterns to date all point in identical directions. One possibility is that the stimulus ISIs are not very variable. Given participants can perceive better at certain oscillatory phases of visual processing (peaks), could they choose to start trials in Ob to align with peaks and thereby improve perceptual acuity? If this occurred, the influence must be weaker than those pulling in the opposite direction, but it could have led to an alternative outcome. If it's a straightforward analysis to perform/report, it may be interesting to see how the oscillations align with events differentially in Ob and Cb.

Essential revisions:

The reviewers enjoyed this paper, which they found to be well written and with excellent figures, but felt that the number of different effects, psychological constructs, and data types presented at times impeded their understanding of a clear take-home message that captures how this paper furthers our understanding of decision-making. Any changes that could help focus the manuscript better in this regard would be welcome. Additionally, there are several re-analyses and changes to the interpretations that are required as detailed below, as well as some further details relating to the analyses. In principle the reviewers think that it should be possible to address these concerns without further data, but if it is more straightforward to conduct further experiments to bolster their conclusions then the authors should feel free to do that.

We thank both the editor and reviewers for their positive assessment of our paper, and for their helpful and thorough comments, which had a substantial impact on the manuscript. Following the editor and reviewers’ feedback:

• We have re-written our introduction to clarify: 1. how the paper contributes to the characterisation of information seeking in decision-making under uncertainty by positioning it with respect to the previous literature on this topic, and 2. the relationship between information seeking and the two related concepts of exploration and changes-of-mind;

• We now present more data from Experiment 3 to address the possibility that our main findings could be interpreted in terms of temporal orientation (retrospective vs. prospective inference) rather than controllability;

• We use several lines of evidence to show that the effects observed in the controllable condition that we attribute to information seeking cannot be explained by higher cognitive demands and/or working memory load;

• Please note that upon the reviewers’ suggestion, we have now renamed the conditions: uncontrollable condition (C-) (ex. cue-based condition) and controllable condition (C+) (ex. outcome-based condition).

Below we describe how we have revised our manuscript in response to each individual comment.

Key re-analyses required

1) The Ob trials require selection of an action, monitoring the action-outcome relationship, and a judgement about the stimulus. The Cb trials, in contrast, simply require monitoring the stimulus (it was unclear whether participants perform an action to start each trial, but regardless, they will not need to keep track of action-outcome mappings). Therefore, the fact that participants are slower to notice the shift in Ob trials and that their judgements are associated with lower confidence could easily be driven by the fact that they have three tasks (or at least two) rather than one. Relatedly, can the fact that α suppression and pupil dilation effects are increased with changes-of-mind in the Ob condition also be explained by the fact participants have more tasks? If there are greater executive/working memory demands, this will reduce α and increase dilation (as already discussed in the manuscript). Participants may detect stimulus changes less readily because there is a greater executive/working memory load in this condition, and this has nothing to do with controllability/action or stability of beliefs per se.

We understand the reviewer’s request for clarifications regarding the paradigm. From a computational standpoint, and although the uncontrollable (C-) and controllable (C+) conditions are different in that they require participants to track different hidden states, they do not differ in the number of cognitive operations (or ‘tasks’) that need to be performed to solve them. We now use model simulations to show that the observed differences between the two conditions are inconsistent with the idea that participants have more cognitive operations to perform in the C+ condition. Finally, in response to the second part of the reviewer’s comment below, we provide new analyses of the behavioral data that speak against a higher load in the C+ condition. We hope that these different additions have clarified why the observed differences between the two conditions cannot be attributed to a difference in executive or working memory load.

In both conditions, participants track a single hidden state, but the nature of this hidden state differs between conditions. Each condition requires tracking a single hidden state of the task: the category (A or B) drawn by the computer in the uncontrollable condition, and the action (left or right) drawing the target category in the C+ condition. In both conditions, participants need to select an action for each stimulus sequence. In the uncontrollable condition, participants select the action most likely to be associated with the category being drawn by the computer, based on a category-action rule defined at the beginning of the current block (A-left and B-right or vice versa). In the C+ condition, participants select the action most likely to be associated with the target category defined at the beginning of the current block. In other words, the uncontrollable condition requires monitoring the drawn category which flips occasionally, but does not require monitoring the category-action rule which is known and fixed over the course of the block. By contrast, the C+ condition requires monitoring the category-action rule which flips occasionally, but does not require monitoring the target category which is known and fixed over the course of the block. Importantly, in both conditions, each trial (sequence of stimuli) is associated with a single action (key press). Therefore, and while the two conditions are indisputably different, because participants have instrumental control over the category being drawn in the C+ condition, they do not differ in the number of task variables that need to be monitored.

We have now rewritten the protocol in our Methods section to make this point clearer (p. 27-28):

“Participants performed two experimental conditions that aimed at examining the influence of the degree of control over stimuli on choice and confidence. In both conditions, participants were required to track a hidden state (category). In the uncontrollable (C-) condition, participants were instructed that the computer draws sequences of stimuli, and were asked to identify the category from which the stimuli were drawn (Figure 1C). An instruction screen indicated the mapping between response keys and color categories (counterbalanced across blocks). In the controllable (C+) condition, participants were instructed to draw stimuli from a given category (Figure 1C). An instruction screen indicated the target color category for each block (counterbalanced across blocks). Consequently, the hidden state differed between conditions: participants monitored changes in the category being drawn in the uncontrollable condition, whereas they monitored changes in the response key drawing from the target category in the controllable condition (Figure 1D). In both conditions, participants perform one action per sequence (after seeing a sequence in the uncontrollable condition, before seeing a sequence in the controllable condition). In other words, the uncontrollable condition requires monitoring the drawn category which flips occasionally, but does not require monitoring the category-action rule which is known and fixed over the course of the block. By contrast, the controllable condition requires monitoring the category-action rule which flips occasionally, but does not require monitoring the target category which is known and fixed over the course of the block. The conditions were therefore otherwise fully symmetric, tightly matched in terms of visual and motor requirements, and working memory demands. The order of condition administration was also counterbalanced pseudo-randomly across participants (C-/C+/C/C+ for odd-numbered subjects, C+/C-/C+/C- for even-numbered subjects).”

Beyond the structure of the paradigm, there are also several aspects of the behavioral and neurophysiological data that are inconsistent with the idea of an increased executive or working memory load in the C+ condition. We agree with the reviewers that reduced attention and/or increased load due to the larger number of tasks to perform in the C+ condition would translate into slower behavioral adaptation to hidden-state reversals, as observed in the C+ condition. However, this alternative account of the behavioral effect would also produce a decrease in the asymptotic reversal rate in the C+ condition – a prediction which conflicts with the increased asymptotic reversal rate observed in this condition (Figure 2B).

To support this argument, we ran model simulations implementing the idea of an increased working memory load. Specifically, we reasoned that increased working memory load would result in attentional lapses, meaning that either a fraction of stimuli within a sequence would be missed (variant 1), or a fraction of sequences would be missed (variant 2). We implemented such a model with attentional lapses corrupting inference at the level of stimuli (variant 1) or sequences (variant 2; Author response image 1). We started from best-fitting parameters in the C- condition, and added attentional lapses to simulate behavior in the C+ condition.

Author response image 1

Download asset Open asset

At the level of stimuli (variant 1, top row), model simulations indicate that 60% of stimuli would need to be missed (i.e., ignored) to obtain a choice-PSE – i.e., the quantity of conflicting evidence needed to switch – similar to the one we observed in the C+ condition. This fraction of attentional lapses seems highly unlikely.

As expected, adaptation to reversals is slower in the presence of lapses. Psychometric fits of model simulations are qualitatively inconsistent with those of human participants: with lapses, the asymptotic reversal rate is strongly decreased relative to the C- condition (without lapses). At the level of sequences (variant 2, bottom row), a similar pattern is obtained: 30% of sequences would need to be missed to obtain a choice-PSE similar to that of the one observed in the C+ condition, which again seems highly unlikely. As for stimulus-level lapses (variant 1), sequence-level lapses produce a slower adaptation to reversals and a reduced asymptotic reversal rate, an effect inconsistent with participants’ behavior in the C+ condition (Figure R1). Together, these patterns are therefore inconsistent with an interpretation that the C+ condition required an increased demand.

Furthermore, attentional lapses due to increased working memory load would decrease the sensitivity to the available objective evidence – a prediction which conflicts with the similar sensitivity found in the two conditions (Figure 2F). An increased working memory load would also predict slower response times (RTs) in the C+ condition on repeat decisions, which we did not observe in our data (please see response to the next comment for a full breakdown). Together, these different considerations make it highly unlikely that observed differences between conditions are due to an increased executive or working memory load in the C+ condition.

Last, there are important aspects in physiological data that are inconsistent with an increased load in the C+ condition. Indeed, an increase in working memory load (due to more tasks to perform in the C+ condition) should trigger increased arousal and attentional signals across all trials in the C+ condition. This prediction does not match our observations (Figure 6F and 7B): indeed, α-band power and pupil dilation do not differ between conditions for trials ending with a repetition of the previous response (the majority of the trials). These two neurophysiological measures differ between conditions only for trials ending with a switch from the previous response (i.e., the minority of trials associated with information seeking). These selective differences between conditions for switch trials only are again inconsistent with the idea of an overall increase in executive or working memory load in the C+ condition.

We have now added a new paragraph in the Discussion section to describe – and rule out – this alternative account of the differences between conditions (p. 24):

“Finally, based on theoretical considerations and empirical results, we also found evidence that our controllability manipulation did not create differences in working memory or executive demands between experimental conditions. We designed our conditions so that they were strictly matched in terms of number of tasks to do, quantities to monitor, sources of uncertainty, and motor actions to perform. At the behavioral level, the lack of a difference in choice accuracy, sensitivity to the objective evidence, and inference noise parameter between conditions makes it unlikely that the C+ condition was more demanding. At the physiological level, an increased load should have triggered changes in attention and arousal signals across all trials, unlike our observations that the neurophysiological measures only differed on switch decisions, whereas no difference was observed on repeat decisions. Together, these considerations make it highly unlikely that observed differences between conditions are due to an increased executive or working memory load in the C+ condition.”

The authors claim that the participants were "equally sensitive" to the available objective evidence across conditions, which would indeed be reassuring, but at the moment it is not clear how solid this inference is. It would be good for the authors to clarify what they mean here – did they test for a significant slope/precision effect in the data shown in Figure 3A? Is it possible to say participants are equally sensitive to the evidence (other than on switch trials), when the insight into their sensitivity is given by a response that only requires them to process the evidence on switch trials? More generally, it remains to be convincingly demonstrated that the Ob-Cb differences are generated by something other than number of tasks and generic working memory/executive differences. For example, it would be important to demonstrate that performance accuracy (i.e. how often participants make the correct choice), and choice reaction times are equivalent between conditions, which could be achieved using a Bayesian approach. (This analysis would need to exclude the switch trials). If there are differences in performance accuracy or reaction times across conditions, then this could undermine the authors' conclusions.

Upon the reviewer’s suggestions, we have performed several additional analyses that provide strong evidence against a difference in executive demands or working memory load between conditions, which we develop below.

1) Sensitivity to the evidence. Two findings support the claim that participants were equally sensitive to the available objective evidence across conditions. It is important to note that participants had to process the evidence provided by stimuli on all trials, not only on switch trials. First, as suggested by the reviewer, we have tested for a difference in slope in the data shown in Figure 2E/F. We found no difference in this slope parameter (t₃₂=1.09, p=.28), as displayed on (ex-Figure 3B) as the ‘sensitivity to evidence’ parameter:

Second, in our computational model, the best-fitting inference noise parameter was not significantly larger in the C+ condition (t₃₂=1.45, p=.16; Figure 4B). This parameter corresponds to a model-based equivalent of the variability of participants’ decisions with respect to the objective evidence presented (inverse of sensitivity to evidence). Together, these results indicate that objective evidence was integrated equally well across experimental conditions.

2) Choice accuracy. There was a difference in choice accuracy (i.e., how often participants make the correct choice) between conditions (t₃₂=-3.103, p=.004). Importantly however, the difference was in the direction of a worse choice accuracy in the C- (78.74% correct) than in the C+ (80.57% correct) condition. This result therefore goes against the alternative interpretation raised by the reviewer that the C+ condition would create more tasks and more cognitive demands, hence a lower choice accuracy. Instead, the performance of participants in the C+ condition being not degraded and actually a bit higher suggests that participants do not miss significantly more information in this condition.

Despite being significant in Experiments 1 and 2A, the magnitude of the difference in choice accuracy is small (a 2% increase). Moreover, across datasets, this result was not consistent. Indeed, no difference in choice accuracy was found in Experiment 2B (t₁₇=.58, p=.569). No difference in choice accuracy was found in Experiment 4 (t₂₃=0.3, p=.784) (Weiss et al., 2021). In an ongoing online version of the task (unpublished) that corresponds to the experimental design of Experiments 1 and 2A, we found no difference in choice accuracy between conditions with N=200 participants (t₁₉₉=-1.46, p=0.145; C-: mean=77.06% correct; C+: mean=77.96% correct). Together, these observations provide further evidence against significant differences in working memory demands between conditions.

3) Response times. Upon the reviewer’s suggestion, we compared RTs between conditions on repeat trials. We found similar RTs between C- and C+ condition (t₃₂=-1.52, p=.14). These results indicate that the differences between conditions are unlikely to be generated by working memory or executive differences. Using JASP, we further performed a Bayesian t-test which provided evidence for a genuine lack of a difference between C- and C+ conditions (repeat decisions: BF₁₀=0.531). Together, these findings provide additional evidence that the working memory demands were very similar across experimental conditions, and that participants did not perform more tasks in the C+ condition (Results section, p. 7):

“Participants’ choice accuracy was higher in the C+ (80.6% correct) than in the C- (78.7% correct) condition (t₃₂=-3.103, p=.004), although this difference was only of 2% correct. Moreover, response times (RTs) on repeat decisions were similar between C- and C+ condition (t₃₂=-1.52, p=.14, BF₁₀=0.531). These initial results indicate that the differences between conditions are unlikely to be generated by working memory or executive demand differences.”

4) Additional condition. Experiment 2B includes a manipulation which effectively changes the number of tasks that need to be performed. We changed the category-action mapping randomly from one trial to the next in the C- condition, and changed the target category randomly from one trial to the next in the C+ condition. This variant requires participants to account for the rule provided on each trial, rather than relying on a fixed rule across trials in the original variant of the two conditions (Experiments 1 and 2A). To measure the effects of an increase in the number of tasks that need to be performed, we compared task metrics between Experiments 2A and 2B variants of the two conditions (done by the same participants). Importantly, we found that the choice-PSE did not increase in the conditions of Experiment 2B where more tasks need to be performed as compared to the conditions of Experiment 2A (C-: t₁₇=.77, p=.45; C+: t₁₇=.73, p=.47). Furthermore, the difference in choice-PSE between the C- and C+ conditions did not differ between Experiments 2A and 2B (t₁₇=.49, p=.63). Together, these additional findings show that increasing the number of tasks that need to be performed does not alter the choice-PSE in either condition. Therefore, even if participants performed more tasks in the C+ condition (something which is highly unlikely based on the evidence detailed above), it is unlikely that this difference would cause the larger choice-PSE in the C+ condition, or the selective change in pupil dilation and α power only in trials ending with response switches (i.e., where participants engage in information seeking).

We now report these new findings in the Results section (p. 10):

“Importantly, even if Experiment 2B effectively changes the number of tasks that need to be performed, we observed a similar Choice-PSE between Experiments 2A and 2B (C-: t₁₇=.77, p=.45; C+: t₁₇=.73, p=.47), and the difference in choice-PSE between the C- and C+ conditions did not differ between Experiments 2A and

2B (t₁₇=.49, p=.63).”

In our Discussion section, we now detail our new analyses and findings regarding the working memory demands of the two conditions (p. 24):

“Finally, based on theoretical considerations and empirical results, we also found evidence that our controllability manipulation did not create differences in working memory or executive demands between experimental conditions. We designed our conditions so that they were strictly matched in terms of number of tasks to do, quantities to monitor, sources of uncertainty, and motor actions to perform. At the behavioral level, the lack of a difference in choice accuracy, sensitivity to the objective evidence, and inference noise parameter between conditions makes it unlikely that the C+ condition was more demanding. At the physiological level, an increased load should have triggered changes in attention and arousal signals across all trials, unlike our observations that the neurophysiological measures only differed on switch decisions, whereas no difference was observed on repeat decisions. Together, these considerations make it highly unlikely that observed differences between conditions are due to an increased executive or working memory load in the C+ condition.”

2) In experiment 3, which aimed to test whether the temporal direction of the inference (prospective vs retrospective) could in fact explain the observed differences between condition, the effects on change of mind are not shown. Instead, only the effects on confidence are displayed – why is that? Additionally, the confidence data presented in Figure S1 still shows some apparent differences: lower overall confidence, higher confidence time-constant, and higher confidence-PSE in the prospective vs retrospective condition. Those differences are reported but the authors then nonetheless conclude that these results constitute evidence that the observed differences between Ob and Cb conditions in the main experiment are only due to controllability and not to the temporal orientation. At present, this conclusion is not supported by the data. One could conclude that temporality has no effect only if no difference in choice or confidence were observed. Again, a Bayesian approach to demonstrate equivalence could be of use here.

We thank the reviewer for raising the important point of the nature of the difference between the C- and C+ conditions, whether controllability or temporal orientation (prospective vs. retrospective). We now also present the choice data in addition to the confidence data from Experiment 3, which we adapted from Weiss et al., 2021 and included in a new Figure 2 supplement 1. Critically, the findings of Experiment 3 reveal that the nature and magnitude of the contrast between retrospective and prospective conditions is different from the contrast between the original C- and C+ conditions, and therefore cannot account for them as an overall explanation of the results, even if a remaining effect of temporal direction of the inference might still be present in the C- vs. C+ conditions, which we now acknowledge in our Discussion section. We also expand below on the differences and similarities between Experiment 3 and the main Experiments 1 and 2A.

Choices. In contrast to the comparison between C- and C+ conditions, there were no significant differences in choice adaptation after a reversal between retrospective and prospective conditions. Participants’ reversal time constants were similar across retrospective and prospective conditions (t₂₄=0.1, p=.98). Critically, the amount of inconsistent evidence required to change their mind (choice-PSE) was the same between retrospective and prospective conditions (t₂₄=1.7, p=.109), unlike between the original C- and C+ conditions.

Confidence. First, there was no difference in overall confidence between retrospective and prospective conditions (t₂₄=-1.2, p=.25). Moreover, the differences between conditions are not the same, of different direction. Even if there was a higher confidence time constant (t₂₄=-2.58, p=.016), the effect is small and noisy: indeed, it depends on the presence of a confidence drop to be measured reliably. In Experiment 3, however, the confidence drop was not different from zero in both retrospective (t₂₄=1.6, p=.12) and prospective (t₂₄=.42, p=.67) conditions. Therefore, the confidence time constant should not be interpreted, and we no longer present it on Figure 2 supplement 1, due to the unreliability of its estimation. Second, even if the confidence-PSE is increased in the prospective as compared to the retrospective condition, the confidence drop is no longer different between retrospective and prospective conditions. Third, the sensitivity of confidence to the evidence on switch decisions, that was about twice larger in C- as compared to C+, is now similar between prospective and retrospective conditions (t₂₄=-0.94, p=.35). This result indicates that participants were less confident both in the face of evidence consistent and inconsistent with their previous choice, which was not the case for the original C- and C+ conditions.

Taken together, these results indicate that even if a small temporality effect possibly remained present in Experiments 1 and 2A, it cannot alone be an overall explanation of our results, and cannot explain the main differences observed between the C- and C+ conditions as a whole, which we therefore ascribe mostly to controllability. We now include a Discussion point for acknowledging this alternative interpretation in our Discussion section (p. 23):

“Furthermore, we sought to validate controllability as the true cause of differences between conditions. In Experiment 3, we examined whether a distinction in temporal focus (prospective instead of retrospective inference in an uncontrollable context) would account for the differences between the original conditions (Figure 2 supplement 1). Although the pattern of choices was markedly different, for confidence it remains possible that a lingering effect of temporality affected the original conditions, even if it cannot account for the results overall.”

Introduction

3) Please unify, or at least operationalize more clearly, the terms that are used to describe the main effect of interest. Is information seeking (which indeed is often studied in bandit-type tasks) the same as changes of mind (which are more often studied in within-trial evidence accumulation paradigms)? How does this relate to belief updating, hypothesis testing, exploration, and the observers' stickiness tendency? While it is laudable that the authors reach across subfields of neuroscience that have their own terminology, in places the text was confusing about what's a general psychological process vs. an effect in the behavioral data vs. a model-derived computational latent variable.

We thank the reviewer for prompting clarity on the meaning and terminology for the different effects of interest. We agree with the reviewer that the term ‘exploration’ in the literature has a heterogeneous meaning, and has been used to refer to a number of different cognitive processes.

We have typically used ‘changes-of-mind’ to refer to the cognitive operation, while ‘response switch’ and ‘switch decision’ refer to the behaviour observable by the experimenter. We consider information seeking to be only possible in the controllable condition, whereas in the uncontrollable condition, only passive sampling occurs. It is the control over evidence conferred to participants in the controllable experimental condition that renders information seeking possible. We have revised the manuscript to use appropriate terms throughout, and we have now updated Figure 1 about the paradigm to explain the interactions between the different experimental and cognitive events. Hidden-state reversals are determined by the experimenter (“task event”). During an exploratory decision, several steps co-occur: a covert change-of-mind (“cognitive event”) about the expected reward of the current option; an overt response switch (“behavioral event”); and information seeking, i.e., an active search for information about the new option being considered, which is only possible in the C+ condition where participants are in control. (Results section, p. 6):

Upon the reviewer’s suggestion, in our introduction section, we now clarify the relationship between information seeking and other related concepts such as belief updating, hypothesis testing and exploration:

“In these paradigms, participants evolve in controllable environments and usually sample one among several options to maximize reward. Therefore, they have to either exploit a currently rewarding option, or sacrifice rewards to explore alternative options and seek information about their possible rewards (Rich and Gureckis, 2018; Wilson et al., 2014). This trade-off means that in these paradigms, exploration differs from exploitation not only in terms of information seeking, but also in terms of other co-occurring cognitive events, including overt response switches and covert changes-of-mind.

These different families of paradigms developed for studying information seeking vary on several dimensions, particularly the sources of uncertainty (Fleming et al., 2018), the stimuli used (Gesiarz et al., 2019), the desirability of the information to be sought (Hertwig et al., 2021), and the degree of control over information sampled (Desender et al., 2018). These differences have made direct comparisons between paradigms extremely challenging. To date, no study has directly manipulated control over evidence sampling in otherwise aligned experimental conditions.” (p. 3-4)

4) Related to the above point, some of the terminology or logical formulation of conclusions was a bit confusing. The authors use the term "information-seeking" specifically to refer to the controllable condition. In the uncontrollable condition, there is no "seeking" of information, only passive sampling. If that understanding is correct, some sentences are redundant. For example, shouldn't the title read "Controllability reveals defining features of information-sampling" or even (and more consistent with the findings) "Controllability over information sampling reveals defining features of changes of mind"? At times "controllability" and "information-seeking" seem to be used interchangeably, while at other times they seem to have different meanings. It would be important to specify those terms better.

We thank the reviewer for prompting clarification on the concepts manipulated. The reviewer is correct that we consider information seeking to be only possible in the controllable condition, whereas in the uncontrollable condition, only passive sampling occurs. We define controllability as the manipulation created by the experimenter. It is the control over evidence conferred to participants in the controllable experimental condition that renders information seeking possible. We have revised the manuscript so as to use the right terms for the right meaning throughout, and we have now updated Figure 1 about the paradigm to explain the interactions between the different experimental and cognitive events. Hidden-state reversals are determined by the experimenter (“task event”). During an exploratory decision, several steps co-occur: a covert change-of-mind (“cognitive event”) about the expected reward of the current option; an overt response switch (“behavioral event”); and information seeking, i.e., an active search for information about the new option being considered, which is only possible in the C+ condition where participants are in control.

Accordingly, following upon the reviewer’s suggestion, we have now changed the title of the study to “Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments”, a title now focusing more specifically on our results.

Although we fully agree with the reviewer’s statement that control is necessary for information seeking, other authors have reasoned otherwise, based on other definitions of information seeking. In particular, Monosov and Rushworth distinguish the notions of ‘passive’ and ‘active’ information seeking (Bromberg-Martin and Monosov, 2020 Current Opinion in Behavioral Science; Monosov and Rushworth, 2022 Neuropsychopharmacology). The authors refer to decision-making tasks under uncertainty, in which non human primates typically can select to sample information about potential rewards, even if the information sampled typically has no direct instrumental value, and does not confer them control over these rewards (e.g., White et al., 2019 Nat Commun). Such ‘non instrumental information sampling’ or ‘passive information seeking’ is proposed to be valuable in and of itself if acquiring this information might be useful in the future for guiding behavior – i.e., that might become instrumental later on in the long run. However, our definition of information seeking is tightly coupled to the presence of active control, in line with what other authors have proposed (Gureckis and Markant, 2012 Psychological Science). We refer to the notion of non-instrumental information sampling (not seeking) in our Discussion section:

“Our results are also generally consistent with medial frontal activations during hypothesis testing (for a review, see Monosov and Rushworth, 2022) and in non-instrumental information sampling, where the gathered information affords to predict but does not influence future outcomes.” (Discussion section, p. 24)

5) It would greatly help the reader to include a short paragraph in the introduction listing the ways in which this work replicates vs. extends the previous paper by Weiss et al.

We thank the reviewer for prompting clarity on the respective purpose of each study. The previous paper by Weiss et al., 2021 focused on the physiological analysis of evidence integration within a sequence. We pooled all trials and focused our analyses on the comparison between how participants process evidence when it is a cue (uncontrollable condition) vs. when it is an outcome (controllable condition). We did not focus on the differences between repeat and switch trials in this previous paper.

In sharp contrast to this previous study, here, we specifically focus on response switches, when participants change their mind about the current hidden state. This focus allows us to study and characterise a moment in which participants seek information in the task. Importantly, for behavioral analyses, we no longer analyse what happens during the course of a sequence of stimuli, we instead take into account the cumulative evidence brought by the whole sequence. Our physiological analyses then focus on studying the contrast between repeat and switch decisions between experimental conditions so as to characterise the neural basis of information seeking. We now clarify the goal of each study in our introduction:

“We previously used these experimental conditions to compare how participants integrate evidence when it is a cue (uncontrollable condition) vs. when it is an outcome (controllable condition) (Weiss et al., 2021). Here, we focus on the comparison of repeat and switch decisions between these conditions so as to isolate the behavioral signatures and neural basis of information seeking, while replicating most of the previously observed effects in Weiss et al. on behavioral choices and their computational modeling (Weiss et al., 2021).” (p. 4)

6) Overall, the introduction is a little thin and would benefit from being expanded. First, the statement that "information-seeking has been mostly studied under 'exploration-exploitation' dilemma is not true – there are many recent studies that have studied information-seeking in humans using other paradigms. Second, and most importantly, the introduction currently lacks the rationale for the proposed work – in addition to dissociating information-seeking from changes-of-mind, which is more of a methodological aim, what are the key questions that the authors trying to address, and what are the hypotheses given the current literature? Finally, the use of MEG and other physiological measures (pupillometric, cardiac patterns) is not motivated at all in the introduction. The introduction should set the stage as to why collecting these data is needed given the question of interest. Similarly, the hypotheses should also set the stage for the specific analysis choices performed later, i.e. in what way does each presented analysis answer the question?

We thank the reviewer for prompting clarity on the purpose of the study, the motivation for our hypotheses, and the conceptual advances on information seeking processes that the study allows.

First, we agree with the reviewer that other important previous work has studied information seeking outside of exploitation-exploration dilemmas, which we now review in more depth. In particular, we have identified several families of paradigms tested in human and non-human primates which we now unpack in our introduction, highlighting the similarities and differences with the present paradigm:

1. A first family of information seeking paradigms are belief updating tasks in which human participants are presented with statements about life events or self-relevant information or judgements from others, and are asked whether they would like to see more information about this (e.g., Sharot et al., 2011 Nat Neurosci; Kelly and Sharot, 2021 Nat Commun; Gesiarz et al., 2019 Plos comput biology). Another “intermediate” study has manipulated not actual controllability, but instructed controllability, i.e., what participants are told about their degree of control (Stolz et al., 2020 Nature communications). Typically, these trials are not decision-making under uncertainty but are one-shots, and do not require learning or adaptation unlike in our study and in more continuous and open-ended ecological environments in which information seeking usually occurs. These decisions are described (in the sense of Hertwig and Erev, 2009) with probabilities or uncertainty depicted directly without ambiguity. Another important difference is that the nature of the information proposed varies in its valence and desirability (e.g., Hertwig et al., 2021 Psychology and Aging).

2. A second family of paradigms relies on perceptual decision-making as a model system to study the cognitive and neurobiological mechanisms underlying perceptual changes-of-mind (e.g., Desender et al., 2018 Psychological Science; Fleming et al., 2018 Nat Neurosci; Rollwage et al., 2020 Nat Commun). In these studies, expected uncertainty typically comes from the same source and is manipulated as the amount of sensory noise present in decision evidence. All trials and independent and there is no notion of volatility or reversals to adapt to – no unexpected uncertainty is manipulated.

3. Non-instrumental information-seeking has been the focus of recent studies in non-human primates (Monosov and Rushworth, 2022 Neuropsychopharmacology). In these studies, typically non-human primates are presented with visual cues predicting upcoming rewards, and acquiring information about these cues presents no immediate or direct instrumental value to the animal (e.g., White et al., 2019 Nat Commun). These paradigms also include no notion of volatility, with independent trials (no hidden state to track).

4. In decision-making under uncertainty, finally, a common family of paradigms is bandit tasks in which participants are asked to sample between two or more slot machines so as to maximize gains or rewards, and/or avoid losses or punishments, leading to exploite-explore dilemmas (e.g., Daw et al., 2006 Nature; Zorowitz and Niv, 2021 BioRxiv; Collins and Frank, 2012 Eur J of Neurosci), or other forms of economic decision-making (Kaanders et al., 2021 J Neuro; Wilson et al., 2014 J Exp Psychol Gen), and foraging (e.g., Kolling et al., 2012 Science). The present paradigm follows this line of research but critically moves away from it and explore-exploit dilemmas and instead focus on isolating information seeking from exploratory decisions, i.e., when people change their mind about the reliability of the current strategy and go explore something else.

These previous families of paradigms vary on many dimensions, in particular the sources of uncertainty sources and the type of stimuli manipulated, which has made their findings’ direct comparison difficult. Critically, none of these previous paradigms has directly manipulated controllability over decision evidence (information acquired). The unique contribution of the present study is to isolate information seeking by contrasting evidence sampling under controllable and uncontrollable conditions, all other decision features such as stimuli and levels and forms of uncertainty being equal.

Second, the main goal of the study in isolating information seeking from changes-of-mind is much more fundamental than a methodological goal. We have now reworked Figure 1 and included a new panel to explain the different concepts at play in the study. Current protocols proposed to study information seeking in the context of exploitation-exploration trade-offs have important confounds, in that information seeking is virtually always associated with changes-of-mind, meaning that different cognitive steps co-occur that are not specific to information seeking per se. It is an important conceptual distinction that has not been addressed by previous work (although for an exception, see Collins and Koechlin, 2012 Plos biology).

Third, we agree with the reviewer that our physiological hypotheses should be presented early on and we now better motivate our physiological analyses in our introduction section. We sought to evaluate whether there were additional correlates of subjective uncertainty associated with the process of information seeking. Because a switch response engages participants’ sense of agency in the C+ but not in the C- condition, we expected this form of evidence sampling to engage different networks, but it has not been examined in previous work whether these different forms of information seeking rely on separate or similar brain systems.

We have now rewritten our introduction to reflect this larger background (p. 3-4):

“In these paradigms, participants evolve in controllable environments and usually sample one among several options to maximize reward. Therefore, they have to either exploit a currently rewarding option, or sacrifice rewards to explore alternative options and seek information about their possible rewards (Rich and Gureckis, 2018; Wilson et al., 2014). This trade-off means that in these paradigms, exploration differs from exploitation not only in terms of information seeking, but also in terms of other co-occurring cognitive events, including overt response switches and covert changes-of-mind.

These different families of paradigms developed for studying information seeking vary on several dimensions, particularly the sources of uncertainty (Fleming et al., 2018), the stimuli used (Gesiarz et al., 2019), the desirability of the information to be sought (Hertwig et al., 2021), and the degree of control over information sampled (Desender et al., 2018). These differences have made direct comparisons between paradigms extremely challenging. To date, no study has directly manipulated control over evidence sampling in otherwise aligned experimental conditions. At the neurophysiological level, exploration is known to be associated with larger pupil-linked arousal (Jepma and Nieuwenhuis, 2011) and increased activity in lateral prefrontal regions in electroencephalographic (EEG) and blood oxygen-level dependent (BOLD) activity (Donoso et al., 2014; Tzovara et al., 2012). However, due to confounds between information seeking and other cognitive events during exploration in these studies, it remains unclear whether information seeking is associated with specific neurophysiological signatures.”

Methods

7) Was the model fitted to all trials together, or separately for each condition?

We have now clarified this point in our Methods section (p. 29):

“All trials were fitted together, but each parameter was allowed to vary between conditions, and we compared the best-fitting parameters using paired t-tests at the group level.”

8) The model validation is rigorous, but would the same patterns also be observed out-of-sample? Since only one model is tested, there is always a risk of overfitting, which could be addressed with out-of-sample validation.

Here, our goal was not to arbitrate between models, but to compare our two conditions under the same model, with parameters between conditions under the same model. This is why we have fitted the same model in both our conditions. We have selected the most parameterised model, with all the free parameters allowed to vary between conditions. Even if we were to be overfitting, this is not an issue here, because our statistics are based on the comparison between best-fitting parameter values across C+ and C- conditions. Overfitting does not inflate the probability of observing significant differences in parameter values across conditions. This is important in order to avoid that parameters’ estimates may be contaminated by variance that is unrelated to the cognitive process each parameter aims to capture. Our approach here is similar to incorporating regressors of no interest in a regression, so as to avoid regressors of interest to capture unrelated variance.

In addition, the behavioral effects associated with each of our parameters at the group level indicate that overfitting is unlikely here. If we were overfitting, meaning that if one of the parameters was essentially capturing noise, we would not observe consistent effects of a given parameter across participants, as is illustrated in Figure 4 supplement 3 (choice parameters) and Figure 4 supplement 4 (confidence parameters). For example, when we do a median split of participants according to their best-fitting perceived hazard rate, we observe a translation in the repetition curve indicating that the lower the hazard rate, the more inconsistent evidence is needed for participants to change their mind. Had hazard rate captured noise in case of overfitting, we would not observe such a consistent effect at the group level.

In sum, our assumption that overfitting is very unlikely to have occurred is validated by the fact that each parameter is specifically associated with a given consistent behavioral pattern (Figure 4 supplement 3 and 4), which would not happen if any of these parameters was capturing noise as in the case of overfitting. We now provide more information about our model validation approach and the risk of overfitting in our Methods section (p. 30):

“Here, we selected the most complete (parameterized) model, even at the risk of overfitting, because our goal was not to arbitrate between different models, but to compare our two conditions and our parameters under the same model. In addition, the behavioral effects associated with each of our parameters at the group level indicate that overfitting is unlikely to have occurred (see ‘Model validation’ below). Indeed, if overfitting had occurred, meaning that if one of the parameters was essentially capturing noise, we would not observe consistent effects of a given parameter across participants (Figure 4 supplement 3 and 4).”

9) Please provides the statistics for the validation data presented on p12 (Figure S2 and S3)

Figure S2 (now Figure 2 supplement 3) provides model simulations from individual best-fitting parameters. Upon the reviewer’s suggestion, we have now performed a full psychometric analysis of these simulation data. Note that the psychometric analysis of model behavior is not as central to our conclusions as that of participants’ behavior. Indeed, the purpose of model validation is to ensure that the model reproduces key signatures or behavioral patterns present in the human data (Palminteri et al., 2017 TICS; Wilson and Collins, 2019 eLife), which our model does very well. For the sake of concision, we have therefore included the statistical values for this full psychometric analysis directly into the figure, but we report them below for completeness.

We observed that the model was slower to adapt after a reversal in the C+ condition, with psychometric fits indicating a higher reversal time constant in this controllable condition (t₃₂=-6.0, p=9.5×10^-7). The model behavior also reached a higher asymptotic reversal rate in the C+ condition (t₃₂=-6.6, p=2.0×10^-7). The model’s simulated confidence decreased after a reversal in both conditions. However, confidence decreased more sharply in the C+ condition as indicated by psychometric fits of a confidence ‘drop’ parameter (t₃₂=5.22, p=1.0×10^-5). The confidence time constant characterizing the slope of confidence increase after a reversal was similar across conditions (paired t-test, t₃₂=-0.50, p=.62).

We found a stronger choice-PSE in the C+ condition, as in human data (t₃₂=-8.3, p=1.7×10^-9), together with a similar sensitivity to the evidence, as in human data (t₃₂=0.79, p=0.44). Finally, we found a significant confidence-PSE difference between conditions (Wilcoxon signed rank test, z=-3.59, p=.00033), indicating that the model needed more inconsistent evidence to be equally confident in switch and repeat decisions in the C+ condition, like humans. We also found that the sensitivity of confidence reports to evidence (slope) was slightly smaller in the C+ than in the C- condition on switch decisions, albeit only borderline significant (t₃₂=1.86, p=0.07).

Figure S3 (now Figure 4 supplement 3 and 4) displays the effects of parameters on the four main behavioral signatures, namely, reversal and repetition curves for choice and confidence, from which we only draw qualitative conclusions. The purpose of these figures is only to visualise the selective effects of each of our free parameters. We do not perform any group inference or statistical analyses at the group level from these analyses. We now make this point clear in our Methods section (p. 30-31):

“We also performed a median split across participants on the best-fitting parameter values (Figure 4 supplement 3 and 4). We then averaged simulations of the model for each subgroup, to further illustrate the independent contribution of each of these parameters. Note that we do not draw group inferences from these median split analyses – the goal is to visualise the effect of each of the parameters qualitatively.”

10) How recoverable are the parameters if the model is re-fit to simulated data?

We have now performed a full parameter recovery analysis, as described in our Methods section (p. 29):

“To ensure that our fitting procedure was unbiased, we performed a parameter recovery analysis (Figure 4 supplement 1). We simulated choice and confidence sequences using generative parameters randomly sampled in a uniform distribution between the minimum and maximum of participants’ best-fitting parameter values in each condition. This procedure ensures that generative parameters were independently sampled. We then fitted these data using the same fitting procedure as for participants (except with three instead of five random starting points) and calculated the correlations between generative and recovered parameters, presented in a confusion matrix (Figure 4 supplement 1).”

We now provide these results in a new Figure 4 supplement 1.

We found a satisfactory parameter recovery, indicating that our fitting procedure was reliable, as explained in our Results section (p. 12):

“Importantly, we also established that our fitting procedure provided a satisfactory parameter recovery (see Methods). All correlations between generative and recovered parameters were high (all rho>0.78, all p<1014), while other correlations were low as indicated in a confusion matrix (Figure 4 supplement 1).”

11) Are parameters correlated with each other? The authors should provide a correlation table of the parameters in the supplement.

We have now inspected the correlation between our five model parameters. Apart from a significant negative correlation between inference noise and hazard rate (rho=-0.51, p=.0025), and a borderline negative correlation between confidence threshold and hazard rate (rho=-0.36, p=.0409), no other correlations were significant (all rho<0.31, all p>.074) (N=33 participants, individual best-fitting parameters averaged between conditions). We now provide a correlation matrix of parameters as a new Figure 4 supplement 2:

We also provide these findings in our Results section and comment on their implications (p. 14):

“We further validated the independent role of each parameter in two ways. First, we examined correlations between best-fitting parameters across participants (Figure 4 supplement 2). We found a significant negative correlation between inference noise and hazard rate (rho=-0.51, p=.0025), in line with a previously reported trade-off between these two sources of noise (Weiss et al., 2021). We also found a borderline correlation between confidence threshold and hazard rate (rho=-0.36, p=.0409). However, all other correlations were not significant (all rho<0.31, all p>.074), indicating that each parameter captured independent portions of the variance (Figure 4 supplement 2). Second, we did a median-split of participants into groups of high and low parameter values (Figure 4 supplement 3 and 4), each parameter having a selective influence on choices and confidence. Even when parameters were similar across conditions (e.g. confidence gain), there was still a substantial inter-individual variability that had a visible effect on participants’ confidence, indicating the necessity of each parameter in capturing qualitative signatures participants’ choices and confidence. Although the model allows for potentially distinct effects between inference noise and hazard rate, here, our participants happened to be distributed along a particular regime, with no independence between these two parameters, as visible in similar effects of lower inference noise and higher hazard rate on behavior (Figure 4 supplement 3), in line with their negative correlation.”

“We examined correlations of the best-fitting parameters averaged across conditions between participants (Figure 4 supplement 2)”. (Methods section, p. 29)

Results

12) It would help to preview the specific diagnostic effects that isolate the process of interest, and then to repeat those in the model fit. Currently, we see four effects in Figure 2, four effects in Figure 3, and three effects in Figure 4. When we get to Figure 5D, slightly different effects are presented in comparison to the model. This was quite confusing and more clarity is required about the specific behavioral pattern of interest. This may just be a matter of slight reordering: the supplement has clear model-data matches, and placing those in the main figure (to highlight that most of the basic effects in Figures2-4 are indeed explained by the model) would allow a better appreciation of where the model fails to capture human behavior.

We thank the reviewer for prompting clarity on the specific behavioral signatures that are key to diagnose the model validity and have now fully reorganised our figures accordingly. We have now grouped ex. Figure 2 and Figure 3 into a single figure with the four key curves and the corresponding psychometric parameters (new Figure 2). To allow for an easy comparison between human and model behaviours, we now have the model validation in a figure aligned with the same signature patterns.

Upon the reviewer’s suggestion, we have separated the ex. Figure 5C and 5D in a new separate Figure 5 to isolate where the model fails to capture human behavior. We hope that this reorganisation now allows to appreciate the similarities between model and human behavior and between experiments.

13) (see also (2) above) The data in Figure S1 are a crucial control, but it is not yet clear that there is no substantial difference in pro- vs. retro-spective decisions, which might (partially) explain the Cb vs. Ob differences. If controllability was the only important factor for behavioral effects, wouldn't we expect no difference between conditions in Figure S1? This would require a change in the authors' strength of interpretation as specifically pertaining to control.

We agree with the reviewer that the strength of our conclusions on the nature of the difference between the C- and C+ conditions, whether controllability or temporal orientation (prospective vs. retrospective), need to be aligned with the evidence provided and we now nuance our interpretations. We refer the reviewer to our detailed response to the main comment 2.

Briefly, we now also present the choice data in addition to the confidence data from Experiment 3. Critically, the findings of Experiment 3 reveal that the nature and magnitude of the contrast between retrospective and prospective conditions is different from the contrast between the original C- and C+ conditions, and therefore cannot account for them as an overall explanation of the results; however, a remaining effect of temporal direction of the inference might still be present in the C- vs. C+ conditions, which we now acknowledge:

“Furthermore, we sought to validate controllability as the true cause of differences between conditions. In Experiment 3, we examined whether a distinction in temporal focus (prospective instead of retrospective inference in an uncontrollable context) would account for the differences between the original conditions (Figure 2 supplement 1). Although the pattern of choices was markedly different, for confidence it remains possible that a lingering effect of temporality affected the original conditions, even if it cannot account for the results overall.” (Discussion section, p. 23)

14) Do the analyses in Figure 4 control for the difference in objective evidence (logL per trial)?

We controlled indeed for the strength of evidence provided by the sequence of stimuli in Figure 2G-2H for the data presented in (ex)Figure 4A (now Figure 3). In other words, Figure 4A presents the same data as Figure 2G but pooled across evidence strength levels. However, because in Figure 4A predictors would be binary, i.e., repeat and switch, and C- and C+ conditions, we considered that an ANOVA was the most relevant and appropriate test here. Unfortunately, too few events per cell are included in Figure 4B and 4C to further be able to do a breakdown as a function of evidence strength in bins, as we did for confidence in Figure 4A. Moreover, it is worth noting that the observed effects are in the opposite direction of what we would expect if evidence strength was a confound. Indeed, we found that in the C+ condition, participants need more evidence to switch, which should lead to higher confidence (more evidence => more confidence). Instead, we found a reduced confidence on switch decisions in the C+ condition.

Regarding Figure 4B and 4C (now Figure 3B and 3C), these analyses do not control for variations in objective evidence. Data for these analyses cannot be further split further as a function of objective evidence due to a low number of events corresponding to each case (cell) studied: as expected, there was not a homogenous amount of evidence leading all of these events. For instance, participants would rarely confirm a change-ofmind with high confidence if it was based on very little objective evidence.

Note, however, that by design, the objective evidence level was matched across conditions on average, and that the PSE psychometric analyses take into account the objective evidence – instead, Figure 4 focuses on the consequences of response switches on participants’ subsequent behaviour.

We now clarify this point in our Methods section (p. 29):

“In a 2 × 2 repeated measures ANOVA, we examined the influence of RESPONSE TYPE (repeat, switch) and CONDITION (C-, C+) on the fraction of high confidence responses (Figure 3A, which corresponds to the same data as Figure 2G pooled over objective evidence levels). For switch trials, we further examined confidence on switch trials that were confirmed on the next trial (“change-of-mind confirmed”) as compared to switch trials after which participants went back to their previous response (“change-of-mind aborted”; Figure 3B). Finally, we examined the fraction of changes-of-mind confirmed (over all changes-of-mind) as a function of whether the change-of-mind was done with high or low confidence (Figure 3C). These analyses are pooled over objective evidence levels due to each of these events not being distributed homogeneously across evidence levels.”

Upon the reviewer’s suggestion, we have now performed a logistic regression to take into account strength of evidence. Specifically, we included the trial-by-trial evidence strength in favour of the previous response as a co-regressor:

Confidence(t) (high/low) ~ Response type(t) (repeat/switch) + Condition(t) (C-/C+) + Response type(t)*Condition(t) + Evidence level(t)

and

Confidence(t) (high/low) ~ change-of-mind(t) (confirmed/aborted) + Condition(t) (C-/C+) + change-ofmind(t)*Condition(t) + Evidence level(t)

We implemented a regularised logistic regression with gaussian priors on each regression coefficient (mean=0, SD=2) in order to account for the fact that some of our participants have few events per cell, which otherwise led to unreliable regression coefficient estimates.

The results are in line with our initial model-free analysis (original Figure 4A). Namely, repeat trials led to higher confidence (t₃₂=7.66, p=9.8×10^-9), confidence was higher in the C- than in the C+ condition (t₃₂=4.05, p=.00029), with a significant interaction between Response type and Condition factors (t₃₂=-4.43, p=.0001), while controlling for evidence level in the same regression model, which positively contributed to confidence, as expected (t₃₂=14.69, p=8.8×10^-16) (Figure 3 supplement 1A).

Likewise, in a logistic regression we have controlled for evidence strength in the analysis relating confidence to whether the change-of-mind of the previous trial was confirmed or aborted (related to original Figure 4B). The results confirm model-free observations, with participants overall being more confident when they confirm than when they abort a change-of-mind (t₃₂=5.49, p=4.6×10^-6), also more confident in the C- than in the C+ condition (t₃₂=3.92, p=.0004), with a significant interaction between these factor (t₃₂=-3.38, p=.0019), while controlling for evidence level (t₃₂=11.9, p=2.49×10^-13) (Figure 3 supplement 1B).

We also computed, for each of the cases, the average evidence quantity in favour of the participant’s previous response, and compared the obtained interactions patterns to those obtained in the original Figure 4 (Figure 3 supplement 1C-D). If the interaction patterns are different from the original results, it means that the original results cannot be explained away by differences in evidence level.

In a 2 × 2 ANOVA on mean evidence level, as expected, we found a main effect of RESPONSE TYPE, F_1,32=147.8, p=1.6x10^-13, because negative evidence will lead participants to switch from their previous response. We found a higher overall evidence level in the C+ than in the C- conditions (main effect of CONDITION, F_1,32=6.19, p=.018). We observed a significant interaction between RESPONSE TYPE and CONDITION (F_1,32=57.1, p=1.3x10^-8), indicating that in the C+ condition, participants have seen more evidence against their previous choice when they switch their response than in the C- condition, in line with choice-PSE results. Having seen more evidence, we would expect them to be more confident, whereas they actually are less confident in that condition (original Figure 4A). Therefore, these findings provide further evidence that the strength of evidence does not explain away our results on confidence in response switches.

Moreover, evidence levels were higher on trials in which switch decisions were confirmed as compared to aborted (F_1,32=264.2, p<.001). There was no main effect of CONDITION (F_1,32=0.67, p=.41), with an interaction between CONDITION and CONFIRM-ABORT factors (F_1,32=42.6, p=2.3x10^-7) (Figure 3 supplement 1B). This pattern reflects that evidence level partly contributes to confidence reports, but is markedly different from the patterns observed for the fraction of high confidence responses (original Figure 4B).

In the case of original Figure 4C, we have displayed proportions, namely, the fraction of changes-of-mind confirmed out of the total changes-of-mind performed with high confidence and low confidence respectively. For this reason, we cannot compute a mean evidence level in each of the four cases in original Figure 4C. Instead, to take into account evidence level, we can compare the mean evidence level on (i) changes-ofmind performed with high confidence and then confirmed, (ii) performed with low confidence and then confirmed, (iii) performed with high confidence and then aborted, and (iv) performed with low confidence and then aborted, separately for the two conditions so a total of 8 cells. However, data for these analyses cannot be further split further as a function of evidence due to a low number of events corresponding to each case (cell) studied: as expected, there was not a homogenous amount of evidence leading all of these events. For instance, participants would rarely confirm a change-of-mind with high confidence if it was based on very little objective evidence. Likewise, a logistic regression would not be appropriate if we have too few data points, as in the case of the original Figure 4C. Note, however, that by design, the objective evidence level was matched across conditions on average, and that the PSE psychometric analyses take into account the objective evidence – instead, Figure 4 focuses on the consequences of response switches on participants’ subsequent behaviour.

We now clarify this point in our Methods section (p. 29):

“In a 2 × 2 repeated measures ANOVA, we examined the influence of RESPONSE TYPE (repeat, switch) and CONDITION (C-, C+) on the fraction of high confidence responses (Figure 3A, which corresponds to the same data as Figure 2G pooled over objective evidence levels). For switch trials, we further examined confidence on switch trials that were confirmed on the next trial (“change-of-mind confirmed”) as compared to switch trials after which participants went back to their previous response (“change-of-mind aborted”; Figure 3B). Finally, we examined the fraction of changes-of-mind confirmed (over all changes-of-mind) as a function of whether the change-of-mind was done with high or low confidence (Figure 3C). These analyses are pooled over objective evidence levels due to each of these events not being distributed homogeneously across evidence levels.”

We also include these new logistic regressions controlling for evidence level in our revised manuscript:

“To take into account any effects of evidence strength in these change-of-mind analyses, we further performed two logistic regressions (Figure 3 supplement 1A-B). First, we aimed to predict confidence using Response type, Condition, their interaction, and trial-by-trial evidence strength in favour of the previous response as a co-regressor (related to Figure 3A). Second, we aimed to predict confidence using change-ofmind (confirmed vs. aborted), Condition, their interaction, and trial-by-trial evidence strength in favour of the previous response (related to Figure 3B). We implemented regularised logistic regressions with gaussian priors on each regression coefficient (mean=0, SD=2) in order to account for the fact that some of our participants have few events per cell, which otherwise led to unreliable regression coefficient estimates. Group-level significance of regression coefficients was assessed using one-sample t-tests. We also computed the average evidence quantity in favour of the participant’s previous response, and compared the obtained interactions patterns to those obtained in the change-of-mind analyses in Figure 3 (Figure 3 supplement 1C-D).” (Methods section, p. 31)

“A logistic regression confirmed these results (see Methods). Repeat trials led to higher confidence than switch decisions (t₃₂=7.66, p=9.8×10^-9), confidence was higher in the C- than in the C+ condition (t₃₂=4.05, p=.00029), with a significant interaction between RESPONSE TYPE and CONDITION (t₃₂=-4.43, p=.0001), while controlling for evidence level in the same regression model, which positively contributed to confidence, as expected (t₃₂=14.69, p=8.8×10^-16) (Figure 3 supplement 1A).” (Results section, p. 11)

“A logistic regression confirmed these results (see Methods), with participants overall being more confident when they confirm than when they abort a change-of-mind (t₃₂=5.49, p=4.6×10^-6), also more confident in the C- than in the C+ condition (t₃₂=3.92, p=.0004), with a significant interaction between these factor (t₃₂=-3.38, p=.0019), while controlling for evidence level (t₃₂=11.9, p=2.49×10^-13) (Figure 3 supplement 1B). Moreover, while these results indicate that evidence partly contributes to confidence, as expected, the patterns of evidence strength were markedly different from those of changes-of-mind (compare Figure 3A-B to Figure 3 supplement 1C-D).” (Results section, p. 12)

15) It is frequently stated that the Ob condition bestows instrumental control to participants over the sampling of stimuli. These statements should be modified, given that participants are told which category of stimulus they must generate. They do not have complete control over which stimuli they sample, unless they are willing to upset the experimenter!

The reviewer is correct that the control is indeed over the category of stimuli produced, not over each stimulus in the sequence itself. In other words, participants have control / can decide over which stimulus to sample (left or right action in the controllable condition), although they do not have control over the target category, i.e., the generative category from which samples are drawn.

Controllability is a property of this C+ (ex. Ob) condition, in which participants have a goal, just like in classic n-arm bandit task. It is not a forced choice but they are asked to produce stimuli from a particular target category; in that sense, it is not a pure free sampling but goal-directed sampling. We have now clarified this point in our Methods section (p. 27):

“In the controllable (C+) condition, participants were instructed to draw stimuli from a given category (Figure 1C). An instruction screen indicated the target color category for each block (counterbalanced across blocks). In the C+ condition, participants have control over which stimuli to sample, but are not fully freely sampling, since they asked to produce stimuli from a target color category.”

16) It is not clear what action participants perform to start the trial in the Ob condition. This needs including. In outlining this it would also be great to include consideration of potential overlaps between the responses at the start and end of the trial in Ob, and whether this might have played into the effects.

In both conditions, participants perform exactly only one motor action per trial. In the very beginning of a block of trials of the C+ condition, participants are asked to perform a random action to initiate the very first sequence of stimuli, as they have no idea yet of what the current mapping of categories is. In the C- condition, similarly, participants are asked to initiate the block of trials by performing a random action. Therefore, the first response required before having seen any sequence is random in both conditions, so that motor actions are precisely aligned between the C+ and C- conditions.

We now clarify this point in our Results (p. 5) and Methods (p. 26-27) sections:

“In-between each sequence, participants were asked to provide a response regarding the current hidden state of the task, together with a binary confidence estimate in their response (high or low confidence) using four response keys (Figure 1B). […] In both conditions, participants perform one action per trial.” (p. 5)

“The conditions were otherwise fully symmetric, tightly matched in terms of visual and motor requirements. In both conditions, participants perform one action per sequence (after seeing a sequence in the C- condition, before seeing a sequence in the C+ condition).” (p. 26-27)

17) Please add the change-of-mind data (i.e. equivalent of Figure 2A-B and 3A-B) for experiment 3, so it is possible to dissociate which effects are unique to the main experiment and which effects may be due to the temporal direction of the decisions. It is important to clarify this distinction as it can actually help narrow down the unique effect of controllability over temporality.

We have now added the choice psychometric analysis (equivalent of ex Figure 2A-B, adapted from Weiss et al., 2021) and the confidence psychometric analysis (equivalent of ex Figure 3A-B) in a new Figure 2 supplement 1 (printed in response to comment 2). We fully agree with the reviewer about the importance of isolating the unique effect of controllability over temporality using Experiment 3, and we refer the reviewer to our responses to comment (2) that addresses this point in depth.

18) It would be useful to provide follow-up analyses to determine how much of the behavioral results can be explained by the difference in prior belief reported on p19, as well as discuss how much of these findings are then more consistent with a role of controllability per se, rather than a role for motivated beliefs/confirmation bias due to the presence of a target.

We agree with the reviewer that these notions are related, and we now provide a number of pieces of evidence in our data that speak against an interpretation in terms of confirmation bias or motivated beliefs.

First, by using a computational model, we seek to provide a mechanistic explanation for our observed behavioral effects. Specifically, stronger prior beliefs are actually what will produce the effects we see. Controllability is only a high-level notion manipulated by the experimenter, while the computational model has no notion of controllability built into it. But by fitting the computational model to participants’ data in the two controllability conditions, the model happens to explain participants’ choices and confidence by the presence of stronger prior beliefs in the controllable (C+) condition. It is those stronger prior beliefs that produce behavior, not the other way around.

Second, Experiment 2B also addresses this point to some extent. In Experiment 2B, the target goal is not constant during a block and varies between trials. This means that if confirmation bias were an explanation for the findings, the direction of the confirmation bias would change from one trial to the next, and its impact on behaviour should therefore decrease.

Third, at the neural level, in our previous study, we have argued that “the strength of belief-inconsistent evidence could be decoded [from MEG signals] with equal precision across conditions”, something at odds with a confirmation bias which would decrease the weighting of belief-inconsistent evidence. Instead, we found an absence of difference between the neural coding of belief-consistent and belief-inconsistent evidence.

Fourth, at the behavioral level, the reviewer is correct that a confirmation bias in the C+ would be associated with an increased choice-PSE, which we observe in the data. Confirmation bias has previously been defined as a selective weighting of belief-consistent evidence (Bronfman et al., 2015 Proc R Soc B; Peters et al., 2017 Nat human behaviour; Talluri et al., 2018 Current Biol).

However, a confirmation bias in the C+ condition would also be accompanied by a significant decrease in the sensitivity to the objective evidence in the C+ condition, an aspect that is absent from the human data (Figure 2). Indeed, a confirmation bias would predict that in the C+ condition, the evidence used by the participant is not the objective evidence, but a biased subjective representation of the evidence where the inconsistent evidence is underweighted relative to the consistent evidence. Hence, sensitivity to the objective evidence (measured as our psychometric parameter) would be decreased in the presence of a confirmation bias in the C+ condition. In other words, we have considered the objective evidence with respect to participants’ previous response (Figure 2E-H) as a key relevant variable for decision-making, but if participants use irrelevant or wrong variables (e.g., confirmation bias), it should have produced a degraded sensitivity to the objective evidence (Beck et al. 2012 Neuron).

To falsify more directly a confirmation bias explanation, we have conducted a new analysis. We fitted the differences in response reversal and response repetitions curves (Figure 2) between the C- and C+ conditions in Experiment 4 using two alternative accounts: 1. a change in perceived hazard rate in the C+ condition, and 2. a confirmation bias in the C+ condition (controlled by a gain parameter assigned to belief-inconsistent evidence). As expected, a change in perceived hazard rate produced a minimal change in the sensitivity to evidence which matched human data (interaction with participants: F_1,23=0.4, p=0.54). In contrast, a confirmation bias produced a substantial decrease in the sensitivity to evidence in the C+ condition which did not match human data (interaction with participants: F_1,23=10.0, p=.004). This interaction ‘falsifies’ (in the meaning of Palminteri et al., 2017 TICS) the most standard account of confirmation bias.

We now provide evidence to rule out this alternative explanation in terms of “confirmation bias” in our Discussion section (p. 23-24):

“Despite stimuli carrying no explicitly affective or rewarding value, it remains possible that the mere presence of a target in the C+ condition makes the target category desirable, and may produce a form of confirmation bias (Talluri et al., 2018). However, at the behavioral level, a confirmation bias would predict that in the controllable condition, the sensitivity to evidence should be degraded when the sequence of evidence is inconsistent with participants’ previous choice, a prediction that is absent from human data (Figure 2). At the neural level, a confirmation bias would decrease the weighting of belief-inconsistent evidence. Instead, in our previous study, we found an absence of difference between the neural coding of belief-consistent and belief-inconsistent evidence (Weiss et al., 2021). In addition, the findings of Experiment 2B, where the target category changes from trial to trial, also mean that the differences between conditions are unlikely to reflect a bias in direction of the target category (Gesiarz et al., 2019). Indeed, the direction of this bias would change from one trial to the next, and should therefore decrease in Experiment 2B – which we did not observe.”

Discussion

19) There is some concern about the interpretation of the results – in particular, whether the observed differences between the cue-based and outcome-based conditions could be better explained by the presence of a target-induced confirmation bias in the outcome-based condition, which would induce a strong motivation to confirm that the chosen action does lead to the target. In other words, it is possible that having a target (e.g. "draw orange") may bias subjects towards believing they are drawing from that target even when they are not, and as a result needing less evidence to reach that conclusion (similar to the effect shown in Gesiarz, Cahill and Sharot, 2019, Plos Computational Biology). This could in turn lead to the observed patterns of results, i.e. needing more evidence against orange to switch, being less confident when switching etc. Additionally, the result that prior beliefs are stronger in the Ob condition (p18-19) is consistent with this idea, since confirmation bias is usually associated with stronger prior beliefs.

We would like to refer the reviewer to the series of arguments in response to comment (18), which we hope have thoroughly addressed the alternative explanation of a confirmation bias producing our pattern of results.

We also note that the types of motivated beliefs in the paradigm of Gesiarz et al., 2019 are beliefs that are associated with desirable states: one prefers to be in a state with greater rewards than losses rather than the other way around. In contrast, here, participants had no particular reason to be more motivated that the blue or the orange category of perceptual stimuli is the current hidden state. Moreover, no reward or losses were manipulated explicitly, for instance using monetary outcomes. No notion of valence or affective value is different between the C- and C+ conditions, or between the blue and orange generative categories, unlike in other confirmation bias paradigms where stimulus statements carry a strong affective content (e.g., Sharot et al., 2011 Nat Neurosci).

Nevertheless, it remains possible that the mere presence of a target category in the C+ condition could constitute a goal and create desirability to find more evidence in favour of that target category (Castagnetti et al., 2021 Sci Advances); however, as detailed in response to the previous comment, this would have translated into specific effects which we do not observe.

We now unpack this point in our Discussion section (p. 24):

“Despite stimuli carrying no explicitly affective or rewarding value, it remains possible that the mere presence of a target in the C+ condition makes the target category desirable, and may produce a form of confirmation bias (Talluri et al., 2018). However, at the behavioral level, a confirmation bias would predict that in the controllable condition, the sensitivity to evidence should be degraded when the sequence of evidence is inconsistent with participants’ previous choice, a prediction that is absent from human data (Figure 2). At the neural level, a confirmation bias would decrease the weighting of belief-inconsistent evidence. Instead, in our previous study, we found an absence of difference between the neural coding of belief-consistent and belief-inconsistent evidence (Weiss et al., 2021). In addition, the findings of Experiment 2B, where the target category changes from trial to trial, also mean that the differences between conditions are unlikely to reflect a bias in direction of the target category (Gesiarz et al., 2019). Indeed, the direction of this bias would change from one trial to the next, and should therefore decrease in Experiment 2B – which we did not observe.”

20) The authors claim that there is a causal role for confidence in controlling changes-of-mind (p11). While this is an interesting idea, it is not clear that it can be fully evidenced in this task without an experimentally-controlled manipulation of confidence. The reason is that there could be a common cause to both high confidence and high propensity to confirm switch decisions without the two processes actually being causally related. One such common cause could be the strength of evidence. Therefore, this conclusion needs to be tempered and this issue mentioned in the discussion.

We agree with the reviewer about the existence of a common cause being possible. Note that we do take into account the strength of the evidence, one of such possible common causes, in our analyses on confidence (Figure 3, now Figure 2G-H). We would also like to refer the reviewer to our response to comment (14) and (xiii), in which we provide new analyses taking into account the strength of evidence, which shows consistent results. Here, in writing “a role for confidence”, we meant causality in a weak sense: the sense of temporal precedence. Yet, because we have not performed a direct causal manipulation of confidence here, we now have now tempered the claims associated with these findings in our Results section, paragraph ‘Separable effects of confidence and controllability on changes-of-mind’:

‘A possible role for confidence in changes-of-mind’ (title of Figure 3, p. 11)

‘We found a main effect of CONFIDENCE on the fraction of switches confirmed (F_1,27=30.4, p=7.8×10^-6), meaning that participants confirmed their switch more often when it was made with high confidence. This suggests a causal role for confidence in controlling changes-of-mind, even though we acknowledge that there was no experimentally causal manipulation of confidence here.’ (p. 12)

21) There is no discussion of alternative ways in which these data might have turned out. The discussion entirely reports claims and findings with which the present data are consistent. Are there no data or theories that could have led to alternative predictions? Currently the manuscript portrays the impression of such high consensus that there was little point running the studies, suggesting that the empirical patterns to date all point in identical directions. One possibility is that the stimulus ISIs are not very variable. Given participants can perceive better at certain oscillatory phases of visual processing (peaks), could they choose to start trials in Ob to align with peaks and thereby improve perceptual acuity? If this occurred, the influence must be weaker than those pulling in the opposite direction, but it could have led to an alternative outcome. If it's a straightforward analysis to perform/report, it may be interesting to see how the oscillations align with events differentially in Ob and Cb.

The purpose of the present study was to examine changes-of-mind in controllable and uncontrollable while otherwise comparable conditions, an endeavor that has not been examined before, to the best of our knowledge. It was not an easy or a given or trivial observation that the results turned out the way they have: indeed, information seeking has been mostly studied in separate fields of the decision-making literature, among which for instance:

(i) in the context of perceptual decision-making tasks, where participants are presented with evidence over which they have no control (e.g., Desender et al., 2018 Psychological Science);

(ii) in the context of belief update paradigms, where the information to be acquired is controllable and the proposed information typically varies in its desirability (e.g., Gesiarz et al., 2019 Plos computational biology); (iii) in the context of value-based decision-making tasks, where participants have control over options sampled (e.g., Daw et al., 2006 Nature).

It had not been investigated whether the different forms of information sampling at play would be similar or turn out to be different in controllable and uncontrollable environments, which we could investigate in the present paradigm. We have now highlighted this in our Introduction section (p. 3-4):

“In these paradigms, participants evolve in controllable environments and usually sample one among several options to maximize reward. Therefore, they have to either exploit a currently rewarding option, or sacrifice rewards to explore alternative options and seek information about their possible rewards (Rich and Gureckis, 2018; Wilson et al., 2014). This trade-off means that in these paradigms, exploration differs from exploitation not only in terms of information seeking, but also in terms of other co-occurring cognitive events, including overt response switches and covert changes-of-mind.

These different families of paradigms developed for studying information seeking vary on several dimensions, particularly the sources of uncertainty (Fleming et al., 2018), the stimuli used (Gesiarz et al., 2019), the desirability of the information to be sought (Hertwig et al., 2021), and the degree of control over information sampled (Desender et al., 2018). These differences have made direct comparisons between paradigms extremely challenging. To date, no study has directly manipulated control over evidence sampling in otherwise aligned experimental conditions. At the neurophysiological level, exploration is known to be associated with larger pupil-linked arousal (Jepma and Nieuwenhuis, 2011) and increased activity in lateral prefrontal regions in electroencephalographic (EEG) and blood oxygen-level dependent (BOLD) activity (Donoso et al., 2014; Tzovara et al., 2012). However, due to confounds between information seeking and other cognitive events during exploration in these studies, it remains unclear whether information seeking is associated with specific neurophysiological signatures.”

Regarding the alternative interpretation proposed by the reviewer, it is important to note that in both conditions, participants’ key presses trigger the presentation of the next sequence of stimuli after the same jitter delay in the two conditions. Therefore, participants could align their perceptual acuity with the stimuli equally well in the two conditions. Nevertheless, it is also important to note that stimuli are presented at 2 Hz, a rate that is sufficiently low to perceive stimuli properly. Here, the difficulty is to integrate, i.e., infer, the category source properly, despite perceiving stimuli easily. In other words, stimuli are of high contrast, they are not masked, and they are presented for sufficient time. Therefore, the cognitive bottleneck in our experiment is not on sensory processing, but on inference (see Drugowitsch et al., 2016 Neuron for an extensive decomposition of decision variability in terms of sensory, inference and action selection sources of variability).

We now clarify this point in our Methods section (p. 29):

“Stimuli were displayed at an average rate of 2 Hz with an inter-stimulus interval of 500 ± 50 ms. This rate is sufficiently low to perceive stimuli properly; here, the cognitive bottleneck was not on sensory processing, but on inference (Drugowitsch et al., 2016 Neuron). The last stimulus of each sequence was followed by a longer delay of 1000 ± 50 ms, before participants were probed for their response.”

Author details

1. Marion Rouault

   1. Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France
   2. Institut Jean Nicod, Centre National de la Recherche Scientifique (CNRS), Paris, France
   3. Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences et Lettres (PSL University), Paris, France

   Contribution

   Formal analysis, Investigation, Writing – original draft, Writing – review and editing

   For correspondence

   marion.rouault@gmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-6586-3788

2. Aurélien Weiss

   1. Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France
   2. Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences et Lettres (PSL University), Paris, France
   3. Université de Paris, Paris, France

   Contribution

   Data curation, Formal analysis, Investigation, Methodology

   Competing interests

   No competing interests declared

3. Junseok K Lee

   1. Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France
   2. Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences et Lettres (PSL University), Paris, France

   Contribution

   Data curation, Investigation

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-8622-6259

4. Jan Drugowitsch

   Department of Neurobiology, Harvard Medical School, Boston, United States

   Contribution

   Investigation, Methodology, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7846-0408

5. Valerian Chambon

   1. Institut Jean Nicod, Centre National de la Recherche Scientifique (CNRS), Paris, France
   2. Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences et Lettres (PSL University), Paris, France

   Contribution

   Data curation, Methodology, Writing – review and editing

   Contributed equally with

   Valentin Wyart

   Competing interests

   No competing interests declared

6. Valentin Wyart

   1. Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France
   2. Département d’Études Cognitives, École Normale Supérieure, Université Paris Sciences et Lettres (PSL University), Paris, France

   Contribution

   Conceptualization, Formal analysis, Supervision, Funding acquisition, Investigation, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Valerian Chambon

   For correspondence

   valentin.wyart@gmail.com

   Competing interests

   Reviewing editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0001-6522-7837

• 1,435

  Page views

• 209

  Downloads

• 2

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.