Many complex tasks we perform daily, though different in their details, share an abstract structure. For example, riding a bike and driving a car differ in many details, including the actions they require, but they share an abstract similarity as modes of transportation. It has long been proposed that humans can learn this abstract structure and leverage it to think creatively, make novel inferences and rapidly generalize knowledge to unique problems we have never encountered before (Gick and Holyoak, 1980; Harlow, 1949; Tenenbaum et al., 2011; Tolman, 1948) – e.g. applying rules learned about when to break on a bicycle to an automobile. Reproducing this generativity continues to vex even the most impressive artificial intelligences (Lake et al., 2017; Russin et al., 2020), and is arguably one of the most defining features of human cognition (Penn et al., 2008).

In reinforcement learning, the structure of a task is described by its decomposition into ‘states’ – variables that specify the condition of the environment (Sutton and Barto, 2018). The definition of states is critical to determining how learning takes place, such as whether new information is restricted to a specific set of circumstances or is shared between conditions with overlapping features. States that cannot be resolved from sensory information alone are said to be ‘hidden’ or ‘latent’ – in such cases, the organization of these states be must inferred from the distribution of rewards, or other outcomes, throughout the task. In practice, latent states (LSs) have been defined as distributions over the spatio-temporal occurrence of rewards or punishments (Gershman et al., 2013; Nassar et al., 2019), task stimuli or stimulus features (Collins and Frank, 2013; Gershman and Niv, 2013; Tomov et al., 2018), or conditionalization of action values based on recent task history (Schuck et al., 2016; Zhou et al., 2019). From this distribution over task features, an agent can infer which conditions might belong to the same LSs and which do not. This information can then be used to segregate or lump together observations, making learning more efficient, and enabling generalization of learning between settings that share the same LSs (Gershman and Niv, 2010; Niv, 2019).

Recent work has provided evidence that the orbitofrontal cortex (OFC) and hippocampus (HPC) maintain structured, abstract representations of tasks during planning and navigation in conceptual spaces, including latent task states (Liu et al., 2019; McKenzie et al., 2014; Schuck et al., 2016; Tavares et al., 2015; Wilson et al., 2014; Zhou et al., 2019). Likewise, other studies have investigated the neural processes supporting fast acquisition of stimulus-response rules based on latent task knowledge (Badre et al., 2010; Collins et al., 2014; Eichenbaum et al., 2020; Frank and Badre, 2012). However, an essential feature of an abstract representation is that it can be used to generalize behaviors to new settings, in absence of feedback, through a process of inference. To date, no study has investigated the neural systems that maintain an abstract task representation that is observed to satisfy this criterion. As a consequence, it remains unknown how the brain carries out this essential function. To address this gap, we used fMRI to test the hypothesis that latent, generalizable task representations used to control our behavior during a task are instantiated in neural activity, particularly in OFC and HPC.

Participants completed a task where they gathered rewards in an environment where a latent, generalizable structure was available. Throughout the experiment, images of trial-unique items from three categories appeared beneath a context, denoted by a scene (Figure 1). Participants decided whether to ‘sell’ each item or pass. If they sold, they would receive or lose reward probabilistically, as determined by the combination of item category and context. Participants saw each of these combinations in short batches of trials termed mini-blocks. Importantly, the contexts could be clustered based on the expected values associated with each item category. This structure provided an opportunity to link these contexts together using an abstract, LS representation based on the distribution of category-value associations over contexts. We hypothesized that participants would form, and later use, this representation to generalize adaptive behaviors to new conditions without any need for reinforcing feedback.

Figure 1 with 1 supplement see all

Download asset Open asset

This hypothesis was tested across three phases (Figure 1—figure supplement 1). During phase 1, participants learned about the abstract LSs using three item categories (hands, foods, and leaves) across nine contexts (Figure 1b). In the next phase, participants learned the values of three new item categories (faces, animals, and objects) in three of the nine contexts; one drawn from each LS (Figure 1c). Then, in a final generalization phase, they were tested on the remaining six contexts using the three new image categories, without feedback. Optimal performance depended on generalizing the new category values learned during the second phase to the held-out contexts (Figure 1d). We expected that this inference would depend on those contexts being linked to a LS representation within a generalizable task representation based on the expected values of categories encountered during the first phase.

Participants were tested in three sessions. In session 1, participants completed a behavioral version of the task where performance in the generalization phase determined their inclusion in the fMRI experiment in sessions 2 and 3. 48% of participants passed a criterion of ≥70% accuracy in all 18 generalization conditions and so were recruited for two fMRI sessions. Participants who failed to meet this criterion performed the same task in two additional behavioral sessions rather than in the scanner. Of these participants, 50% ultimately passed the accuracy criterion by the third session. Thus, the majority of participants (75%) could carry out this generalization task given sufficient experience (Figure 2—figure supplement 1).

In session 2, all participants carried out the same task, but with new context stimuli – requiring them to learn which contexts linked to which LSs anew. This session also included additional blocks of the generalization phase with conditions pseudo-randomized (rather than organized in mini-blocks). fMRI participants completed these additional blocks in the scanner and the rest of the experiment behaviorally. In session 3, all participants completed a shortened version of the training as a reminder of the task and were presented with new pseudo-randomized generalization blocks (in the scanner for fMRI participants).

Abstract task structure affords inference of novel behaviors without feedback

Analysis of learning curves within mini-blocks of trials demonstrates that participants immediately made use of the abstract task structure in the first segment of the generalization phase. Through the first few trials of each mini-block of the initial and new category training, fMRI participants’ accuracy steadily improved. In contrast, in the generalization phase, accuracy was near ceiling from the first trial (mean p(correct)=0.96, SD = 0.06; Figure 2a). The rate of this learning curve in the generalization phase was significantly lower than either of the other phases (Figure 2b), consistent with participants inferring an adaptive behavior without need for additional experience or feedback (Wilcoxon signed rank tests: Z’s ≥ 3.21, p’s ≤ 0.004, Bonferroni-corrected for multiple comparisons). Similar results were observed in participants who completed the behavioral version of the task in session 2 and passed the generalization accuracy criterion (Figure 2—figure supplement 2a), with a significantly lower rate of change in the accuracy learning curve in the generalization phase compared to the initial training or new category training phases (Wilcoxon signed rank tests: W’s = 45, p’s = 0.01, Bonferroni-corrected for multiple-comparisons).

Figure 2 with 2 supplements see all

Download asset Open asset

Notably, high accuracy during generalization was accompanied by elevated reaction times (RT; mean = 4.04 s, SD = 1.7 s) on the first trial, with a sharp drop-off subsequently (mean = 1.15 s, SD = 0.08 s; Figure 2c). The rate of speeding was significantly steeper in the generalization phase compared to the initial or new category training phases averaged across the first three contexts in each phase (Wilcoxon signed rank tests: Z ≥ 3.46, p≤0.002, Bonferroni corrected for multiple comparisons; Figure 2d). The same pattern in the rate of speeding was observed in participants from the behavioral group who passed the generalization criterion in session 2 (generalization versus initial training RT rates: W = 36, p=0.047, Bonferroni-corrected for multiple-comparisons; Figure 2—figure supplement 2d). Thus, rather than immediately applying a structure learned incidentally during training, participants took additional time to infer values during the first generalization trials – indicative of a concerted, deliberative process.

As each LS was repeated twice via two contexts during generalization, we could test how previously encountering a LS impacted RTs during a repetition. Participants were faster for the second presentation of contexts from the same LS during generalization compared to contexts from different LSs (repeated measures t-tests: (15)≥2.13, p’s < 0.05, d’s ≥ 0.53), indicating that additional inferences were made more quickly when a LS had been accessed previously (Figure 3a). Comparison of the rate of exponential curves fit to these RT data in each participant demonstrated that these curves were steeper in the first context compared to the second context averaged across LSs (Wilcoxon signed rank test: Z = 3.36, p=0.0007), but there were no differences in the rate of these curves according to the sequential order of LSs (Wilcoxon signed rank test: Z ≤ 1.13, p≥0.2, uncorrected; Figure 3b). A numerically similar pattern of results was observed for the rate of change for learning curves in the behavioral group who passed the generalization criterion, but did not reach statistical significance (Figure 3—figure supplement 1). These data argue that participants were faster in completing the transfer of new category values to contexts if a context belonging to the same LS representation had already been activated, but did not necessarily use information about other LSs they had encountered in narrowing this inference problem.

Figure 3 with 1 supplement see all

Download asset Open asset

Generalization behavior is best explained by a hierarchical LS representation

Notably, generalization in this task might not only be achieved through the use of a LS representation to bridge contexts. For example, an associative retrieval process could also explain these results if newly trained category values could be transferred through mediated retrieval of contexts held-out of the generalization phase via shared category-value associations learned in the initial training (Kumaran and McClelland, 2012; Schapiro et al., 2017).

To test this possibility, we compared four alternative computational models that generated predictions for participants’ RTs based on different memory network configurations where contexts, category-values, and LSs were represented as nodes with different levels of activation.

First, the conjunctive associative retrieval (CAR) model represented each condition as a combination of category and context features, and linked these conjunctive representations to each other based on shared features, but had no representation of LS (Figure 4a).

Figure 4 with 1 supplement see all

Download asset Open asset

The independent associative retrieval (IAR) model similarly did without a LS representation, but rather than only representing contexts and categories as conjunctions, maintained independent representations of each and linked contexts to each other based on shared category-value associations (Figure 4b).

In contrast to these associative retrieval models, we included two LS models. A simple LS model connected category-values to three LSs, but forewent any representation of context (Figure 4c). A hierarchical latent state (HLS) model connected category-values to contexts, and further clustered these contexts around LSs, so that activation of each context and category was inherited from the LS representation (Figure 4d).

Each of these models relied on the basic assumption that RTs depend on the degree of activation in the network nodes currently activated. On each trial, activation within the network would increase at different rates depending on the relation of nodes to those that were activated during retrieval. Three rate parameters determined this increase for nodes that were directly activated on a trial (α₁), nodes directly connected to those activated (α₂), and the rest of the network which might be searched but was not directly relevant to the current trial (α₃). To establish that these models differed substantially in their predictions, we tested each model on data simulated using the best fit model parameters from itself and the three alternatives. This cross-model comparison clearly demonstrated that the predictions of these four models were dissimilar and each model provided a much better fit to itself than its alternative counterparts (Figure 4—figure supplement 1a).

The HLS model provided the best fit to participants’ RT data in the fMRI group in sessions 1 and 2, and the nine participants in the behavioral group who passed the generalization accuracy criterion in session 2 (Table 1; Figure 4—figure supplement 1a). Comparison of the three rate parameters of this model showed an ordering consistent with the expectation that the degree of activation of items should fall off at a distance from the items presented on the current trial (Figure 4—figure supplement 1b), where values for α₁ were significantly greater than α₂ and α₃ (Wilcoxon signed rank test: Z’s = 3.51, p’s = 0.001, Bonferroni corrected), and values for α₂ were significantly greater than α₃ (Z = 2.68, p=0.02, Bonferroni corrected).

Table 1

	Conjunctive associative retrieval	Independent associative retrieval	Latent state	Hierarchical latent state
fMRI group – session 1	3537.78 (1/16)	3227.38 (2/16)	3428.23 (0/16)	3061.04 (13/16)
fMRI group – session 2	3849.60 (0/16)	3413.20 (4/16)	3569.10 (0/16)	3264.13 (12/16)
Behavioral group – session 2	2063.65 (0/9)	1851.76 (0/9)	1921.61 (0/9)	1713.73 (9/9)

Table 1—source data 1

Sum of negative log-likelihoods for individual participants in fMRI and behavioral groups for each model in each session.

https://cdn.elifesciences.org/articles/63226/elife-63226-table1-data1-v2.zip

Download elife-63226-table1-data1-v2.zip

To understand why the HLS model better accounted for participants’ data, we compared RTs simulated from the best fit model parameters for each participant with each of the four models, focusing on data from session 2 for the fMRI group, as it was more pertinent to the neuroimaging results (Figure 4a–d). We organized these data by the order of presentation of each context associated with each LS (as in Figure 3), switches of category mini-block, and all other trials within a mini-block and then compared the root-mean squared deviation (RMSD) of each model for each trial type. We found that both the LS models provided a better fit on category switch trials compared to the CAR model (repeated-measures t-test, t(15) ≥ 4.76, p’s ≤ 0.002, d ≥ 1.19, Bonferroni-corrected for multiple comparisons). Numerically, the HLS model also provided a better fit to these trials than the IAR model, though this difference was not statistically significant (t(15) = 2.02, p=0.06, d = 0.50, uncorrected). The two associative retrieval models tended to systematically overestimate RTs on these category switch trials, likely because category-values associated with the trained context in the new category training phase were only activated by mediated retrieval, whereas in the LS models these category-values benefited from inherited direct retrieval of the higher-order LS representation. The HLS model also had lower RMSD for the first presentation of the second context compared to the LS model (t(15) = 4.76, p=0.001, d = 1.19, Bonferroni-corrected for multiple comparisons). As the LS model had no separate representation of context, any activation related to a prior LS would be fully transferred to its second presentation through another context, resulting in an underestimation of RTs. Instead, the results were more consistent with separate context representations that inherit activation from a higher-order LSs, that is, the use of a HLS structure.

Representation of an abstract task representation in multi-voxel activity patterns

Having verified that participants form and use an abstract task representation to control behavior, we sought to test the neural systems that support this representation. We carried out a representational similarity searchlight analysis (RSA) using multiple linear regression to compare empirical representational dissimilarity matrices (RDMs) from pattern activity to hypothesis RDMs quantifying the predicted distances between conditions based on LSs, contexts, item categories, expected value, interactions between these factors, and control regressors (Figure 5—figure supplement 1).

We found evidence for a LS representation in bilateral dorsal- and ventrolateral prefrontal cortex (PFC) with a rostral distribution, as well as the bilateral precuneus, left middle temporal gyrus, and bilateral inferior parietal lobules (Figure 5a; Supplementary file 1). Comparing this statistical map with resting-state functional networks from Yeo et al., 2011 revealed that this LS representation overlapped most with a frontoparietal network centered on the inferior frontal and intraparietal sulci (Figure 5—figure supplement 2).

Figure 5 with 5 supplements see all

Download asset Open asset

In contrast, context was associated with activity in the bilateral fusiform gyri (Figure 5b). Expected value was associated with activity in OFC, as well as left superior frontal and angular gyri (Figure 5c). Item category was robustly represented throughout visual areas and PFC (caudally on the left and broadly on the right; Figure 5—figure supplement 3).

To test whether these results were consistent across fMRI sessions, we carried out separate whole-brain searchlight RSAs within each session. The resultant statistical maps for the main effects of interest were similar in both sessions (Figure 5—figure supplement 4). Contrasts between statistical maps for all terms in the multiple regression model only revealed a stronger effect for value in the left middle temporal gyrus in session 2 compared to session 3 (MNI coordinates: −48,–22, 0), and no other significant differences in either direction.

As a secondary question, we were also interested in whether higher-order representations of latent task states and expected value influenced each other, or perceptual-semantic representations of item category. To test this, we included regressors for hypothesis RDMs that captured interactions between LSs, value, and item category, as well as control regressors for interactions of these factors with context. Interactions between item category with value and LS were not significant; though items with same value in the same LS were represented more similarly in ventral temporal cortex (VTC) (Figure 5—figure supplement 1), observations recapitulated in ROI-based analyses of activity within the VTC (Figure 6b).

Figure 6 with 1 supplement see all

Download asset Open asset

We also conducted ROI-based RSAs focused on HPC and OFC, given a priori expectation that these regions are involved in representing latent task structure during task performance. RSAs within these ROIs were null for LS representations (Figure 6a,c). To test these effects at a finer grain, we conducted second-level tests of the whole-brain searchlight analysis masked by these ROIs. Here, we found signals consistent with LS and expected value representations in central OFC, but not HPC (Figure 6—figure supplement 1; Supplementary file 2). There were significant effects for category representations in both of these ROIs, in line with broad discrimination between semantically and perceptually distinct categories of stimuli in these regions (Chikazoe et al., 2014; Kuhl et al., 2012; McNamee et al., 2013; Pegors et al., 2015).

Univariate contrast of accuracy

Lastly, we tested a univariate contrast of correct and error responses. Although less functionally specific than the above analyses, we expected this contrast to reveal regions broadly involved in engaging with this generalization task. This contrast revealed greater activation in the HPC and OFC on correct responses (Figure 7; Supplementary file 3).

Figure 7

Download asset Open asset

Here, we observed that a LS representation capable of supporting generalization of knowledge to new settings is instantiated most strongly within activity in mid-lateral PFC and parietal cortex. This pattern overlaps with a network previously implicated in contextual modulation of behavior in hierarchical reinforcement learning and cognitive control tasks (Badre et al., 2010; Choi et al., 2018; Collins et al., 2014). Like hierarchical task structure in those experiments, latent task states enable faster learning, reduced memory load, and greater behavioral flexibility (Frank and Badre, 2012; Koechlin and Summerfield, 2007). Recent work has similarly implicated mid-lateral and inferior parietal cortex in inferring latent causes within tasks (Tomov et al., 2018) and discovering hierarchical rules (Collins et al., 2014; Eichenbaum et al., 2020). Our findings extend these observations to indicate that this network is not just involved in discovering and maintaining task structure, but more generally represents tasks in an abstract format that facilitates novel behaviors, even in the absence of reinforcement. Once formed, such a fully abstract task representation could potentially be used not only for online, stable task control in the face of changing circumstances, but also as the generative mechanism for rapidly generalizing diverse knowledge between settings, affording great behavioral flexibility with minimal training.

Our behavioral results offer insights into the timing and nature of processes that form this abstract task representation. While some accounts of generalization argue that integration of related items occurs during encoding (Shohamy and Wagner, 2008; Zeithamova et al., 2012a), others have suggested that this can happen online during retrieval through statistical inference (Kumaran and McClelland, 2012). In this task, participants could build their task representation at any point prior to the generalization phase, including during the initial training phase. The lengthy RTs during the first trials of generalization suggest that this inference required additional online processing, and this abstract task structure may not have been fully constructed until it was probed. These data are consistent with accounts arguing that generalization during acquired equivalence can also occur at retrieval (De Araujo Sanchez and Zeithamova, 2020) and depends on the demand to make new inferences. It is likely that linking of contexts to a LS occurs at both encoding and retrieval, with the timecourse of this process varying across individuals.

Modeling of RTs during the first mini-blocked section of the generalization phase has also provided some insights into the process of how generalization of task information takes place. An associative retrieval model without any LS representation underestimated RTs on category switch trials, as this model did not separate category-values by context or LS. However, a fully integrative representation of LSs was also insufficient, as this alternative collapsed related contexts into the same LS, and underestimated the time needed to make additional inferences about contexts that had not been presented earlier. Instead, our data were most consistent with an intermediate solution where related contexts were bridged by a LS, and category-values were separated by contexts. Thus, our data were consistent with participants accomplishing generalization via an integrated task representation on the basis of shared features (Schlichting and Preston, 2015; Shohamy and Wagner, 2008; Zeithamova et al., 2012b), while still retaining independent representations of each context.

Our task design may have shaped this representation: the instructions strongly encouraged participants to link contexts to an unseen latent variable (see Materials and methods), and the mini-block trial structure likely encouraged learning the values of each category within each context. Thus, ours is an evidence that we can form such LS representations, use them for generalization, and represent them in frontoparietal network while performing a task based on them. We do not rule out that people cannot also use associative retrieval or that they may do so instead of using a LS in other task contexts. Nonetheless, evidence that we can, and do, form LSs under certain circumstances is an important observation that confirms a long held hypothesis in the field.

Behavioral evidence also indicated that this LS representation was maintained throughout the scanned generalization task. Analysis of RTs showed switch costs for latent task states that exceeded those for changes of context alone. In principle, this representation was not necessary for this portion of the task, as participants could have instead consulted the inferred values of the 18 context-category combinations, or simply memorized which six conditions were associated with negative expected values. Instead, participants appeared to continue referencing this more abstract, and compact, representation of the three latent task states to aid decision-making. Maintaining such a representation would be particularly helpful as a means of compressing task information in memory, and it could be useful in case of further need to update information about these latent task states and propagate this information to linked task conditions (e.g. in a reversal of the expected values for a particular context-category combination). Characterizing if, and how, the neural representation of latent task states described here comes to influence decision-making and learning in such settings remains an important outstanding question.

In this experiment, we were able to cross two factors previously associated with OFC function within the same experiment: latent task states and expected value (Stalnaker et al., 2015). We found limited evidence for OFC representing latent task states, though expected value was robustly represented in this ROI. These data are consistent with rodent electrophysiological data indicating that expected value signals contribute more to the variance of activity within this region than representations of task space (Farovik et al., 2015; Zhou et al., 2019). However, both OFC and HPC were activated for correct responses compared to errors, possibly consistent with their role in retrieving relevant task knowledge, if not in actively representing latent task structure in this experiment.

Prior observations may be reconciled with our results if distinct neural circuits are involved in representing latent task states under different cognitive demands. Much recent work has focused on the involvement of the medial PFC, OFC, and HPC in learning and representing structured knowledge, from the relations of paired associates to complex schemas (Baldassano et al., 2018; Chen et al., 2017; Garvert et al., 2017; Ghosh et al., 2014; van Kesteren et al., 2013). However, the network identified here as representing latent task states shares greater similarity with regions in lateral PFC and inferior parietal cortex identified by past studies focused on abstract representations of a task in cognitive control settings (Loose et al., 2017; Woolgar et al., 2011). While these regions belong to mostly distinct functional networks (Choi et al., 2018; Yeo et al., 2011), information may be shared via strong connections between mid-lateral PFC with medial PFC, OFC, and HPC (Averbeck and Seo, 2008; Petrides, 2005), and regions that straddle these networks, such as the angular and superior frontal gyri (Margulies et al., 2009). Information passed through these select pathways might allow structured knowledge about abstract task elements to inform a control representation in mid-lateral PFC (Badre and Nee, 2018).

Frontoparietal systems may be more involved in representing this kind of abstract task structure when it is necessary for actively controlling behavior contingent on changing contexts, as in this experiment. Other work that has examined analogous abstract task representations in the context of conditional action selection and selecting between causal models have found evidence for the involvement of frontoparietal cortex (Collins et al., 2014; Eichenbaum et al., 2020; Tomov et al., 2018), similar to our results. In contrast, OFC and HPC could be engaged more by resolving uncertainty about the prevailing latent task state (Chan et al., 2016; Saez et al., 2015), or planning within a cognitive map of latent task states (Schuck et al., 2016; Zhou et al., 2019). Other related work indicates that these regions are involved in, and necessary for, constructing abstract representations (Kumaran et al., 2009; Schuck and Niv, 2019; Spalding et al., 2018; Zeithamova et al., 2012a), and so may be particularly engaged by the process of bridging contexts and forming a representation of latent task states. Thus, while OFC and HPC may work together to form an understanding of the latent task structure, mid-lateral PFC and parietal cortex are perhaps more involved in implementing this representation for the purposes of controlling behavior. Indeed, such a division of labor would be consistent with the long-standing observation of a knowledge-action dissociation from the study of patients with frontal lobe disorders (Milner, 1964). As we did not scan participants during training, or the first mini-blocked section of the generalization phase, we cannot yet speak to the neural circuits engaged in initially forming this abstract task representation.

In sum, we have discovered evidence of a neural instantiation of a long-supposed construct of cognitive models: an abstract task representation that enables generalization to new tasks in absence of reinforcement. Future studies could evaluate how and when this representation is formed, when and if this neural representation is necessary for generalization behavior, and why some participants take longer to leverage this abstract task structure. This study thus opens new ground in understanding the neural systems supporting some of the key cognitive processes and behavioral features that distinguish species like humans in their behavioral flexibility and capacity for rapid leaning (Lake et al., 2017; Penn et al., 2008).

Complete behavioral data from all participants who completed all three sessions of this experiment, un-thresholded statistical maps for whole-brain analyses and beta coefficients for ROI-level analyses have been deposited on the project site for this experiment on the Open Science Framework.

1. 1. Vaidya AR
   2. Jones HJ
   3. Castillo J
   4. Badre D

   (2020) Open Science Framework

   ID g8yj6. Category Betting Task.

   https://osf.io/g8yj6

1. Preprint

   1. Afifi M

   (2019) 11k hands: gender recognition and biometric identification using a large dataset of hand images

   arXiv.

   https://arxiv.org/abs/1711.04322

   • Google Scholar
2. 1. Anderson JR

   (1983) A spreading activation theory of memory

   Journal of Verbal Learning and Verbal Behavior 22:261–295.

   https://doi.org/10.1016/S0022-5371(83)90201-3

   • Google Scholar
3. 1. Averbeck BB
   2. Seo M

   (2008) The statistical neuroanatomy of frontal networks in the macaque

   PLOS Computational Biology 4:e1000050.

   https://doi.org/10.1371/journal.pcbi.1000050

   • PubMed
   • Google Scholar
4. 1. Badre D
   2. Kayser AS
   3. D'Esposito M

   (2010) Frontal cortex and the discovery of abstract action rules

   Neuron 66:315–326.

   https://doi.org/10.1016/j.neuron.2010.03.025

   • PubMed
   • Google Scholar
5. 1. Badre D
   2. Nee DE

   (2018) Frontal cortex and the hierarchical control of behavior

   Trends in Cognitive Sciences 22:170–188.

   https://doi.org/10.1016/j.tics.2017.11.005

   • PubMed
   • Google Scholar
6. 1. Baldassano C
   2. Hasson U
   3. Norman KA

   (2018) Representation of Real-World event schemas during narrative perception

   The Journal of Neuroscience 38:9689–9699.

   https://doi.org/10.1523/JNEUROSCI.0251-18.2018

   • PubMed
   • Google Scholar
7. 1. Blechert J
   2. Meule A
   3. Busch NA
   4. Ohla K

   (2014) Food-pics: an image database for experimental research on eating and appetite

   Frontiers in Psychology 5:617.

   https://doi.org/10.3389/fpsyg.2014.00617

   • PubMed
   • Google Scholar
8. 1. Brodeur MB
   2. Guérard K
   3. Bouras M

   (2014) Bank of standardized stimuli (BOSS) phase II: 930 new normative photos

   PLOS ONE 9:e106953.

   https://doi.org/10.1371/journal.pone.0106953

   • PubMed
   • Google Scholar
9. 1. Chan SC
   2. Niv Y
   3. Norman KA

   (2016) A probability distribution over latent causes, in the orbitofrontal cortex

   The Journal of Neuroscience 36:7817–7828.

   https://doi.org/10.1523/JNEUROSCI.0659-16.2016

   • PubMed
   • Google Scholar
10. 1. Chen J
    2. Leong YC
    3. Honey CJ
    4. Yong CH
    5. Norman KA
    6. Hasson U

    (2017) Shared memories reveal shared structure in neural activity across individuals

    Nature Neuroscience 20:115–125.

    https://doi.org/10.1038/nn.4450

    • PubMed
    • Google Scholar
11. 1. Chikazoe J
    2. Lee DH
    3. Kriegeskorte N
    4. Anderson AK

    (2014) Population coding of affect across stimuli, modalities and individuals

    Nature Neuroscience 17:1114–1122.

    https://doi.org/10.1038/nn.3749

    • PubMed
    • Google Scholar
12. 1. Choi EY
    2. Drayna GK
    3. Badre D

    (2018) Evidence for a functional hierarchy of association networks

    Journal of Cognitive Neuroscience 30:722–736.

    https://doi.org/10.1162/jocn_a_01229

    • PubMed
    • Google Scholar
13. 1. Collins AG
    2. Cavanagh JF
    3. Frank MJ

    (2014) Human EEG uncovers latent generalizable rule structure during learning

    Journal of Neuroscience 34:4677–4685.

    https://doi.org/10.1523/JNEUROSCI.3900-13.2014

    • PubMed
    • Google Scholar
14. 1. Collins AG
    2. Frank MJ

    (2013) Cognitive control over learning: creating, clustering, and generalizing task-set structure

    Psychological Review 120:190–229.

    https://doi.org/10.1037/a0030852

    • PubMed
    • Google Scholar
15. Preprint

    1. De Araujo Sanchez MA
    2. Zeithamova D

    (2020) Generalization and source memory in acquired equivalence

    PsyArXiv.

    https://doi.org/10.31234/osf.io/txjsh

    • Google Scholar
16. 1. Devereux BJ
    2. Clarke A
    3. Marouchos A
    4. Tyler LK

    (2013) Representational similarity analysis reveals commonalities and differences in the semantic processing of words and objects

    Journal of Neuroscience 33:18906–18916.

    https://doi.org/10.1523/JNEUROSCI.3809-13.2013

    • PubMed
    • Google Scholar
17. 1. Eichenbaum A
    2. Scimeca JM
    3. D'Esposito M

    (2020) Dissociable neural systems support the learning and transfer of hierarchical control structure

    The Journal of Neuroscience 40:6624–6637.

    https://doi.org/10.1523/JNEUROSCI.0847-20.2020

    • PubMed
    • Google Scholar
18. 1. Farovik A
    2. Place RJ
    3. McKenzie S
    4. Porter B
    5. Munro CE
    6. Eichenbaum H

    (2015) Orbitofrontal cortex encodes memories within value-based schemas and represents contexts that guide memory retrieval

    Journal of Neuroscience 35:8333–8344.

    https://doi.org/10.1523/JNEUROSCI.0134-15.2015

    • PubMed
    • Google Scholar
19. 1. Frank MJ
    2. Badre D

    (2012) Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis

    Cerebral Cortex 22:509–526.

    https://doi.org/10.1093/cercor/bhr114

    • PubMed
    • Google Scholar
20. 1. Garvert MM
    2. Dolan RJ
    3. Behrens TE

    (2017) A map of abstract relational knowledge in the human hippocampal-entorhinal cortex

    eLife 6:e17086.

    https://doi.org/10.7554/eLife.17086

    • PubMed
    • Google Scholar
21. 1. Gershman SJ
    2. Jones CE
    3. Norman KA
    4. Monfils MH
    5. Niv Y

    (2013) Gradual extinction prevents the return of fear: implications for the discovery of state

    Frontiers in Behavioral Neuroscience 7:164.

    https://doi.org/10.3389/fnbeh.2013.00164

    • PubMed
    • Google Scholar
22. 1. Gershman SJ
    2. Niv Y

    (2010) Learning latent structure: carving nature at its joints

    Current Opinion in Neurobiology 20:251–256.

    https://doi.org/10.1016/j.conb.2010.02.008

    • PubMed
    • Google Scholar
23. 1. Gershman SJ
    2. Niv Y

    (2013) Perceptual estimation obeys Occam's razor

    Frontiers in Psychology 4:623.

    https://doi.org/10.3389/fpsyg.2013.00623

    • PubMed
    • Google Scholar
24. 1. Ghosh VE
    2. Moscovitch M
    3. Melo Colella B
    4. Gilboa A

    (2014) Schema representation in patients with ventromedial PFC lesions

    Journal of Neuroscience 34:12057–12070.

    https://doi.org/10.1523/JNEUROSCI.0740-14.2014

    • PubMed
    • Google Scholar
25. 1. Gick ML
    2. Holyoak KJ

    (1980) Analogical problem solving

    Cognitive Psychology 12:306–355.

    https://doi.org/10.1016/0010-0285(80)90013-4

    • Google Scholar
26. Software

    1. Greve DN

    (2002) Optseq2. Optseq Home Page

    Optseq2. Optseq Home Page.

    https://surfer.nmr.mgh.harvard.edu/optseq/
27. 1. Harlow HF

    (1949) The formation of learning sets

    Psychological Review 56:51–65.

    https://doi.org/10.1037/h0062474

    • Google Scholar
28. 1. Kahnt T
    2. Chang LJ
    3. Park SQ
    4. Heinzle J
    5. Haynes JD

    (2012) Connectivity-based parcellation of the human orbitofrontal cortex

    Journal of Neuroscience 32:6240–6250.

    https://doi.org/10.1523/JNEUROSCI.0257-12.2012

    • PubMed
    • Google Scholar
29. 1. Koechlin E
    2. Summerfield C

    (2007) An information theoretical approach to prefrontal executive function

    Trends in Cognitive Sciences 11:229–235.

    https://doi.org/10.1016/j.tics.2007.04.005

    • PubMed
    • Google Scholar
30. 1. Konkle T
    2. Brady TF
    3. Alvarez GA
    4. Oliva A

    (2010) Scene memory is more detailed than you think: the role of categories in visual long-term memory

    Psychological Science 21:1551–1556.

    https://doi.org/10.1177/0956797610385359

    • PubMed
    • Google Scholar
31. 1. Kuhl BA
    2. Rissman J
    3. Wagner AD

    (2012) Multi-voxel patterns of visual category representation during episodic encoding are predictive of subsequent memory

    Neuropsychologia 50:458–469.

    https://doi.org/10.1016/j.neuropsychologia.2011.09.002

    • PubMed
    • Google Scholar
32. Conference

    1. Kumar N
    2. Belhumeur PN
    3. Biswas A
    4. Jacobs DW
    5. Kress WJ
    6. Lopez IC
    7. Soares JVB

    (2012) Leafsnap: a computer vision system for automatic plant species identificationLecture notes in computer science (Including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics)

    European Conference on Computer Vision.

    https://doi.org/10.1007/978-3-642-33709-3_36

    • Google Scholar
33. 1. Kumaran D
    2. Summerfield JJ
    3. Hassabis D
    4. Maguire EA

    (2009) Tracking the emergence of conceptual knowledge during human decision making

    Neuron 63:889–901.

    https://doi.org/10.1016/j.neuron.2009.07.030

    • PubMed
    • Google Scholar
34. 1. Kumaran D
    2. McClelland JL

    (2012) Generalization through the recurrent interaction of episodic memories: a model of the hippocampal system

    Psychological Review 119:573–616.

    https://doi.org/10.1037/a0028681

    • PubMed
    • Google Scholar
35. 1. Lake BM
    2. Ullman TD
    3. Tenenbaum JB
    4. Gershman SJ

    (2017) Building machines that learn and think like people

    Behavioral and Brain Sciences 40:e253.

    https://doi.org/10.1017/S0140525X16001837

    • Google Scholar
36. 1. Liu Y
    2. Dolan RJ
    3. Kurth-Nelson Z
    4. Behrens TEJ

    (2019) Human replay spontaneously reorganizes experience

    Cell 178:640–652.

    https://doi.org/10.1016/j.cell.2019.06.012

    • PubMed
    • Google Scholar
37. 1. Loose LS
    2. Wisniewski D
    3. Rusconi M
    4. Goschke T
    5. Haynes JD

    (2017) Switch-Independent task representations in frontal and parietal cortex

    The Journal of Neuroscience 37:8033–8042.

    https://doi.org/10.1523/JNEUROSCI.3656-16.2017

    • PubMed
    • Google Scholar
38. 1. Ma DS
    2. Correll J
    3. Wittenbrink B

    (2015) The Chicago face database: a free stimulus set of faces and norming data

    Behavior Research Methods 47:1122–1135.

    https://doi.org/10.3758/s13428-014-0532-5

    • PubMed
    • Google Scholar
39. 1. Margulies DS
    2. Vincent JL
    3. Kelly C
    4. Lohmann G
    5. Uddin LQ
    6. Biswal BB
    7. Villringer A
    8. Castellanos FX
    9. Milham MP
    10. Petrides M

    (2009) Precuneus shares intrinsic functional architecture in humans and monkeys

    PNAS 106:20069–20074.

    https://doi.org/10.1073/pnas.0905314106

    • Google Scholar
40. 1. McKenzie S
    2. Frank AJ
    3. Kinsky NR
    4. Porter B
    5. Rivière PD
    6. Eichenbaum H

    (2014) Hippocampal representation of related and opposing memories develop within distinct, hierarchically organized neural schemas

    Neuron 83:202–215.

    https://doi.org/10.1016/j.neuron.2014.05.019

    • PubMed
    • Google Scholar
41. 1. McNamee D
    2. Rangel A
    3. O'Doherty JP

    (2013) Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex

    Nature Neuroscience 16:479–485.

    https://doi.org/10.1038/nn.3337

    • PubMed
    • Google Scholar
42. 1. Milner B

    (1964) Some effects of frontal lobectomy in man

    The Frontal Granular Cortex and Behavior 298:211–226.

    https://doi.org/10.1098/rstb.1982.0083

    • Google Scholar
43. 1. Nassar MR
    2. McGuire JT
    3. Ritz H
    4. Kable JW

    (2019) Dissociable forms of uncertainty-driven representational change across the human brain

    The Journal of Neuroscience 39:1688–1698.

    https://doi.org/10.1523/JNEUROSCI.1713-18.2018

    • PubMed
    • Google Scholar
44. 1. Nichols TE
    2. Holmes AP

    (2002) Nonparametric permutation tests for functional neuroimaging: a primer with examples

    Human Brain Mapping 15:1–25.

    https://doi.org/10.1002/hbm.1058

    • PubMed
    • Google Scholar
45. 1. Nili H
    2. Wingfield C
    3. Walther A
    4. Su L
    5. Marslen-Wilson W
    6. Kriegeskorte N

    (2014) A toolbox for representational similarity analysis

    PLOS Computational Biology 10:e1003553.

    https://doi.org/10.1371/journal.pcbi.1003553

    • PubMed
    • Google Scholar
46. 1. Niv Y

    (2019) Learning task-state representations

    Nature Neuroscience 22:1544–1553.

    https://doi.org/10.1038/s41593-019-0470-8

    • PubMed
    • Google Scholar
47. 1. Pegors TK
    2. Kable JW
    3. Chatterjee A
    4. Epstein RA

    (2015) Common and unique representations in pFC for face and place attractiveness

    Journal of Cognitive Neuroscience 27:959–973.

    https://doi.org/10.1162/jocn_a_00777

    • PubMed
    • Google Scholar
48. 1. Penn DC
    2. Holyoak KJ
    3. Povinelli DJ

    (2008) Darwin's mistake: explaining the discontinuity between human and nonhuman minds

    Behavioral and Brain Sciences 31:109–130.

    https://doi.org/10.1017/S0140525X08003543

    • PubMed
    • Google Scholar
49. 1. Petrides M

    (2005) Lateral prefrontal cortex: architectonic and functional organization

    Philosophical Transactions of the Royal Society B: Biological Sciences 360:781–795.

    https://doi.org/10.1098/rstb.2005.1631

    • Google Scholar
50. Book

    1. Rescorla RA
    2. Wagner AR

    (1972)

    A theory of Pavlovian conditioning: Variations in the effective- ness of reinforcement and non-reinforcement

    In: Black A. H, Prokasy W. F, editors. Classical Conditioning. II. Current Research and Theory. Appleton-Century-Crofts. pp. 64–99.

    • Google Scholar
51. 1. Rolls ET
    2. Joliot M
    3. Tzourio-Mazoyer N

    (2015) Implementation of a new parcellation of the orbitofrontal cortex in the automated anatomical labeling atlas

    NeuroImage 122:1–5.

    https://doi.org/10.1016/j.neuroimage.2015.07.075

    • PubMed
    • Google Scholar
52. 1. Russin J
    2. O’Reilly RC
    3. Bengio Y

    (2020) Deep learning needs a prefrontal cortex

    Work Bridging AI Cogn Sci 107:603–616.

    https://doi.org/10.1016/j.neuron.2020.06.014

    • Google Scholar
53. 1. Russo N
    2. Hagmann CE
    3. Andrews R
    4. Black C
    5. Silberman M
    6. Shea N

    (2018) Validation of the C.A.R.E. stimulus set of 640 animal pictures: name agreement and quality ratings

    PLOS ONE 13:e0192906.

    https://doi.org/10.1371/journal.pone.0192906

    • PubMed
    • Google Scholar
54. 1. Saez A
    2. Rigotti M
    3. Ostojic S
    4. Fusi S
    5. Salzman CD

    (2015) Abstract Context Representations in Primate Amygdala and Prefrontal Cortex

    Neuron 87:869–881.

    https://doi.org/10.1016/j.neuron.2015.07.024

    • Google Scholar
55. 1. Schapiro AC
    2. Turk-Browne NB
    3. Botvinick MM
    4. Norman KA

    (2017) Complementary learning systems within the Hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning

    Philosophical Transactions of the Royal Society B: Biological Sciences 372:20160049.

    https://doi.org/10.1098/rstb.2016.0049

    • Google Scholar
56. 1. Schlichting ML
    2. Preston AR

    (2015) Memory integration: neural mechanisms and implications for behavior

    Current Opinion in Behavioral Sciences 1:1–8.

    https://doi.org/10.1016/j.cobeha.2014.07.005

    • PubMed
    • Google Scholar
57. 1. Schuck NW
    2. Cai MB
    3. Wilson RC
    4. Niv Y

    (2016) Human orbitofrontal cortex represents a cognitive map of state space

    Neuron 91:1402–1412.

    https://doi.org/10.1016/j.neuron.2016.08.019

    • PubMed
    • Google Scholar
58. 1. Schuck NW
    2. Niv Y

    (2019) Sequential replay of nonspatial task states in the human Hippocampus

    Science 364:eaaw5181.

    https://doi.org/10.1126/science.aaw5181

    • PubMed
    • Google Scholar
59. 1. Shohamy D
    2. Wagner AD

    (2008) Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events

    Neuron 60:378–389.

    https://doi.org/10.1016/j.neuron.2008.09.023

    • Google Scholar
60. 1. Spalding KN
    2. Schlichting ML
    3. Zeithamova D
    4. Preston AR
    5. Tranel D
    6. Duff MC
    7. Warren DE

    (2018) Ventromedial prefrontal cortex is necessary for normal associative inference and memory integration

    The Journal of Neuroscience 38:3767–3775.

    https://doi.org/10.1523/JNEUROSCI.2501-17.2018

    • PubMed
    • Google Scholar
61. 1. Stalnaker TA
    2. Cooch NK
    3. Schoenbaum G

    (2015) What the orbitofrontal cortex does not do

    Nature Neuroscience 18:620–627.

    https://doi.org/10.1038/nn.3982

    • PubMed
    • Google Scholar
62. Book

    1. Sutton R
    2. Barto A

    (2018)

    Reinforcment Learning

    MIT Press.

    • Google Scholar
63. 1. Tavares RM
    2. Mendelsohn A
    3. Grossman Y
    4. Williams CH
    5. Shapiro M
    6. Trope Y
    7. Schiller D

    (2015) A map for social navigation in the human brain

    Neuron 87:231–243.

    https://doi.org/10.1016/j.neuron.2015.06.011

    • PubMed
    • Google Scholar
64. 1. Tenenbaum JB
    2. Kemp C
    3. Griffiths TL
    4. Goodman ND

    (2011) How to grow a mind: statistics, structure, and abstraction

    Science 331:1279–1285.

    https://doi.org/10.1126/science.1192788

    • PubMed
    • Google Scholar
65. 1. Thorat S
    2. Proklova D
    3. Peelen MV

    (2019) The nature of the animacy organization in human ventral temporal cortex

    eLife 8:e47142.

    https://doi.org/10.7554/eLife.47142

    • PubMed
    • Google Scholar
66. 1. Tolman EC

    (1948) Cognitive maps in rats and men

    Psychological Review 55:189–208.

    https://doi.org/10.1037/h0061626

    • PubMed
    • Google Scholar
67. 1. Tomov MS
    2. Dorfman HM
    3. Gershman SJ

    (2018) Neural computations underlying causal structure learning

    The Journal of Neuroscience 38:7143–7157.

    https://doi.org/10.1523/JNEUROSCI.3336-17.2018

    • PubMed
    • Google Scholar
68. 1. van Kesteren MT
    2. Beul SF
    3. Takashima A
    4. Henson RN
    5. Ruiter DJ
    6. Fernández G

    (2013) Differential roles for medial prefrontal and medial temporal cortices in schema-dependent encoding: from congruent to incongruent

    Neuropsychologia 51:2352–2359.

    https://doi.org/10.1016/j.neuropsychologia.2013.05.027

    • PubMed
    • Google Scholar
69. 1. Walther A
    2. Nili H
    3. Ejaz N
    4. Alink A
    5. Kriegeskorte N
    6. Diedrichsen J

    (2016) Reliability of dissimilarity measures for multi-voxel pattern analysis

    NeuroImage 137:188–200.

    https://doi.org/10.1016/j.neuroimage.2015.12.012

    • PubMed
    • Google Scholar
70. 1. Wilson RC
    2. Takahashi YK
    3. Schoenbaum G
    4. Niv Y

    (2014) Orbitofrontal Cortex as a Cognitive Map of Task Space

    Neuron 81:267–279.

    https://doi.org/10.1016/j.neuron.2013.11.005

    • Google Scholar
71. 1. Woolgar A
    2. Thompson R
    3. Bor D
    4. Duncan J

    (2011) Multi-voxel coding of stimuli, rules, and responses in human frontoparietal cortex

    NeuroImage 56:744–752.

    https://doi.org/10.1016/j.neuroimage.2010.04.035

    • PubMed
    • Google Scholar
72. 1. Yeo BT
    2. Krienen FM
    3. Sepulcre J
    4. Sabuncu MR
    5. Lashkari D
    6. Hollinshead M
    7. Roffman JL
    8. Smoller JW
    9. Zöllei L
    10. Polimeni JR
    11. Fischl B
    12. Liu H
    13. Buckner RL

    (2011) The organization of the human cerebral cortex estimated by intrinsic functional connectivity

    Journal of Neurophysiology 106:1125–1165.

    https://doi.org/10.1152/jn.00338.2011

    • PubMed
    • Google Scholar
73. 1. Zeithamova D
    2. Dominick AL
    3. Preston AR

    (2012a) Hippocampal and ventral medial prefrontal activation during retrieval-mediated learning supports novel inference

    Neuron 75:168–179.

    https://doi.org/10.1016/j.neuron.2012.05.010

    • PubMed
    • Google Scholar
74. 1. Zeithamova D
    2. Schlichting ML
    3. Preston AR

    (2012b) The Hippocampus and inferential reasoning: building memories to navigate future decisions

    Frontiers in Human Neuroscience 6:70.

    https://doi.org/10.3389/fnhum.2012.00070

    • PubMed
    • Google Scholar
75. 1. Zhou J
    2. Montesinos-Cartagena M
    3. Wikenheiser AM
    4. Gardner MPH
    5. Niv Y
    6. Schoenbaum G

    (2019) Complementary task structure representations in Hippocampus and orbitofrontal cortex during an odor sequence task

    Current Biology 29:3402–3409.

    https://doi.org/10.1016/j.cub.2019.08.040

    • PubMed
    • Google Scholar

Acceptance summary:

Combining a clever experimental design with rigorous computational cognitive modeling and neuroimaging, the researchers provide evidence suggesting that latent state structures are encoded by a network of fronto-parietal regions. The bridge between abstract reasoning and adaptive decision-making is an important contribution.

Decision letter after peer review:

Thank you for submitting your article "Neural representation of abstract task structure during generalization" for consideration by eLife. Your article has been reviewed by Richard Ivry as the Senior Editor, Mimi Liljeholm as Reviewing Editor, and two reviewers. The following individuals involved in review of your submission have agreed to reveal their identity: Charan Ranganath (Reviewer #1); Sebastian Michelmann (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

We would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). Specifically, when editors judge that a submitted work as a whole belongs in eLife but that some conclusions require a modest amount of additional new data, as they do with your paper, we are asking that the manuscript be revised to either limit claims to those supported by data in hand, or to explicitly state that the relevant conclusions require additional supporting data.

Our expectation is that the authors will eventually carry out additional experiments and report on how they affect the relevant conclusions either in a preprint on bioRxiv or medRxiv, or if appropriate, as a Research Advance in eLife, either of which would be linked to the original paper.

Summary:

Vaidya and colleagues present a carefully designed study in order to investigate how the brain represents abstract task structure in a generalization task. Subjects in this study learned "latent states" (i.e., sets of statistically associated scenes) that determined the reward probabilities of objects from different categories. Later, new object categories were introduced and associated with the previously learned task states. Behaviorally, subjects were able to use their knowledge of the abstract task structure to speed new learning and inference on the new object categories. Pattern similarity analyses revealed that abstract task structure, independent from visual information, was represented in left-mid lateral prefrontal cortex, bilateral precuneus, and inferior parietal cortex.

Essential revisions:

All the reviewers thought that this study addresses an important question: Identifying how the brain represents abstract task structure in a generalization task. They agreed that this study uses a very clever behavioral design and a nice combination of behavioral and neuroscientific methods. However, several claims need to be tampered or better justified, and a much more thorough consideration of a broad, heterogeneous and highly relevant literature is warranted.

1) A major concern is that the main claim of the paper – that behavioral and neural data reflect a "latent state" – cannot be evaluated theoretically unless the nature of the latent state representation is carefully and concretely defined. This will require that the authors address the link to prior research on abstract representations of tasks and contexts more broadly, clarifying how they are similar or different in the context of your experiment in both the Introduction and Discussion.

2) As a case in point, the design appears to be one of acquired equivalence (AE) – an extensively studied paradigm in which distinct cues are paired with a common outcome, and subsequent training with one of the cues with a novel outcome generalizes to the second cue during test: that is, given A◊O1, B◊O1 followed by A◊O2, tests reveal expectations of O2 given B. The consensus explanation for such effects is the associative retrieval of O1 by A, and of B by O1, allowing for mediated conditioning of B to O2 during A◊O2 trials. While the current study employs more complex contingencies than those commonly used in studies on AE, it is not clear that some form of associative mediation and retrieval can be ruled out for these effects. The question becomes, then, whether the process or content of complex associative retrieval is what the authors mean by a "latent state representation", or whether they are instead referring to the explicit abstraction of some invariant feature, more in line with previous work on relational reasoning. Either way, the definitions of terms such as "latent state" and "abstract structure" must be clearly laid out and framed in the context of previous relevant work.

3) The authors do not show a clear link between their neural and behavioral results. Some link between these two effects may help strengthen the conclusions that the authors draw. A couple of additional analyses should assess (i) if individual differences in phase 3 accuracy or RT exponential fit relate to subjects latent state representation, (ii) if there is a relationship between switch costs (between latent states) and the fidelity of neural task representation, (iii) To further support that regions in the brain are representing latent states and tracking which state the participant is in, a univariate analysis that identifies regions that are active during latent state switches should be performed. For example, the authors could contrast activity immediately following a latent state switch vs. within a latent state (e.g. trial 1 vs. trial 4 w/in mini block). Is this response bigger when both the context and latent state switches?

4) The latent task RSA analysis revealed a network of regions that is very similar to a network of regions representing "event schemas" across individuals (e.g. Baldassano et al., 2018; Chen et al., 2017). These results should be discussed in a broader context of how the brain represents structure more generally. The supplemental analysis is nice and shows the link to prior work but is not sufficient. The authors do find evidence for "category" representations in a priori ROIs but do not really discuss what the interpretation of these results is. Why is the hippocampus and OFC representing category information during generalization? What does this mean for the field as a whole? For example, it is well established that hippocampus is essential for creating orthogonal representations that are used to disambiguate similar events. How do these results agree or disagree with this interpretation?

Essential revisions:

All the reviewers thought that this study addresses an important question: Identifying how the brain represents abstract task structure in a generalization task. They agreed that this study uses a very clever behavioral design and a nice combination of behavioral and neuroscientific methods. However, several claims need to be tampered or better justified, and a much more thorough consideration of a broad, heterogeneous and highly relevant literature is warranted.

1) A major concern is that the main claim of the paper – that behavioral and neural data reflect a "latent state" – cannot be evaluated theoretically unless the nature of the latent state representation is carefully and concretely defined. This will require that the authors address the link to prior research on abstract representations of tasks and contexts more broadly, clarifying how they are similar or different in the context of your experiment in both the Introduction and Discussion.

We agree that this is an important point to address, as a lack of precision can add confusion to the literature regarding this observation. Our revision attempts to clarify how we are using the term “latent state” in the introduction. We define “states” consistent with their use in the reinforcement learning literature: variables that describe the condition of the environment. In the case of a gambling task, for example, the shape or color of a “bandit” could define the state, in other tasks the location on a screen or in a maze might serve the same purpose. States are thus a key substrate for learning, and their formulation dictates how information about the task is shared or separated between different settings. For example, if only the color of the bandit determines the likelihood of reward, it may be useful to represent all bandits of the same color as belonging to a single state, rather than forming separate representations for each bandit by shape and color. This example is a form of generalization, or state abstraction, where information about the value of bandits of the same color is shared across locations, which would speed learning along the color dimension.

Building on this definition, we define “latent states” as states that are not defined by sensory evidence alone, but must be inferred from a distribution of outcomes or other task-relevant information over a set of task features. Thus, the concept “latent state” has been operationalized differently across studies depending on the task at hand. For example, other experiments have defined latent states based on the temporal frequency of tone-shock pairing (Gershman et al., 2013), the spatiotemporal distribution of rewards (Nassar et al., 2019), or action-values conditioned on recent task history (Zhou et al., 2019). In the case of our experiment, a latent state is defined by distribution of rewards and punishments for categories in contexts during the initial training phase. As such, latent states in the present experiment share a similarity to past work examining generalization behavior over latent task sets (Collins and Frank, 2013) or latent causes (Tomov et al., 2018).

The definition of latent state in our task aside, we believe the way latent states are used in our task might provide another account for the difference between our observations and prior reports that have highlighted a role for the hippocampus and OFC in representing latent states. In particular, our task focused on generalization in a phase where there was no external feedback or new information that would allow participants to update their representation of how contexts were related to a latent state. In contrast, other work that focused on active inference of the current latent state from available evidence has found a representation of this information in OFC (Chan et al., 2016). Similarly, other work has focused on latent state representations in OFC and hippocampus that are putatively involved in planning future actions based on recent task history (Schuck et al., 2016; Schuck and Niv, 2019). We discuss this point in the revised manuscript.

2) As a case in point, the design appears to be one of acquired equivalence (AE) – an extensively studied paradigm in which distinct cues are paired with a common outcome, and subsequent training with one of the cues with a novel outcome generalizes to the second cue during test: that is, given A◊O1, B◊O1 followed by A◊O2, tests reveal expectations of O2 given B. The consensus explanation for such effects is the associative retrieval of O1 by A, and of B by O1, allowing for mediated conditioning of B to O2 during A◊O2 trials. While the current study employs more complex contingencies than those commonly used in studies on AE, it is not clear that some form of associative mediation and retrieval can be ruled out for these effects. The question becomes, then, whether the process or content of complex associative retrieval is what the authors mean by a "latent state representation", or whether they are instead referring to the explicit abstraction of some invariant feature, more in line with previous work on relational reasoning. Either way, the definitions of terms such as "latent state" and "abstract structure" must be clearly laid out and framed in the context of previous relevant work.

This is an insightful point that raises the question: Do participants integrate contexts into a representation of a latent task state to help complete the generalization task, or do they generalize through a process of associative retrieval? We believe this is important to resolve because a mediated associative retrieval account would suggest that an integrated task representation is not used to carry out generalization, and it would have different implications for the interpretation of our functional imaging results. To distinguish between these alternatives, we conducted a computational modeling analysis to formalize these hypotheses and compare how well they account for participants’ reaction time data in the first mini-blocked section of the generalization phase.

We developed a set of simple learning models to compare and contrast different explanations for how participants carried out the transfer of option values during the initial generalization phase. The models express retrieval and generalization in terms of spreading activation among a set of nodes that represent the various task elements. In each model, we structured the memory network differently according to the hypothesis tested. However, these models relied on the same basic assumptions and parameters: (1) Reaction times are a function of the activation strength of the nodes containing retrieved items (Anderson, 1983). (2) The activation strength for each item is initialized at zero and increases according to a Rescorla-Wagner learning rule (Rescorla and Wagner, 1972), controlled by a single rate parameter bounded by 0 to 1 (α₁), each time an option is retrieved, until it reaches an asymptotic value of one. (3) Activation spreads to contexts and categories linked with retrieved items, controlled by a ‘mediated retrieval’ parameter (α₂). (4) Memory search increases activation in all other task-related items in each trial according to an ‘incidental retrieval’ parameter (α₃). (5) Increases in activation that are inherited from connected nodes falls off according to a power law where the rates of successive steps are multiplied together.

We tested four model variants that each differed in how they represented this memory network, and how activation of task elements propagates during generalization (see Figure 4). Importantly, these models are not meant simulate any particular neural system, but only describe how the format of these representations in memory might differentially affect the retrieval process and, by extension, reaction times:

1) Conjunctive associative retrieval – This model implements generalization through associative retrieval by assuming that the memory system stores conjunctions of task-relevant features which can be retrieved if any one of these features is later encountered. This memory network was represented as a matrix of contexts-by-category conjunctions. On each generalization trial, the presented context (e.g. A1) would activate related context-category-value conjunctions from the initial training based on the rate parameter α₁ (e.g. A1-leaves- (negative value), A1-food-, A1-hands+ (positive value)). This would lead to the mediated activation of context-category conjunctions with the same value associations from initial training with the rate parameter α₂ (e.g. A3-hands+, A2-hands+, etc.), which would in turn lead to activation of conjunctions of new categories associated with the context that appeared in the second training phase (e.g. A3-objects-, A3-Faces+ etc.), as determined by the rate of α₂². At the same time, the category presented on the current trial (e.g. objects) would activate conjunctions with that category from the new category training phase (e.g. A3-objects-, B3-objects+) at a rate determined by α₁. The conjunction shown on the current trial (e.g. A1-objects-) would inherit activation from the related conjunction from the new category training phase (e.g. A3-objects-) and increase in activation at a rate determined by α₁²*α₂². Similarly, the same category conjunction with the held out related context (e.g. A2-objects-) would receive a mediated increase in activation at a rate of α₁^2*α₂³. All other nodes not activated by the current trial would also increase their activation as a rate of α₃.

2) Independent associative retrieval – This model also implements generalization through associative retrieval, but rather than relying on episodic conjunctions of context and category, it stores each individual task element independently and links them together based on experience. The memory network was represented as two vectors of contexts and category-value conjunctions (i.e. separate nodes for faces+ faces-, etc.). Each context was linked to their associated category-values from the initial training phase, and only the held-out contexts were directly linked to category-values from the new category training. In each trial (e.g. A1-objects-), activation in the current context (e.g. A1) would increase with the rate parameter α₁. This activation would result in direct retrieval of category-values from initial training at a rate of α₁², and direct retrieval of the context from the new category training phase with the same category-value associations (e.g. A3) at the rate α₁³. Retrieval of this context then led to activation of the relevant category-value node for the current trial (e.g. objects-) from the new training phase at a rate of α₁⁴, allowing generalization, as well as mediated retrieval of other category-values (e.g. faces+, animals+) at a rate of α₁³*α₂. Activation of category values from the initial training would also lead to mediated retrieval of the other related context (e.g. A2) at a rate of α₁²*α₂. All other nodes would increase their activation with a rate set by α₃.

3) Latent state – This model implements generalization through the formation of a latent state that replaces separate representations of individual contexts. Here the memory network was represented as a vector of latent states and a matrix of latent states-by-categories, so category-values were separately nested in each latent state. In this case, there was no separate representation of a context, so any increase in activation from a trial involving context A1 immediately carried over to context A2. On each trial, activation in the current latent state (e.g. latent state A) would increase according to the rate parameter α₁, and in the current category-value node according to α₁². All other categories nodes linked to the active latent state increase in activation according to α₂, and all other latent states increase activation according to α₃, and their nested categories according to α₃².

4) Hierarchical latent states – This model also implements generalization through a latent state, but builds on the basic latent state model to hierarchically cluster contexts within latent states. This model was equivalent to the structure assumed a priori in our original submission (see schematic in Figure 1). Thus, this model builds on the latent state model but includes a distinct vector for contexts, as well as a matrix of contexts-by-categories. These contexts and context-category nodes are hierarchically nested within each latent state (e.g. contexts A1 and A2 are nested within latent state A). Thus, activation for each of these contexts and category nodes is inherited from activation of the latent states and latent state-category nodes. On each trial, activation in the current context increases at a rate of α₁², and in the current context-category at α₁³. Contexts linked to the current latent state also increase in activation according to α₁*α₂, as these contexts inherit mediated activation from the latent state. Categories nested in these contexts increase activation according to the rate α₁*α₂². Activation in all other latent states increases at a rate of α₃, contexts not linked to the current latent state increase at a rate of α₃², and their nested categories according to α₃³.

For each model, predicted reaction times were calculated simply as the sum of the activation level in the current context (or latent state in case of the third model) and category, subtracted from the maximum asymptotic activation level for both combined. These values were then z-scored and compared to participants’ z-scored reaction times so that both model predictions and behavioral data were on the same scale. The fit of each of these models was calculated as the negative log-likelihood for a simple linear function. Only correct responses were included in this analysis, as the model assumes correct recovery of option values from memory on each trial. Across all 16 participants in the fMRI group, only 17 trials were errors in session 2, and thus accounted for a very small fraction of the data (0.6%). Parameters were optimized using the MATLAB function fmincon, with each model fit for each participant 30 times with random starting points in order to avoid convergence on local minima.

To test whether these models made substantially different reaction time predictions, we carried out a cross-model comparison using simulated data. We simulated data using the optimized parameters from each participant for each model. We then fit each model on these simulated data, in the same way as with participants’ behavioral data, to test if the generative model better fit the simulated data than the two alternatives (Figure 4 —figure supplement 1D). This analysis confirmed that these models made distinct predictions with little confusion between models.

To briefly summarize the results, the hierarchical latent state model provided the best fit to the reaction time data for a large majority of fMRI participants in sessions 1 (13/16) and 2 (12/16), as well as all of the behavioral participants who passed the generalization criterion in session 2 (N = 9; Table 1). We did not examine session 1 of the behavioral group, as they failed the generalization criterion in this session — and our models assumed retrieval of the correct response. We also did not examine data from session 3, as participants had already completed the same generalization task in session 2 and this session served only as a reminder.

Examining the data of fMRI participants in session 2 more closely, we found that, compared to the hierarchical latent state model, the associative retrieval models overestimated RT on category switch trials, likely because category-values associated with the current context only increased activation through mediated retrieval in these two models. In contrast, in the latent state models, these nodes inherited activation from direct retrieval of the latent state. However, the (non-hierarchical) latent state model primarily underestimated reaction times on the presentation of a second context, as this model did not have an independent representation of context from latent state and carried over this activation from the previously presented context. Thus, the hierarchical latent state model that maintained separate representations of contexts clustered around each latent state provided the best fit to these data as it was able to capture both category shifts and responses to the second context in line with participant behavior (see Figure 4—figure supplement 1).

In the revised manuscript, we describe the results of this modeling analysis in the Results section and a description of the analysis and model fitting in the Materials and methods section. We believe that this analysis makes an important new contribution by clarifying the nature of latent state generalization and explicitly formalizing and testing how different formats for this representation would influence behavior.

We believe that these results support our claim that participants used a latent state representation to complete this generalization task. However, we cannot speak to whether participants form such a representation obligatorily to complete acquired equivalence tasks in general, or if this finding reflects specific characteristics of our task. In particular, task instructions emphasizing the existence of “goblin kingdoms” (i.e. latent states) to which contexts were linked, and the mini-blocked structure of the generalization task and training (where categories were presented sequentially in each context), may have encouraged the use of such a hierarchically organized latent state representation. The kind of representation used to complete acquired equivalence tasks remains a point of some debate, and may depend on task demands (De Araujo Sanchez and Zeithamova, 2020; Shohamy and Wagner, 2008). We now raise these points in the Discussion.

It is also worth noting that this modeling analysis focuses on the mini-blocked generalization phase, not the scanned mixed generalization phase that was the focus of our fMRI analyses. In principle, it would be possible for participants to use a different representation to complete this task after having completed the initial generalized (e.g. memorizing the values of the 18 new context-category combinations, or just the six items with negative expected values). However, we found participants had RT switch costs for latent states that exceeded those switch costs for context alone in this phase of the experiment. These data thus argue that participants continued to refer to this latent state representation throughout the scanned generalization phase. We discuss this point in the Discussion section.

3) The authors do not show a clear link between their neural and behavioral results. Some link between these two effects may help strengthen the conclusions that the authors draw. A couple of additional analyses should assess (i) if individual differences in phase 3 accuracy or RT exponential fit relate to subjects latent state representation, (ii) if there is a relationship between switch costs (between latent states) and the fidelity of neural task representation, (iii) To further support that regions in the brain are representing latent states and tracking which state the participant is in, a univariate analysis that identifies regions that are active during latent state switches should be performed. For example, the authors could contrast activity immediately following a latent state switch vs. within a latent state (e.g. trial 1 vs. trial 4 w/in mini block). Is this response bigger when both the context and latent state switches?

We agree that a clear link between the neural evidence for a latent task state representation and behavior would be helpful in clarifying the role of this representation in this task. However, our experiment was not designed for the purpose of detecting such a relationship, and is ill-suited for testing these possibilities, as we explain below. We have pursued each line of analysis suggested by the reviewers, but did not find any result that we felt confident including in the revised manuscript. Nonetheless, we believe that the main results of our manuscript demonstrating evidence of a latent state representation that is generative and demonstrably abstract during generalization are important and worth reporting. We believe that the question of how this representation impacts behavior will ultimately require more work and future experiments designed to answer this question.

Here we address each of these suggested analyses in turn below:

i) If individual differences in phase 3 accuracy or RT exponential fit relate to subjects latent state representation

We took a dense-sampling approach in this experiment, collecting two sessions of fMRI data for a relatively small number of participants (N=16). We expect that we would thus be underpowered to detect any effect driven by between-subjects variance, and would be skeptical of any correlation across these 16 participants in neural and behavioral data.

However, for exploratory purposes, we extracted β coefficients for the latent task state representation from an ROI defined by voxels that exceeded the permutation-based cluster-correction threshold for statistical significance at P < 0.05 (Figure 5). There was a moderate, but not statistically significant, negative correlation between this neural measure and the rate of the exponential function fit to participants’ accuracy (ρ = -0.347, P = 0.18, Spearman correlation), and no relationship between the mean exponential rate for reaction times for the first context in each latent state (r = -0.08, P = 0.77, Pearson correlation). Please see Author response image 1 (left, reaction time (RT) rate; right, accuracy (ACC) rate):

Author response image 1

Download asset Open asset

Visual inspection of the data is suggestive of a trend towards a negative correlation for both the RT and accuracy rates, perhaps offset by one or two participants. However, given the small sample size, we are not confident enough in these results to present them in the manuscript.

ii) If there is a relationship between switch costs (between latent states) and the fidelity of neural task representation

We agree that this is an interesting and important question. However, we used a rapid event-related design that was optimized for the estimation of efficiency of the 18 context-category combinations that were the focus of our main RSA analysis. We did not optimize this design for the estimation of single-trials. Moreover, estimates of neural activity for single-trials in a rapid event-related design are heavily contaminated by adjacent trials (Mumford et al., 2012), which is especially problematic for this question where we would be interested in how the strength of a representation on a previous trial influenced reaction times on the next. In particular, we would expect that switching would be harder, and hence RTs longer, when the representation of the latent state was stronger on the previous trial (similar to an analysis of EEG data by Kikumoto and Mayr, 2020).

These points notwithstanding, we set out to examine if this test was feasible by carrying out a single-trial RSA analysis. We estimated a GLM for each trial in each participant, using the approach of Mumford et al., (2012), where a single regressor is created for the trial of interest in each GLM, and all other trials are modeled with regressors grouped by condition. We then extracted data from voxels in an ROI defined by the whole-brain RSA analysis (as above), and calculated cross-validated Mahalanobis distances between patterns on each trial and the average patterns for each of the 18 conditions.

Given that we did not optimize the design to obtain these effects, we were particularly concerned about backwards contamination of pattern activity on switch trials, such that the hemodynamic response on the trial where the latent state switched (trial n+0) might affect estimates of activity in the preceding trial (trial n-1). This would make it challenging to obtain an estimate of the strength of the latent state from pattern activity on trial n-1 that is not biased by trial n+0. To test this, we compared the cross-validated Mahalanobis distances of trial n-1 to the latent task state in trial n+0 for short and long inter-trial intervals (ITIs) – with the expectation that backward contamination would be worse for shorter ITIs. The analysis bore out this prediction, as this distance was significantly smaller on trials where the ITI was less than the median ITI (t = 2.30, P = 0.036, within-subject t-test). See Author response image 2 showing these distances for all 16 participants, each plotted as a single line. Having confirmed backward contamination from the hemodynamic response in the subsequent trial we would be very hesitant to draw conclusions from any analysis testing how the strength of pattern activity on trial n-1 influenced behavior on trial n. Similarly, forward contamination from the previous trial would certainly impact our ability to draw conclusions about how pattern activity on trial n+0 affects behavior on these switch trials.

Author response image 2

Download asset Open asset

iii) To further support that regions in the brain are representing latent states and tracking which state the participant is in, a univariate analysis that identifies regions that are active during latent state switches should be performed. For example, the authors could contrast activity immediately following a latent state switch vs. within a latent state (e.g. trial 1 vs. trial 4 w/in mini block). Is this response bigger when both the context and latent state switches?

We believe that this suggestion may have partially arisen from a misunderstanding of our experimental design. Conditions in the scanned portion of the generalization phase were presented in a pseudo-randomized order to optimize efficiency for a rapid event-related design, rather than presented in mini-blocks with several trials of the same consecutive type repeated in a row. Participants completed a mini-blocked segment of the generalization phase (similar to the training phases) prior to entering the scanner (this data is presented in Figure 2A-D and Figure 3). Please also see Figure 1—figure supplement 1 for further detail about our study design.

However, this did not prevent us from carrying out the suggested univariate contrast for switch costs related to the latent state. We ran a new GLM with condition regressors for repetitions and switches of category, context and latent state. Contrasting context switch trials where the latent state remained the same or switched, there were no voxels that survived correction for multiple comparisons or passed the peak threshold of P < 0.001, uncorrected.

While it may be somewhat puzzling that we do not observe a neural effect despite finding an effect of latent state switches in our behavioral analysis, it is possible that we are somewhat underpowered for this analysis. Again, we did not design trial ordering in the magnet to optimize efficiency to estimating switch vs. repeat differences. Thus, the latent state only remained the same in approximately 1/3 of trials, making switches far more common and expected —likely mitigating any neural switch cost effect.

In sum, the results of these posthoc analyses did not yield definitive results. However, in the pre-registration of our experimental design and analysis plan we focused on testing for a latent state representation based on an RSA of task conditions and did not power our experiment for individual differences analyses, or univariate contrasts between latent state repeat and switch conditions. We believe our main results still stand without a direct link between behavior and neural data in the scanner, and are an important contribution to understanding the network representing task structure for the purpose of generalization. Moreover, the computational modeling analyses described above that were prompted by the reviewer comments have deepened our understanding of how this latent state representation may be organized and relate to behavior. In the discussion of the revised manuscript, we note the need to draw a more direct connection between this neural representation and behavior, and the importance of addressing this question in future work in the Discussion section.

4) The latent task RSA analysis revealed a network of regions that is very similar to a network of regions representing "event schemas" across individuals (e.g. Baldassano et al., 2018; Chen et al., 2017). These results should be discussed in a broader context of how the brain represents structure more generally. The supplemental analysis is nice and shows the link to prior work but is not sufficient.

We appreciate and agree with the need for a broader comparison of our results to other data in the literature examining representations of task and event structure. To address the specific point regarding the correspondence of these networks, we sought to more formally characterize the overlap of our latent state RSA network with these prior results. A copy of the Pearson correlation coefficient map for between-participant pattern similarity during spoken recall from Chen et al., (2017) was graciously provided to us by the first author (Figure 3B in that paper). Comparison of the overlap of this statistical map with the statistical map from our RSA analysis revealed some overlap in the posterior medial cortex and inferior parietal lobule, but very limited overlap in lateral PFC or medial PFC.

Author response image 3 shows the latent state t-statistic map thresholded at P < 0.001, uncorrected in blue-green in order to be inclusive, and the Pearson correlation map thresholded at r ≥ 0.03 in red-yellow. Numbers above slices correspond to z-coordinate of slices in MNI space.

Author response image 3

Download asset Open asset

While interesting, we think the overlap of these networks is only moderate at most. The region of the posterior medial cortex with the maximum t-statistic in our sample was also fairly posterior and dorsal to that found by Chen et al. These regions of posterior medial cortex differ in their anatomical and functional connectivity with other parts of the brain: the more dorsal peak found in our analysis corresponding closer to a division of the precuneus with projections to rostrolateral PFC, and the peak from Chen et al. and corresponding closer to posterior cingulate cortex — with more connections in medial PFC (Margulies et al., 2009), which may relate to differences in the pattern of overlap in PFC subregions. The event schema results of Baldassano et al., 2018 appear to be similar to those of Chen et al., with results in posterior medial cortex being even more constrained to the posterior cingulate cortex (Figure 2 of that paper).In the revision, we discuss correspondence of our results to these data and others in the Discussion section. In particular, we believe that the regions implicated in this study correspond more closely to frontoparietal networks implicated in cognitive control and abstract stimulus invariant representations of tasks (also supported by our comparison to resting state networks from Thomas Yeo et al., 2011 in Figure 5—figure supplement 1). Networks identified as representing abstract structure related to schematic world knowledge, like those in the papers referenced by the reviewers, may be more involved in processing the congruity between events or items and a schema — while those implicated in the current experiment may be more involved in using this schema-level knowledge to control action selection and decision-making. We do not think these functions are necessarily completely segregated though. Overlap within these networks and anatomical connections bridging the two may prove to be important lines of communication between networks allowing schematic knowledge to be flexibly updated with new experience, and maintained for the purpose of control, during the performance of a task.

The authors do find evidence for "category" representations in a priori ROIs but do not really discuss what the interpretation of these results is. Why is the hippocampus and OFC representing category information during generalization? What does this mean for the field as a whole? For example, it is well established that hippocampus is essential for creating orthogonal representations that are used to disambiguate similar events. How do these results agree or disagree with this interpretation?

We are hesitant to draw strong conclusions from the finding of category information in hippocampal or OFC pattern activity. We do not believe that these results are surprising or particularly revealing about the function of these regions. OFC receives inputs from higher-order visual areas in inferior temporal cortex (Price, 2007), and has neurons tuned to stimuli like faces (Rolls et al., 2006). Past work examining pattern activity in OFC has found evidence for an animacy signal (Chikazoe et al., 2014), like our category regressor, and other work has found that stimulus categories like faces versus scenes, or food versus objects, can be decoded from OFC activity (McNamee et al., 2013; Pegors et al., 2015). We might speculate that the function of this category representation in OFC during our task relates to calculating the value of a decision based on the current context and/or latent state.

The finding of category information in hippocampus may be a bit more surprising for the reasons mentioned by the reviewers, but on consideration we do not think this contradicts conventional ideas about the function of the structure. Human hippocampal neurons have been observed to have selective responses to categories like faces and objects (Kreiman et al., 2000), and pattern activity in hippocampus carries information about visual categories that distinguishes stimuli like faces and scenes (Kuhl et al., 2012). Our results are consistent with these observations. We also do not think that our results are necessarily instructive as evidence for or against the claims about a hippocampal role in orthogonalizing events, per se, as the task did not place particular demands on separating events from each other. In general, the task demanded that participants pool information over multiple events and trial-unique stimuli to infer the values of each stimulus category in each latent state. Thus, while it would be helpful to have orthogonal representations of stimulus category, it would not be helpful to have distinct representations of each event in this task. It is possible, rather, that the hippocampus is using this category information in service of retrieving option values, but this is a speculative conclusion.

We have revised the portion of the Results where we report these effects in order to place these results in this context.

Author details

1. Avinash R Vaidya

   Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, United States

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   avinash_vaidya@brown.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-5423-5655

2. Henry M Jones

   1. Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, United States
   2. Department of Psychology, Stanford University, Stanford, Stanford, United States

   Contribution

   Software, Investigation, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

3. Johanny Castillo

   1. Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, United States
   2. Department of Psychology and Brain Sciences, University of Massachusetts Amherst, Amherst, United States

   Contribution

   Investigation, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

4. David Badre

   1. Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, United States
   2. Carney Institute for Brain Science, Brown University, Providence, United States

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Project administration, Writing - review and editing

   Competing interests

   Reviewing editor, eLife

• 3,012

  Page views

• 520

  Downloads

• 10

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.