Stimulus representation in human frontal cortex supports flexible control in working memory

Reviewer #1 (Public review):

Summary:

In this manuscript, Shao et al. investigate the contribution of different cortical areas to working memory maintenance and control processes, an important topic involving different ideas about how the human brain represents and uses information when no longer available to sensory systems. In two fMRI experiments, they demonstrate that human frontal cortex (area sPCS) represents stimulus (orientation) information both during typical maintenance, but even more so when a categorical response demand is present. That is, when participants have to apply an added level of decision control to the WM stimulus, sPCS areas encode stimulus information more than conditions without this added demand. These effects are then expanded upon using multi-area neural network models, recapitulating the empirical gradient of memory vs control effects from visual to parietal and frontal cortices. Multiple experiments and analysis frameworks provide support for the authors' conclusions, and control experiments and analysis are provided to help interpret and isolate the frontal cortex effect of interest. While some alternative explanations/theories may explain the roles of frontal cortex in this study and experiments, important additional analyses have been added that help ensure a strong level of support for these results and interpretations.

Strengths:

- The authors use an interesting and clever task design across two fMRI experiments that is able to parse out contributions of WM maintenance alone along with categorical, rule-based decisions. Importantly, the second experiment only uses one fixed rule, providing both an internal replication of Experiment 1's effects and extending them to a different situation when rule switching effects are not involved across mini-blocks.

- The reported analyses using both inverted encoding models (IEM) and decoders (SVM) demonstrate the stimulus reconstruction effects across different methods, which may be sensitive to different aspects of the relationship between patterns of brain activity and the experimental stimuli.

- Linking the multivariate activity patterns to memory behavior is critical in thinking about the potential differential roles of cortical areas in sub-serving successful working memory. Figure 3's nicely shows a similar interaction to that of Figure 2 in the role of sPCS in the categorization vs. maintenance tasks. This is an important contribution to the field when we consider how a distributed set of interacting cortical areas supports successful working memory behavior.

- The cross-decoding analysis in Figure 4 is a clever and interesting way to parse out how stimulus and rule/category information may be intertwined, which would have been one of the foremost potential questions or analyses requested by careful readers.

- Additional ROI analyses in more anterior regions of the PFC help to contextualize the main effects of interest in the sPCS (and no effect in the inferior frontal areas, which are also retinotopic, adds specificity). And, more explanation for how motor areas or preparation are likely not involved strengthens the takeaways of the study (M1 control analysis).

Weaknesses:

- An explicit, quantitative link between the RNN and fMRI data is perhaps a last point that would integrate the RNN conclusion and analyses in line with the human imaging data.

- As Rev 2 mentions, multiple types of information codes may be present, and the response letter Figure 5 using representational similarity (RSA) gets at this question. It would strengthen the work to, at minimum, include this analysis as an extended or supplemental figure.

To sum up the results, a possible, brief schematic of each cortical area analyzed and its contribution to information coding in WM and successful subsequent behavior may help readers take away important conclusions of the cortical circuitry involved.

Reviewer #2 (Public review):

Summary:

The author provide evidence that helps resolve long-standing questions about the differential involvement of frontal and posterior cortex in working memory. They show that whereas early visual cortex shows stronger decoding of memory content in a memorization task vs a more complex categorization task, frontal cortex shows stronger decoding during categorization tasks than memorization tasks. They find that task-optimized RNNs trained to reproduce the memorized orientations show some similarities in neural decoding to people. Together, this paper presents interesting evidence for differential responsibilities of brain areas in working memory.

Strengths:

This paper was overall strong. It had a well-designed task, best-practice decoding methods, and careful control analyses. The neural network modeling adds additional insight into the potential computational roles of different regions.

Weaknesses:

Few. While more could be perhaps done to understand the RNN-fMRI correspondence, the paper contributes a compelling set of empirical findings and interpretations that can inform future research.

Author response:

The following is the authors’ response to the original reviews.

Reviewer #1 (Public Review):

Summary:

In this manuscript, Shao et al. investigate the contribution of different cortical areas to working memory maintenance and control processes, an important topic involving different ideas about how the human brain represents and uses information when it is no longer available to sensory systems. In two fMRI experiments, they demonstrate that the human frontal cortex (area sPCS) represents stimulus (orientation) information both during typical maintenance, but even more so when a categorical response demand is present. That is, when participants have to apply an added level of decision control to the WM stimulus, sPCS areas encode stimulus information more than conditions without this added demand. These effects are then expanded upon using multi-area neural network models, recapitulating the empirical gradient of memory vs control effects from visual to parietal and frontal cortices. In general, the experiments and analyses provide solid support for the authors' conclusions, and control experiments and analyses are provided to help interpret and isolate the frontal cortex effect of interest. However, I suggest some alternative explanations and important additional analyses that would help ensure an even stronger level of support for these results and interpretations.

Strengths:

- The authors use an interesting and clever task design across two fMRI experiments that is able to parse out contributions of WM maintenance alone along with categorical, rule-based decisions. Importantly, the second experiment only uses one fixed rule, providing both an internal replication of Experiment 1's effects and extending them to a different situation when rule-switching effects are not involved across mini-blocks.

- The reported analyses using both inverted encoding models (IEM) and decoders (SVM) demonstrate the stimulus reconstruction effects across different methods, which may be sensitive to different aspects of the relationship between patterns of brain activity and the experimental stimuli.

- Linking the multivariate activity patterns to memory behavior is critical in thinking about the potential differential roles of cortical areas in sub-serving successful working memory. Figure 3 nicely shows a similar interaction to that of Figure 2 in the role of sPCS in the categorization vs. maintenance tasks.

- The cross-decoding analysis in Figure 4 is a clever and interesting way to parse out how stimulus and rule/category information may be intertwined, which would have been one of the foremost potential questions or analyses requested by careful readers. However, I think more additional text in the Methods and Results to lay out the exact logic of this abstract category metric will help readers bet0ter interpret the potential importance of this analysis and result.

We thank the reviewer for the positive assessment of our manuscript. Please see lines 366-372, 885-894 in the revised manuscript for a detailed description of the abstract category index, and see below for a detailed point-by-point response.

Weaknesses:

- Selection and presentation of regions of interest: I appreciate the authors' care in separating the sPCS region as "frontal cortex", which is not necessarily part of the prefrontal cortex, on which many ideas of working memory maintenance activity are based. However, to help myself and readers interpret these findings, at a minimum the boundaries of each ROI should be provided as part of the main text or extended data figures. Relatedly, the authors use a probabilistic visual atlas to define ROIs in the visual, parietal, and frontal cortices. But other regions of both lateral frontal and parietal cortices show retinotopic responses (Mackey and Curtis, eLife, 2017: https://elifesciences.org/articles/22974) and are perhaps worth considering. Do the inferior PCS regions or inferior frontal sulcus show a similar pattern of effects across tasks? And what about the middle frontal gyrus areas of the prefrontal cortex, which are most analogous to the findings in NHP studies that the authors mention in their discussion, but do not show retinotopic responses? Reporting the effects (or lack thereof) in other areas of the frontal cortex will be critical for readers to interpret the role of the frontal cortex in guiding WM behavior and supporting the strongly worded conclusions of broad frontal cortex functioning in the paper. For example, to what extent can sPCS results be explained by visual retinotopic responses? (Mackey and Curtis, eLife, 2017: https://elifesciences.org/articles/22974).

We thank the reviewer for the suggestions. We have added a Supplemental Figure 1 to better illustrate the anatomical locations of ROIs.

Following the reviewer’s suggestion, we defined three additional subregions in the frontal cortex based on the HCP atlas [1], including the inferior precentral sulcus (iPCS, generated by merging 6v, 6r, and PEF), inferior frontal sulcus (IFS, generated by merging IFJp, IFJa, IFSp, IFSa, and p47r), and middle frontal gyrus (MFG, generated by merging 9-46d, 46, a9-46v, and p9-46v). We then performed the same analyses as in the main text using both mixed-model and within-condition IEMs. Overall, we found that none of the ROIs demonstrated significant orientation representation in Experiment 1, for either IEM analysis (Author response image 1A and 1C). In Experiment 2, however, the IFS and MFG (but not iPCS) demonstrated a similar pattern to sPCS for orientation representation, though these results did not persist in the within-condition IEM with lower SNR (Author response image 1B and 1D). Moreover, when we performed the abstract category decoding analysis in the three ROIs, only the MFG in Experiment 2 showed significant abstract category decoding results, with no significant difference between experiments (Author response image 1E). To summarize, the orientation and category results observed in sPCS in the original manuscript were largely absent in other frontal regions. There was some indication that the MFG might share some results for orientation representation and category decoding, although this pattern was weaker and was only observed in some analyses in Experiment 2. Therefore, although we did not perform retinotopic mapping and cannot obtain a direct measure of retinotopic responses in the frontal cortex, these results suggest that our findings are unlikely to be explained by visual retinotopic responses: the iPCS, which is another retinotopic region, did not show the observed pattern in any of the analyses. Notably, the iPCS results are consistent with our previous work demonstrating that orientation information cannot be decoded from iPCS during working memory delay [2]. We have included these results on lines 395-403, 563-572 in the revised manuscript to provide a more comprehensive understanding of the current findings.

Author response image 1.

Orientation reconstruction and abstract category decoding results in iPCS, IFS, and MFG.

- When looking at the time course of effects in Figure 2, for example, the sPCS maintenance vs categorization effects occur very late into the WM delay period. More information is needed to help separate this potential effect from that of the response period and potential premotor/motor-related influences. For example, are the timecourses shifted to account for hemodynamic lag, and if so, by how much? Do the sPCS effects blend into the response period? This is critical, too, for a task that does not use a jittered delay period, and potential response timing and planning can be conducted by participants near the end of the WM delay. For example, the authors say that " significant stimulus representation in EVC even when memoranda had been transformed into a motor format (24)". But, I *think* this paper shows the exact opposite interpretation - EVC stimulus information is only detectable when a motor response *cannot* be planned (https://elifesciences.org/articles/75688). Regardless, parsing out the timing and relationship to response planning is important, and an ROI for M1 or premotor cortex could also help as a control comparison point, as in reference (24).

We thank the reviewer for raising this point. We agree that examining the contribution of response-related activity in our study is crucial, as we detail below:

First, the time course results in the manuscript are presented without time shifting. The difference in orientation representation in Figure 2 emerged at around 7 s after task cue onset and 1 s before probe onset. Considering a 4-6 s hemodynamic response lag, the difference should occur around 1-3 s after task cue onset and 5-7 s prior to probe onset. This suggests that a substantial portion of the effect likely occurred during the delay rather than response period.

Second, our experimental design makes it unlikely that response planning would have influenced our results, as participants were unable to plan their motor responses in advance due to randomized response mapping at the probe stage on a trial-by-trial basis. Moreover, even if response planning had impacted the results in sPCS, it would have affected both conditions similarly, which again, would not explain the observed differences between conditions.

Third, following the reviewer’s suggestion, we defined an additional ROI (the primary motor cortex, M1) using the HCP atlas and repeated the IEM analysis. No significant orientation representation was observed in either condition in M1, even during the response period (Figure S3), further suggesting that our results are unlikely to be explained by motor responses or motor planning.

Based on the evidence above, we believe motor responses or planning are unlikely to account for our current findings. We have included these results on lines 264-267 to further clarify this issue.

Lastly, upon re-reading the Henderson et al. paper [3], we confirmed that stimulus information was still decodable in EVC when a motor response could be planned (Figure 2 of Henderson et al.). In fact, the authors also discussed this result in paragraph 5 of their discussion. This finding, together with our results in EVC, indicates that EVC maintains stimulus information in working memory even when the information is no longer task-relevant, the functional relevance of which warrants further investigation in future research.

- Interpreting effect sizes of IEM and decoding analysis in different ROIs. Here, the authors are interested in the interaction effects across maintenance and categorization tasks (bar plots in Figure 2), but the effect sizes in even the categorization task (y-axes) are always larger in EVC and IPS than in the sPCS region... To what extent do the authors think this representational fidelity result can or cannot be compared across regions? For example, a reader may wonder how much the sPCS representation matters for the task, perhaps, if memory access is always there in EVC and IPS? Or perhaps late sPCS representations are borrowing/accessing these earlier representations? Giving the reader some more intuition for the effect sizes of representational fidelity will be important. Even in Figure 3 for the behavior, all effects are also seen in IPS as well. More detail or context at minimum is needed about the representational fidelity metric, which is cited in ref (35) but not given in detail. These considerations are important given the claims of the frontal cortex serving such an important for flexible control, here.

We thank the reviewer for raising this point. We agree that the effect sizes are always larger in EVC and IPS. This is because the specific decoding method we adopted, IEM, is based on the concept of population-level feature-selective responses, and decoding results would be most robust in regions with strong feature-tuning responses, such as EVC and parts of IPS. Therefore, to minimize the impact of effect size on our results, we avoided direct comparisons of representational strength across ROIs, focusing instead on differences in representational strength between conditions within the same ROI. With this approach, we found that EVC and IPS showed high representational fidelity throughout the trial, but only in sPCS did we observe significant higher fidelity in categorization condition, where orientation was actually not a behavioral goal but was manipulated in working memory to achieve the goal. Moreover, although representational fidelity in the EVC was the highest, its behavioral predictability decreased during the delay period, unlike sPCS. These results suggest that the magnitude of fidelity alone is not the determining factor for the observed categorization vs. maintenance effect or for behavioral performance. We have included further discussion on this issue on lines 208-211 of the revised manuscript.

The reviewer also raised a good point that IPS showed similar behavioral correlation results as sPCS. In the original manuscript, we discussed the functional similarities and distinctions between IPS and sPCS in the discussion. We have expanded on this point on lines 610-627 in the revised manuscript:

“While many previous WM studies have focused on the functional distinction between sensory and frontoparietal cortex, it has remained less clear how frontal and parietal cortex might differ in terms of WM functions. Some studies have reported stimulus representations with similar functionality in frontal and parietal cortex [4, 5], while others have observed differential patterns [6-8]. We interpret the differential patterns as reflecting a difference in the potential origin of the corresponding cognitive functions. For example, in our study, sPCS demonstrated the most prominent effect for enhanced stimulus representation during categorization as well as the tradeoff between stimulus difference and category representation, suggesting that sPCS might serve as the source region for such effects. On the other hand, IPS did show visually similar patterns to sPCS in some analyses. For instance, stimulus representation in IPS was visually but not statistically higher in the categorization task. Additionally, stimulus representation in IPS also predicted behavioral performance in the categorization task. These results together support the view that our findings in sPCS do not occur in isolation, but rather reflect a dynamic reconfiguration of functional gradients along the cortical hierarchy from early visual to parietal and then to frontal cortex.”

Lastly, following the reviewer’s suggestion, we have included more details on the representational fidelity metric on lines 201-206, 856-863 in the revised manuscript for clarity.

Recommendations:

Figure 3 layout - this result is very interesting and compelling, but I think could be presented to have the effect demonstrated more simply for readers. The scatter plots in the second and third rows take up a lot of space, and perhaps having a barplot as in Figure 2 showing the effects of brain-behavior correlations collapsed across the WM delay period timing would make the effect stand out more.

We thank the reviewer for the suggestion. We have added a subplot (C) to Figure 3 to demonstrate the brain-behavior correlation collapsed across the late task epoch.

When discussing the link between sPCS representations and behavior, I think this paper should likely be cited ([https://www.jneurosci.org/content/24/16/3944](https://www.jneurosci.org/content/24/ 16/3944)), which shows univariate relationships between sPCS delay activity and memory-guided saccade performance.

We thank the reviewer for the suggestion and have included this citation on lines 278-279 in the revised manuscript.

Interpretation of "control" versus categorization - the authors interpret that "It would be of interest to further investigate whether this active control in the frontal cortex could be generalized to tasks that require other types of WM control such as mental rotation." I think more discussion on the relationship between categorization and "control" is needed, especially given the claim of "flexible control" throughout. Is stimulus categorization a form of cognitive control, and if so, how?

We thank the reviewer for raising this point. Cognitive control is generally defined as the process by which behavior is flexibly adapted based on task context and goals, and most theories agree that this process occurs within working memory [9, 10]. With this definition, we consider stimulus categorization to be a form of cognitive control, because participants needed to adapt the stimulus based on the categorization rule in working memory for subsequent category judgements. With two categorization rules, the flexibility in cognitive control increased, because participants need to switch between the two rules multiple times throughout the experiment, instead of being fixed on one rule. We now clarify these two types of controls on lines 112-116 in the introduction.

However, we agree that the latter form of control could be more related to rule switching that might not be specific to categorization per se. For instance, if participants perform rule switching in another type of WM task that requires WM control such as mental rotation, it remains to be tested whether similar results would be observed and/or whether same brain regions would be recruited. We have included further information on this issue on lines 572-575 in the revised manuscript.

Reviewer #2 (Public Review):

Summary:

The authors provide evidence that helps resolve long-standing questions about the differential involvement of the frontal and posterior cortex in working memory. They show that whereas the early visual cortex shows stronger decoding of memory content in a memorization task vs a more complex categorization task, the frontal cortex shows stronger decoding during categorization tasks than memorization tasks. They find that task-optimized RNNs trained to reproduce the memorized orientations show some similarities in neural decoding to people. Together, this paper presents interesting evidence for differential responsibilities of brain areas in working memory.

Strengths:

This paper was strong overall. It had a well-designed task, best-practice decoding methods, and careful control analyses. The neural network modelling adds additional insight into the potential computational roles of different regions.

We thank the reviewer for the positive assessment of our manuscript.

Weaknesses:

While the RNN model matches some of the properties of the task and decoding, its ability to reproduce the detailed findings of the paper was limited. Overall, the RRN model was not as well-motivated as the fMRI analyses.

We are grateful for the reviewer’s suggestions on improving our RNN results. Please see below for a detailed point-by-point response.

Recommendations:

Overall, I thought that this paper was excellent. I have some conceptual concerns about the RNN model, and minor recommendations for visualization.

(1) I think that the RNN modelling was certainly interesting and well-executed. However, it was not clear how much it contributed to the results. On the one hand, it wasn't clear why reproducing the stimulus was a critical objective of the task (ie could be more strongly motivated on biological grounds). On the other hand, the agreement between the model and the fMRI results is not that strong. The model does not reproduce stronger decoding in 'EVC' for maintenance vs categorization. Also, the pattern of abstract decoding is very different from the fMRI (eg the RNN has stronger categorical encoding in 'EVC' than 'PFC' and larger differences between fixed and flexible rules in earlier areas than is evident in the fMRI). Together, the RNN modelling comes across as a little ad hoc, without really nailing the performance.

We thank the reviewer for prompting us to further elaborate on the rationale for our RNN analysis. In our fMRI results, we observed a tradeoff between maintaining stimulus information in more flexible tasks (Experiment 1) and maintaining abstract category information in less flexible tasks (Experiment 2). This led to the hypothesis that participants might have employed different coding strategies in the two experiments. Specifically, in flexible environments, stimulus information might be preserved in its original identity in the higher-order cortex, potentially reducing processing demands in each task and thereby facilitating efficiency and flexibility; whereas in less flexible tasks, participants might generate more abstract category representations based on task rules to facilitate learning. To directly test this idea, we examined whether explicitly placing a demand for the RNN to preserve stimulus representation would recapitulate our fMRI findings in frontal cortex by having stimulus information as an output, in comparison to a model that did not specify such a demand. Meanwhile, we totally agree with the reviewer that there are alternative ways to implement this objective in the model. For instance, changing the network encoding weights (lazy vs. rich regime) to make feedforward neural networks either produce high-dimensional stimulus or low-dimensional category representations [11]. However, we feel that exploring these alternatives may fall outside the scope of the current study.

Regarding the alignment between the fMRI and RNN results: for the stimulus decoding results in EVC, we found that with an alternative decoding method (IEM), a similar maintenance > categorization pattern was observed in EVC-equivalent module, suggesting that our RNN was capable of reproducing EVC results, albeit in a weaker manner (please see our response to the reviewer’s next point). For the category decoding results, we would like to clarify that the category decoding results in EVC was not necessarily better than those in sPCS. Although category decoding accuracy was numerically higher in EVC, it was more variable compared to IPS and sPCS. To illustrate this point, we calculated the Bayes factor for the category decoding results of RNN2 in Figure 6C, and found that the amount of evidence for category decoding as well as for the decoding difference between RNNs in IPS and sPCS modules was high, whereas the evidence in the EVC was insufficient (Response Table 1).

Author response table 1.

Bayes factors for category decoding and decoding differences in Figure 6C lower panel.

Nevertheless, we agree with the reviewer that all three modules demonstrated the category decoding difference between experiments, which differs from our fMRI results. This discrepancy may be partially due to differences in signal sensitivity. RNN signals typically have a higher SNR compared to fMRI signals, as fMRI aggregates signals from multiple neurons and single-neuron tuning effects can be reduced. We have acknowledged this point on lines 633-636 in the revised manuscript. Nonetheless, the current RNNs effectively captured our key fMRI findings, including increased stimulus representation in frontal cortex as well as the tradeoff in category representation with varying levels of flexible control. We believe the RNN results remain valuable in this regard.

Honestly, I think the paper would have a very similar impact without the modelling results, but I appreciate that you put a lot of work into the modeling, and this is an interesting direction for future research. I have a few suggestions, but nothing that I feel too strongly about.

- It might be informative to use IEM to better understand the RNN representations (and how similar they are to fMRI). For example, this could show whether any of the modules just encode categorical information.

- You could try providing the task and/or retro cue directly to the PFC units. This is a little unrealistic, but may encourage a stronger role for PFC.

- You might adjust the ratio of feedforward/feedback connections, if you can find good anatomical guidance on what these should be.

Obviously, I don't have much - it's a tricky problem!

We thank the reviewer for the suggestions. To better align the fMRI and RNN results, we first performed the same IEM analyses used in the fMRI analyses on the RNN data. We found that with IEM, the orientation representation in the EVC module demonstrated a pattern similar to that in the fMRI data, showing a negative trend for the difference between categorization and maintenance, although the trend did not reach statistical significance (Author response image 2A). Meanwhile, the difference between categorization and maintenance remained a positive trend in the sPCS module.

Second, following the reviewer’s suggestion, we adjusted the ratio of feedforward/feedback connections between modules to 1:2, such that between Modules 1 and 2 and between Modules 2 and 3, there were always more feedback than feedforward connections, consistent with recent theoretical proposals [12]. We found that, this change preserved the positive trend for orientation differences in the sPCS module, but in the meantime also made the orientation difference in the EVC and IPS modules more positive (Author response image 2B).

To summarize, we found that the positive difference between categorization and maintenance in the sPCS module was robust across difference RNNs and analytical approaches, further supporting that RNNs with stimulus outputs can replicate our key fMRI findings in the frontal cortex. By contrast, the negative difference between categorization and maintenance in EVC was much weaker. It was weakly present using some analytical methods (i.e., the IEM) but not others (i.e., SVMs), and increasing the feedback ratio of the entire network further weakened this difference. We believe that this could be due to that the positive difference was mainly caused by top-down, feedback modulations from higher cortex during categorization, such that increasing the feedback connection strengthens this pattern across modules. We speculate that enhancing the negative difference in the EVC module might require additional modules or inputs to strengthen fine-grained stimulus representation in EVC, a mechanism that might be of interest to future research. We have added a paragraph to the discussion on the limitations of the RNN results on lines 629-644.

Author response Image 2.

Stimulus difference across RNN modules. (A). Results using IEM (p-values from Module 1 to 3: 0.10, 0.48, 0.01). (B). Results using modified RNN2 with changed connection ratio (p-values from Module 1 to 3: 0.12, 0.22, 0.08). All p-values remain uncorrected.

(2) Can you rule out that during the categorization task, the orientation encoding in PFC isn't just category coding? You had good controls for category coding, but it would be nice to see something for orientation coding. e.g., fit your orientation encoding model after residualizing category encoding, or show that category encoding has worse CV prediction than orientation encoding.

We thank the reviewer for raising this point. To decouple orientation and category representations, we performed representational similarity analysis (RSA) in combination with linear mixed-effects modeling (LMEM) on the fMRI data. Specifically, we constructed three hypothesized representational dissimilarity matrices (RDMs), one for graded stimulus (increasing distance between orientations as they move farther apart, corresponding to graded feature tuning responses), one for abstract category (0 for all orientations within the same category and 1 for different categories), and another for discrete stimulus (indicating equidistant orientation representations). We then fit the three model RDMs together using LMEM with subject as the random effect (Author response image 3A). This approach is intended to minimize the influence of collinearity between RDMs on the results [13].

Overall, the LMEM results (Author response image 3B-D) replicated the decoding results in the main text, with significant stimulus but not category representation in sPCS in Experiment 1, and marginally significant category representation in the same brain region in Experiment 2. These results further support the validity of our main findings and emphasize the contribution of stimulus representation independent of category representation.

Author response image 3.

Delineating stimulus and category effects using LMEM. (A) Schematic illustration of this method. (B) Results for late epoch in Experiment 1, showing the fit of each model RDM. (C) Results for early epoch in Experiment 2. (D) Results for late epoch in Experiment 2.

(3) Is it possible that this region of PFC is involved in categorization in particular and not 'control-demanding working memory'?

We thank the reviewer for raising this possibility. Cognitive control is generally defined as the process by which behavior is flexibly adapted based on task context and goals, and most theories agree that this process occurs within working memory [9, 10]. With this definition, we consider stimulus categorization to be a form of cognitive control, because participants need to adapt the stimulus based on the categorization rule in working memory for subsequent category judgements. However, in the current study we only used one type of control-demanding working memory task (categorization) to test our hypothesis, and therefore it remains unclear whether the current results in sPCS can generalize to other types of WM control tasks.

We have included a discussion on this issue on lines 572-575 in the revised manuscript.

(4) Some of the figures could be refined to make them more clear:

a. Figure 4 b/c should have informative titles and y-axis labels.

b. Figure 5, the flexible vs fixed rule isn't used a ton up to this point - it would help to (also include? Replace?) with something like exp1/exp2 in the legend. It would also help to show the true & orthogonal rule encoding in these different regions (in C, or in a separate panel), especially to the extent that this is a proxy for stimulus encoding.

c. Figure 6: B and C are very hard to parse right now. (i) The y-axis on B could use a better label. (ii) It would be useful to include an inset of the relevant data panel from fMRI that you are reproducing. (iii) Why aren't there fixed rules for RNN1?

We thank the reviewer for the suggestions and have updated the figures accordingly as following:

Overall I think this is excellent - my feedback is mostly on interpretation and presentation. I think the work itself is really well done, congrats!

References

(1) Glasser, M.F., et al., A multi-modal parcellation of human cerebral cortex. Nature, 2016. 536(7615): p. 171-178.

(2) Yu, Q. and Shim, W.M., Occipital, parietal, and frontal cortices selectively maintain taskrelevant features of multi-feature objects in visual working memory. Neuroimage, 2017. 157: p. 97-107.

(3) Henderson, M.M., Rademaker, R.L., and Serences, J.T., Flexible utilization of spatial- and motor-based codes for the storage of visuo-spatial information. Elife, 2022. 11.

(4) Christophel, T.B., et al., Cortical specialization for attended versus unattended working memory. Nat Neurosci, 2018. 21(4): p. 494-496.

(5) Yu, Q. and Shim, W.M., Temporal-Order-Based Attentional Priority Modulates Mnemonic Representations in Parietal and Frontal Cortices. Cereb Cortex, 2019. 29(7): p. 3182-3192.

(6) Li, S., et al., Neural Representations in Visual and Parietal Cortex Differentiate between Imagined, Perceived, and Illusory Experiences. J Neurosci, 2023. 43(38): p. 6508-6524.

(7) Hu, Y. and Yu, Q., Spatiotemporal dynamics of self-generated imagery reveal a reverse cortical hierarchy from cue-induced imagery. Cell Rep, 2023. 42(10): p. 113242.

(8) Lee, S.H., Kravitz, D.J., and Baker, C.I., Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat Neurosci, 2013. 16(8): p. 997-9.

(9) Miller, E.K. and Cohen, J.D., An integrative theory of prefrontal cortex function. Annu Rev Neurosci, 2001. 24: p. 167-202.

(10) Badre, D., et al., The dimensionality of neural representations for control. Curr Opin Behav Sci, 2021. 38: p. 20-28.

(11) Flesch, T., et al., Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron, 2022. 110(7): p. 1258-1270 e11.

(12) Wang, X.J., Theory of the Multiregional Neocortex: Large-Scale Neural Dynamics and Distributed Cognition. Annu Rev Neurosci, 2022. 45: p. 533-560.

(13) Bellmund, J.L.S., et al., Mnemonic construction and representation of temporal structure in the hippocampal formation. Nat Commun, 2022. 13(1): p. 3395.