Flexible brain state engagement predicts cognitive control transdiagnostically

Reviewer #1 (Public review):

Summary:

This paper uses three different datasets to study the relationship between the standard deviation of dynamic brain state time series (state engagement variability or SEV) and measures of cognition. Results show associations between SEV and cognitive measures, with stronger associations in patients than controls (at least for inhibition).

Strengths:

Strengths include the use of innovative dynamic approaches to study cognition and the validations across three independent datasets.

Weaknesses:

With a highly innovative approach, it can be challenging to provide enough context for the reader to understand and interpret the results. In particular, the paper would benefit from:

(1) More detail on the brain state calculation, multiple comparison control, and added benchmarking of the novel summary SEV measure.

(2) Guidance on the interpretation of relatively low prediction performance, negative t-statistics, and more broadly regarding the justification for the multi-step approach going from 4 brain states to 1 SEV to a network of edges.

(3) Removal of the moment-to-moment alignment results given the circularity of the edge time series extraction with overlapping contributions to SEV and cognitive control time series.

(4) Adjustment of text to avoid causal interpretations and to reduce the emphasis on transdiagnostics.

Major Points:

While the brain states were developed in prior work, SEV is a new metric and therefore warrants careful benchmarking in terms of test-retest reliability, sensitivity to scan length/quality, and associations with demographic variables like age and sex (which do not appear to be controlled for in analyses).

Although the external validation approach is appreciated, the prediction performance is pretty low (predicted-observed correlation 0.17-0.3). It would be good to also report other metrics of performance, such as balanced accuracy.

The steps in the paper are somewhat convoluted by going from 4 brain states to 1 SEV, back to specific FC networks. This makes the paper a bit complex and difficult to interpret. It would be helpful to provide a clear justification for these steps and/or a figure to orient the readers.

Many results are reported in the manuscript, and it is unclear whether/what multiple comparisons control was adopted where.

The moment-to-moment change section tries to test whether inter-individual variation in SEV maps onto cognitive control, which is very interesting. However, both measures were operationalized using edge-timeseries calculated from the same data with shared inputs (as shown in Figure 4B). As such, the 'alignment' (i.e., correlation) between resulting time series appears somewhat circular given that it is likely driven by the shared inputs. More broadly, edge timeseries were summed across edges (and subtracted between edges with positive and negative CPM associations), which further complicates their interpretability in the context of 'cognitive control'. I would recommend removing this section or using behavioral data to quantify cognitive control.

The descriptions of how brain states were derived are unclear. In line 466, what do 'these fMRI data' refer to? Was the least-squares regression performed across subjects (given that it results in one beta value per time point)? Was this performed as a multiple regression and - if so - what was the collinearity between brain state inputs?

Reviewer #2 (Public review):

Summary:

A relatively new measure of flexible brain state engagement (SEV - State Engagement Variability) is used here. It simply measures time-to-time variation in brain activity in terms of how it matches pre-specified motifs of activity. This metric seems to be predictive of behavioural data measuring cognitive control abilities. This was found to be the case in two independent datasets with different (though related) behavioural measures.

Strengths:

Use of multiple datasets is a clear strength. The use of both replication and out-of-sample model prediction is another.

Weaknesses:

(1) It is not clear to me how specific the SEV metric is for telling us about brain state engagement flexibility. Resting state fluctuations have been described as quasi-periodic changes that can be mapped onto "states", but the fluctuations could easily be a reflection of vascular flow, which may indirectly correlate with cognition.

(2) If SEV is calculated using other state descriptors (e.g. a random parcellation of the brain into 4 networks) - would the result still hold? Or are the motifs important (this would rule out, to some extent, the vascular argument from (1) above)?

(3) Figure 1 confused me a little. Why not show all the combinations (patient v full sample), inhibition vs shift, and main vs validation? Instead, a subset of 4 was selected?

(4) The inhibition/patient/main correlation seems to be driven by 4 patients with particularly high inhibition measures?

(5) Why is SEV negative in some cases (e.g., Figure 1) if it's a std measure? Has it been demeaned or orthogonalised wrt another variable?

(6) The external analysis is great, but why should the model predict a relationship between SEV and inhibition if the claim is that it is only true for patients? Why would it only be true for patients in the first place?

(7) I can't get my head around the results shown in Figure 3. How can one have both positive and negative correlations being significant or meaningful in the same pairs of networks? I think this set of results could benefit from more explanation.

(8) I struggled with Figure 4 analysis. What is the SEV network? How do we know that it is specific enough to the SEV concept? Looking at co-fluctuations with the cognitive network, are we not simply looking at the old anti-correlation between the default mode and the rest of the brain (I note that the correlations in the y-axes of Figure 4 are negative)?