Identifying the optimal number of brain latent states based on the criterion of statistical robustness and cognitive sensibility.

(a) For each candidate K ranging from 2 to 10, we trained an HMM model on n-1 subjects and applied it to decode the time course of state expression for the test subject. The decoded time course was then used to compute a Calinski-Harabasz score, with a larger value indicating better clustering performance, and to decipher which narrative (out of three) was heard by the subject. The two measurements were first assessed at the individual level and then averaged across participants. (b) Model performance as a function of K. With the increase of K, the model’s clustering performance tended to decline while the ability to decipher narrative contents tended to improve. We combined the two indices by converting them independently to z scores and summed them up. Notably, at K=3, the summed z-score reached its highest point, therefore it was set as the optimal number of latent states.

Spatial and temporal features of latent states revealed by HMM.

(a) The activity loadings of each state on the nine networks. These activity values preserved the scale of the original BOLD time courses which were z-score normalized. (b) The ebb and flow of state expression over the time course of narrative comprehension, plotted using data from a representative participant. The curves of the three states are stacked showing the relative strength of activation probability at each time interval. (c) Between-state transition probabilities. Both State #1 and State #3 were more likely to switch to State #2 (denoted by the pink lines) than switching directly to each other. The differences in transition probabilities were larger than most of the instances from surrogate data. (d). Topological properties of whole-brain networks when occupied by each of the three states. Brain occupied by State #2 demonstrated the highest global efficiency (G) and the lowest modularity (Q). The upper panel shows the results of graph constructed using state-specific time series extracted from individual participants. The lower panel shows results of graph constructed using FC matrix derived from the HMM. *p < 0.05, **p <0.01, ***p<0.0005 for t-tests.

Selective modulation of state expression by different narrative features.

(a) The expression probability of State #1, as well as of State #2, was positively modulated by the temporal envelope of speech. The blue shaded area represents the original amplitude of a speech signal clip, while the orange line shows the smoothed, low-frequency contour of amplitude changes over time (i.e., the sound envelope) (b) Only the expression probability of State #2 was modulated by word-level semantic coherence. (c) Only the expression probability of State #3 was modulated by clause-level semantic coherence. Semantic coherence was measured by cosine similarity between the embeddings (obtained by BERT) of each word (or clause) and the word (clause) immediately before it. Those effects were greater than most instances from permutation where the time courses of state expression were randomly shuffled 5000 times. *p < 0.05, **p < 0.01, ***p < 0.005 for t-tests.

Correlation of state expression with behavior.

Participants’ alignment with both the best performer(s) and the group mean in terms of brain state expression predicted their narrative comprehension scores. The alignment with the best performer in head movement trajectory, which probably reflected inter-subject similarity in the fluctuation of task engagement or attention, also correlated with narrative comprehension. After adjusting this effect using partial correlation, the significant correlations between inter-subject alignment in states expression and narrative comprehension still existed.

Comparisons of brain states across conditions.

(a) During rest, the activity patterns of latent states were similar to those during narrative comprehension, but State #3 became the transitional hub. (b) When listening to the unintelligible narrative (in Mongolian, MG), the activity patterns of latent states varied substantially from that during narrative comprehension, but State#2 was still the transitional hub. (c) The fractional occupation of State#2 increased with greater involvement in linguistic computations, while that of State#3 decreased. A similar pattern was found on the dwelling time of states.

Spatial maps of the 11 networks derived from whole-brain parcellation using data from 64 participants engaged in narrative comprehension. The two unlabeled networks consisted of only one or two nodes each and were therefore excluded from further analyses.

Replicating the findings with the 7-network atlas to parcellate brain networks. (a) Model selection. The model with K=3 achieved the overall best performance in terms of clustering and accuracy in classifying three narratives. (b) Establishing State correspondence between parcellation scheme. Each of the three states from the 7-network scheme (treated as candidate states) was best matched to State #1, State #2, and State #3 from the 9-network scheme (treated as predefined states), respectively. The degree of actual overlap for the three states exceeded 100%, 100% and 99.97% of the permutation instances. (c) Activity patterns of latent states. The network activity values preserved the scale of the original BOLD time courses which were z-scored normalized. (d) Between-state switching probabilities. State#2 was the transitional hub. (e) Topological properties of whole brain networks when occupied by each state. At State#2, the brain exhibited the highest global efficiency and lowest modularity. (f) Modulation of state expression probability by narrative properties. (g) Alignment with the best performer predicted participants’ narrative comprehension performance.

Replicating the major results on a different dataset consisting of older adults. (a) Model selection. The model with K=3 achieved the best overall performance in terms of clustering and accuracy in classifying the two narratives. (b) Establishing State correspondence between datasets. Each of the three states from the older-adult dataset (treated as candidate states) was best matched to State #1, State #2, and State #3 from the young-adult dataset (treated as predefined states), respectively. The degree of actual overlap for the three states exceeded 95.7%, 93.04% and 90.30% of the permutation instances. (c) Activity patterns of latent states. The network activity values preserved the scale of the original BOLD time courses which were z-scored normalized (d) Between-state switching probabilities. State#2 was the transitional hub. (e) Topological properties of whole brain networks when occupied by each state. At State#2, the brain exhibited the highest global efficiency and lowest modularity. (f) Modulation of state expression probability by narrative properties.

Reconstructing the tripartite-state space from smaller states.

Left and middle panels: States inferred by HMM with K= 4,10, and 12 were hierarchically grouped into three clusters based on between-states transition probability matrix. Right panels: The reconstructed states (clusters) from the 4- and 10-state models show high and exclusive similarity to the three states (targets) inferred by HMM with K=3.

Comparison in inter-subject correlation (ISC) in brain state dynamics between the young and older age groups. ISC was assessed by iteratively leaving out one participant at a time. For each left-out participant, we calculated the Pearson correlation between their time course of state expression probabilities and the average time course of the remaining participants, all of whom were all engaged in the same narrative. For all three states, the ISCs were significantly higher among the young adults than that among the older adults. For State #1: t(59, 49) = 3.32, p = 0.001; for State #2: t(59, 49) = 3.94, p < 0.001; and for State #3: t(59, 49) = 11.96, p < 10-6.