Identifying the optimal number of brain latent states based on the criterion of statistical robustness and cognitive sensibility.

(a) For each candidate K ranging from 2 to 10, we trained an HMM model on n-1 subjects and applied it to decode the time course of state expression for the test subject. The decoded time course was then used to compute a Calinski-Harabasz score, with a larger value indicating better clustering performance, and to decipher which narrative (out of three) was heard by the subject. The two measurements were first assessed at the individual level and then averaged across participants. (b) Model performance as a function of K. With the increase of K, the model’s clustering performance tended to decline while the ability to decipher narrative contents tended to improve. We combined the two indices by converting them independently to z scores and summed them up. Notably, at K=3, the summed z score reached its highest point, therefore it was set as the optimal number of latent states.

Spatial and temporal features of latent states revealed by HMM.

(a). The activity loadings of each state on the nine networks. For visualization purpose, the spatial map was normalized to the range [2, 10] with min-max normalization. (b). The ebb and flow of state expression over the time course of narrative understanding, plotted using data from a representative participant. The curves of the three states are stacked showing the relative strength of activation probability at each time interval. (c). Between-state transition probabilities. Both State #1 and #3 were more likely to switch to State #2 than switching directly to each other. The differences in transition probabilities were larger than most of the instances from surrogate data. (d). Topological properties of whole-brain networks when occupied by each of the three states. Brain occupied by State #2 demonstrated the highest global efficiency (G) and the lowest modularity (Q). The upper panel shows the results of graph constructed using state-specific time series extracted from individual participants. The lower panel shows results of graph constructed using FC matrix derived from the HMM. *p < 0.05, **p <0.01, ***p<0.0005 for t-tests.

Selective modulation of state expression by different narrative features.

The expression probability of State #1, as well as of State #2, was positively modulated by the temporal envelope of speech. The expression probability of State #2 was also modulated by word-level semantic coherence, while that of State #3 was modulated by clause-level semantic coherence. Semantic coherence was measured by cosine similarity between the embeddings (obtained by BERT) of each word (or clause) and the word (clause) immediately before it. Those effects were greater than most of the instances from permutation where the time courses of state expression were randomly shuffled 5000 times. *p < 0.05, **p< 0.01, ***p < 0.005 for t-tests.

Correlation of state expression with behavior.

Participants’ alignments with both the best performer(s) and the group mean in terms of brain state expression predicted their narrative comprehension scores. The alignment with the best performer in head movement trajectory, which probably reflected inter-subject similarity in the fluctuation of task engagement or attention, also correlated with narrative comprehension. After adjusting this effect using partial correlation, the significant correlations between inter-subject alignment in states expression and narrative comprehension still existed.

Comparisons of brain states across conditions.

(a) During rest, the activity patterns of latent states were similar to those during narrative comprehension, but State #3 became the transitional hub. (b) When listening to the unintelligible narrative (in Mongolian, MG), the activity patterns of latent states varied substantially from that during narrative comprehension, but State#2 was still the transitional hub. (c) The fractional occupation of State#2 increased with greater involvement in linguistic computations, while that of State#3 decreased. A similar pattern was found on the dwelling time of states.

The spatial maps of 11 networks derived from whole-brain parcellation using data from 64 participants engaged in narrative comprehension. The last unlabeled two networks consisted of only one or two nodes (parcels), and therefore were not included in further analyses.

Replicating the findings with the 7-network atlas to parcellate brain networks. (a) Model selection. The model with K=3 achieved the overall best performance in terms of clustering and accuracy in classifying three narratives. (b) Activity patterns of latent states. (c) Between-state switching probabilities. State#2 was the transitional hub. Topological properties of whole brain networks when occupied by each state. At State#2, the brain exhibited the highest global efficiency and lowest modularity. (e) The modulation of state expression probability by narrative properties. (f) Alignment with the best performer predicted participants’ narrative comprehension performance.

Replicating the major result on an independent dataset consisting of older adults. (a) Model selection. The model with K=3 achieved the overall best performance in terms of clustering and accuracy in classifying two narratives. (b) Activity patterns of latent states. Note, each of the three state was exclusively correlated to one of the target states (from the young group) in activity patterns, as demonstrated in the matrix. (c) Between-state switching probabilities. State#2 was the transitional hub. (d) Topological properties of whole brain networks when occupied by each state. At State#2, the brain exhibited the highest global efficiency and lowest modularity. (d) The modulation of state expression probability by narrative properties.

Reconstructing the tripartite-state space from smaller states.

Left and middle panels: States inferred by HMM with K= 4,10, and 12 were hierarchically grouped into three clusters based on between-states transition probability matrix. Right panels: The reconstructed states (clusters) from the 4- and 10-state models show high and exclusive similarity to the three states (targets) inferred by HMM=3.