Introduction

Resting state fMRI serves as a tool to map brain function (1, 2). Functional connectivity (FC) at rest, estimated with methods to gauge correlated spontaneous activity between two or more regions, serves as a measure of functional brain integrity (1, 3). Individual differences in FC at rest are associated with differences in various cognitive domains (4–6). Associations of this nature have been reported for several large-scale networks, including the default mode network (DMN) (7–9), the frontoparietal network (10–12), the dorsal attention network (13), and the salience network (14). Moreover, such associations have also been identified in more comprehensive investigations spanning the entire functional brain repertoire of the brain (15, 16). Most previous reports focused on associations, not predictions, utilizing the correlation between FC and behavioral phenotypes, which tend to overfit the data and therefore fail to generalize (17). Proper cross-validation, preferably in an independent sample, is therefore important to assert reliable population-level inferences (18). Recent machine learning-based predictive frameworks offer powerful tools for assessing the predictability of individual behavioral phenotypes based on brain connectivity (19–23). In particular, deep neural networks (DNN) methods have been successfully applied to behavioral and disease prediction (24–26), and have been found to outperform other machine learning approaches (27–29).

The functional connectome has demonstrated predictive utility regarding trait-like cognitive phenotypes (30–33). The predictive-modeling framework of the functional connectome has been applied to various cognitive domains, including fluid intelligence (34), working memory (WM) (35), visuospatial ability (36), attention (37), creativity (38), as well as personality traits (39). Understanding the patterns contributing to predictions could offer insights into the functional organization underlying cognitive phenotypes, serving as biomarkers indicating current or prospective health conditions (40–42). Moreover, the whole-brain functional connectome acts as a fingerprint with accurate identification of subjects from a large population (43) within the same cognitive state (e.g., rest-to-rest) but also across different states (e.g., rest-to-task). Overall, past research suggests that the functional connectome is relatively robust within individuals, is unique across individuals, and can predict cognitive and personality phenotypes.

Despite consensus on the value of resting state functional connectivity for mapping brain function, there is an ongoing debate about whether rest is the optimal brain state for investigating individual differences in neurocognitive function (44–47). A study using data from the human connectome project has shown that resting state fMRI predicts differences in brain activity during various tasks, including social, language, relational, and motor tasks (48). This finding supports the notion that individual differences in neural activation can be predicted from resting state (46). However, results from the same dataset revealed that task-based FC outperforms resting state FC in predicting individual differences in fluid intelligence, with task-based data explaining 20 % of the variance compared to just 6 % explained by rest (49). Consistent with this result, previous studies have investigated trait-as well as state-dependent FC, supporting the utility of an integrative approach (11, 47). More recent studies suggested that naturalistic viewing, such as movie-watching, may serve as a happy medium between unconstraint rest and overlay-constrained tasks in predicting behavior differences (50, 51). Despite the presence of similar spatiotemporal activity patterns across individuals during movie-watching (52), notable individual differences in activity and functional connectivity (53) persist alongside these idiosyncratic features. This suggests that tasks which align individuals’ functional connectome more closely to an optimal level, neither completely unconstrained like rest nor overly synchronized like a task, also render them easiest to identify (44). Hence, it is plausible that FC during naturalistic paradigms improves sensitivity to predict behavioral differences (50, 54).

In this study, we first assess the predictive efficacy of the functional connectome across various states for two commonly used cognitive functions. Specifically, we focused on working memory i.e., the ability to store and manipulate information over shorter durations of time, as well as its integral involvement in problem-solving, reasoning, and other key components of fluid intelligence (e.g., (55–57)). In contrast, episodic memory (EM) entails the recollection of specific experiences and events (58), which is regarded as an important element of mind-wandering (59) during resting state.

Past studies have used neuroimaging data, including resting state to predict brain age (60–62). These studies show that brain age, based on biological phenotypes, and their deviation from the chronological age (known as brain age gap prediction), could serve as a biomarker in characterizing disease risk (62–64). Importantly, an older-appearing brain has been shown to be associated with exacerbated physiological and cognitive aging, and risk of mortality (61). However, a recent study suggests that brain age predictions contribute minimally compared to chronological age for explaining cognitive decline (65), implying that cognitive predictions are more reliable. Building upon this study and extending the concept of brain-age gap to brain-cognition gap, it is plausible that a potential disconnection between the brain’s biological state and observed cognitive state might emerge. If the brain predicts lower performance than is observed (i.e., a negative brain-cognition gap), it may be compensating for underlying issues not yet apparent through cognitive assessments. By this view, individuals with negative brain-cognition gaps should be less healthy than those whose brains predict higher cognitive function than their actual performance (i.e., a positive brain-cognition gap). Considering the significance of life-style and cardiovascular risk for maintaining healthy brain function (66, 67), the secondary aim of this study was to assess whether individuals with positive versus negative brain-cognitive gaps differ in terms of physical activity habits and cardiovascular risk factor burden.

The third aim of the current study is to uncover the contribution of dopamine (DA) integrity to brain-cognition gaps. Specifically, we investigate whether individuals with lower DA receptor levels exhibit higher functional variability, and in turn greater brain-cognition gaps. DA is a vital neuromodulator with critical implications for motor function, reward-seeking behavior, and various higher-order cognitive functions (68–75). Insufficient DA modulation can affect neurocognitive functions detrimentally (69, 74, 76–78). DA is important for enhancing the signal-to-noise ratio (SNR) of neural transmission (79, 80), and insufficient modulation may contribute to increased variability in blood-oxygen-level-dependent (BOLD) signal (81, 82) and reduced coherence in the functional connectome (83). Indeed, pharmacological studies have shown that DA depletion increases the variability of the BOLD signal, subsequently leading to less synchronized connectivity within resting state networks (84). Consequently, we expect individuals with inadequate DA levels to exhibit increased regional signal variability, a less unique functional connectome, and greater brain-cognition gaps.

Using data from the DopamiNe, Age, connectoMe, and Cognition (DyNAMiC) study (n =180, 20-79 years, 50% female) (54) (Fig. 1), we evaluated the predictive power of the functional connectome during resting state, movie-watching, and n-back tasks for two different cognitive domains: episodic memory and working memory. Based on recent research indicating that movie-watching enhances predictability by highlighting key features of FC (50), we hypothesized that FC during movie-watching would outperform FC during rest, and possibly during task, in predicting both cognitive measures. To achieve this objective, we employed a deep neural network approach, a specific subtype of artificial intelligence (AI), to predict cognitive scores from the functional connectome. Deep learning approaches overcome the limitation of predictive techniques that solely rely on linear associations between connectivity and behavioral phenotypes (17). Considering the importance of individual characteristics, such as age, in predicting behavior from FC (33), we conducted cross-validation of our model, initially derived from an age-heterogeneous sample, in an age-homogeneous sample (from the Cognition, Brain, and Aging (COBRA) study (85)). We subsequently investigated whether individuals with positive prediction gaps differ from those with negative gaps in terms of lifestyle and cardiovascular disease risk factors. Moreover, we tested the hypothesis of whether individuals with lower striatal dopamine D1-like receptor availability (D1DR), the brain’s most abundant DA receptor subtype, have a less distinctive FC pattern (i.e., more regional variability) and, in turn, a larger brain-cognition gap. Finally, we conducted a cross-validation of the link between DA and the prediction gap in an independent cohort with estimates of DA D2-like receptor (D2DR) availability (85).

Figure 1.

Table 1.

Results

AI-Driven Predictive Modeling of Cognition Scores from the Functional Connectome

We used fMRI data from the DyNAMiC project, in which each subject underwent rest, movie-watching, and working memory n-back tasks. These data were parcellated into 273 nodes (264 with 9 additional subcortical nodes) using a previously published whole-brain functional atlas (86). The averaged time series of 273 regions were subsequently correlated, to create the FC matrix for each participant and cognitive state (rest, movie-watching, and n-back). We found FC to be significantly correlated between cognitive states (r² for rest versus movie-watching =.81; r² for rest versus n-back =.76; r² for movie-watching versus n-back =.71), suggesting that a common functional architecture is present across various tasks.

We trained a convolutional neural network, DenselyAttention, derived from DenseNet (87) on FC matrices from each condition (resting state, movie-watching, and n-back) to generate cognition-specific prediction models of two memory domains (EM and WM). Model performance was quantified as the correlation between observed and predicted cognitive performance. Each model was then tested on all three conditions to examine the generalizability of each model across cognitive states. For example, the model trained on the resting state data (orange circles in Fig. 1) was used to predict EM scores using the test dataset derived from resting state (Fig. 1a), movie-watching, and n-back. Significance of our main predictions was assessed via linear correlations, and uncorrected p values are presented in Tables 1-2.

Table 2.

Resting state and movie-watching models outperform n-back in episodic-memory prediction, with resting state offering the best generalizability

We first started with all cases in which congruent conditions were used for model building and prediction. Only models derived from the rest and movie-watching datasets yielded significant predictions of EM (Figs. 1a-b and Table 1), with resting state yielding the best-performing model (r² =25.0%, p <.0001) followed by movie-watching (r² =24.0%, p <.0001) (Table 1). While there was no significant difference between resting state and movie watching in predicting episodic memory (Mann–Whitney U test, p =.17), both models yielded better EM prediction than n-back (Mann–Whitney U test, p <.0001; Fig. 1e).

To test the generalizability of these models, two types of validation analyses were performed: cross-condition and cross-data set. In the cross-condition analysis, models trained on one condition (e.g., rest) were tested on an incongruent condition (e.g., movie-watching, n-back; Table 1). Interestingly, the model trained on resting state significantly predicted EM when tested on movie-watching (r² =19.36%, p <.001) and n-back (r² =14.44%, p =.003) conditions. In contrast, models trained on movie-watching or n-back could not be generalized to other conditions, unable to significantly predict EM (p’s >.1), except for significant generalizability from the movie to the rest condition (Table 1).

In a cross-dataset validation analysis, the best-performing model from the age-heterogenous DyNAMiC dataset was tested on the corresponding condition in an age-homogeneous cohort from the COBRA dataset. By doing this, we found that the resting state model derived from DyNAMiC significantly predicted EM performance in the COBRA dataset (r² =5.76%, p <.0001).

Overall, our results suggest that both resting state and movie-watching FC effectively predict EM performance, although resting state models provide better generalizability across different cognitive states and datasets. This finding could further be replicated using the Schaefer Parcellation (Supplementary Figs. 1-2 and Tables S1-S2).

Next, we aimed to delineate the relative contributions of different brain regions for the best-performing model––the model trained on the “resting state data” in predicting episodic memory. Utilizing the Grad-CAM algorithm, saliency maps were generated for the 120 FC matrices used during the training of the winning model. An averaged and interpolated description of all saliency maps is depicted in Figure 1f. The saliency map highlights specific edges, especially within the default-mode network, edges between the default-mode network and subcortical areas, and edges between the default-mode and the cerebellar network. These edges, indicated by a salience intensity of ≥.5, exert a significant influence on the model (Fig. 1f).

Movie-watching and n-back models outperform resting state in working-memory predictions, with movie-watching offering the best generalizability

We next investigated whether the superiority of resting state in predicting EM was unique to this domain, considering that previous research demonstrated the advantages of task-based fMRI and naturalistic viewing in predicting fluid intelligence (88). To do so, we compared the predictive power of different states for WM, which is shown to be more directly associated with fluid intelligence compared to EM.

The model derived from the resting state failed to predict and generalize regarding WM (p’s >.10; Fig. 2a and Table 2). By contrast, models trained on movie-watching and n-back yielded significant predictions of WM (Figs. 2b-c and Table 2), with the movie-watching model emerging as the best-performing model (r² =32.49%, p <.0001) followed by n-back (r²=22.09%, p <.0001). While there was no significant difference between movie-watching and n-back in predicting WM (Mann–Whitney U test, p =.81), these models yielded better WM prediction than resting state (p <0.0001, Mann–Whitney U test; Fig. 2e).

Figure 2.

In cross-condition validation (Table 2), the movie-watching model yielded a significant WM prediction during both resting state (r² =17.64%, p <.0001) and n-back (r² =21.16%, p =.004). The model derived from n-back yielded a significant WM prediction during movie-watching (r² =21.16%, p =.0002), but not during the resting state (r² =4%, p =.12).

For cross-dataset validation, the best-performing model in the DyNAMiC dataset (i.e., movie-watching) was applied to predict WM to the COBRA dataset. However, since COBRA does not include a movie-watching paradigm, we applied the model to the n-back task in COBRA. This approach revealed that the model from the DyNAMiC movie-watching condition yielded a significant WM prediction in the COBRA n-back task (r² =22.09%, p < 0.0001).

Overall, our results suggest that movie-watching-based WM predictions generalize across different cognitive states and datasets. This finding could be further replicated using a different functional parcellation (Supplementary Figs. 1-2 and Tables S1-S2).

The Grad-CAM algorithm generated saliency maps of 120 FC maps from the DyNAMiC dataset employed for training. Figure 2f depicts an average of all Grad-CAM generated maps. The saliency map unveils that certain edges, specifically within network connectivity of task-positive regions such as the frontoparietal task control network, dorsal/ventral attention, visual, and subcortical networks as well as between-network connectivity. FC between task-positive dorsal and ventral attention networks, and between the DMN and the fronto-parietal network, contributed to the best-performing model derived from the movie-watching dataset.

The brain-cognition gap is related to physical activity levels and cardiovascular risk factors

Given the importance of lifestyle and cardiovascular risk for maintaining healthy brain function (66, 67, 89), we examined whether individuals with positive versus negative prediction gaps differ in physical activity habits, education, and Framingham cardiovascular disease (CVD) risk score. Our primary focus is on EM predictions derived from resting state data, as this was the common condition in the DyNAMiC and COBRA datasets. We computed the difference between predicted and observed EM scores to generate a gap score. A positive brain-cognition gap indicates that an individual’s brain predicted better-than-observed EM performance, whereas a negative gap indicates more compromised brains relative to actual performance. In the test sample of the DyNAMiC data (n =60) and the entire COBRA sample (n =177), we found that individuals with a negative brain-cognition gap exhibited lower levels of physical activity and higher CVD risk scores compared to those with positive gaps (Fig. 3). Moreover, individuals with negative brain-cognitive gaps were less educated compared to those with positive brain-cognitive gaps in the DyNAMiC dataset.

Figure 3.

Dopamine D1 and D2 receptor availability are associated with brain-cognition gaps

We hypothesized that inadequate DA levels might adversely affect neurocognitive function by reducing the neural signal-to-noise ratio, thereby resulting in a less unique functional connectome, consequently leading to a greater prediction gap. We therefore initially investigated the relationship between DA receptor levels and predictive gaps across different types of DA receptors in the DyNAMiC and COBRA samples.

In the DyNAMiC sample, we found a significant correlation between striatal D1DR and prediction gap (positive gap: r =−.46, p =.03; negative gap: r =.38, p =.01) (Fig. 4a), suggesting that lower D1DR is associated with greater brain-cognition gaps.

Figure 4.

We replicated our finding in the COBRA sample using D2-like receptor availability (D2DR), revealing a significant relationship between striatal D2DR and prediction gap (positive gap: r =−.38, p =.01; negative gap: r =.39, p =.004) (Fig. 4b). Our findings provide support the view that lower D1DR/D2DR is associated with larger brain-cognition prediction gaps.

Regional variability mediates the direct impact of dopamine on brain-cognition gaps

To evaluate whether functional variability mediated the relationship between D1DR and prediction gap connectivity, we conducted additional mediation analyses. We computed BOLD signal entropy, which estimates within-region signal variability (84), that was averaged across all regions of the parcellation. In the DyNAMiC sample, for the group exhibiting a negative brain-cognitive gap, we observed a significant indirect effect of D1DR on the gap mediated by entropy, β =2.41, 95% CI [0.89, 4.51], p <.0001, explaining 56.81% of the total effect of D1DR on the gap. The direct effect of D1DR, however, was not significant, β =1.83, 95% CI [−.86, 3.79], p =.19. Similarly, for those with a positive brain-cognition gap, an indirect effect of D1DR on the gap through entropy was observed, β =-6.78, 95% CI [−11.26, −3.05], p <.0001, accounting for 89.41% of the D1DR effect on the gap. The direct effect of D1DR was again non-significant, β =-0.803, 95% CI [−.48, 2.27], p =.26. These findings suggest that lower D1DR levels contribute to increased signal variability, which in turn leads to reduced specificity of FC and, consequently, a larger brain-cognition gap.

In the COBRA sample, within the negative gap group, we detected a significant indirect effect of D2DR on both the negative and positive gap groups through entropy. For the negative gap, β =2.18, 95% CI [0.01, 4.25], p =.04, accounting for 63.43% of the D2DR effect on the gap; and for the positive gap, β =−.2.18, 95% CI [−3.87, −.04], p =.04, accounting for 61.49% of the D2DR effect on the gap.

Discussion

Using deep learning models, we examined the predictive power of the functional connectome during various states (resting state, movie-watching, and n-back) on two different cognitive domains (EM and WM). Both rest and movie-watching states yielded significant predictions of EM, with the model derived from resting state generalized across states and datasets. Differences between the DyNAMiC and COBRA datasets make cross-dataset prediction a harder problem, as the age ranges of samples significantly vary, and prior studies highlight the importance of individual characteristics like age in predicting behavior from FC (33). Possibly as a consequence of these age differences, model performance decreased when using deep learning models to predict EM from COBRA data. Validation outcomes thus affirm that the models, particularly those constructed from rest data, are robust to the particulars of the dataset. The saliency map generated from the final layer of the deep learning model indicates that certain edges within DMN, between DMN and the subcortical network contributed significantly to the prediction. Building on a recent finding by Kurkela and Ritchy (90), our finding reveals that a fraction of the known-memory subnetwork within the DMN, as well as whole-brain multivariate pattern, which notably encompasses interactions of the DMN with other networks, such as the subcortical network, made a more substantial contribution to prediction. Importantly, this prediction generalizes across conditions and datasets, suggesting that features derived from resting state FC serve as a stable marker of individual differences in EM.

Our findings are in contrast to recent work suggesting that task paradigms, in general, and movie-watching, in particular, outperform resting state data in predicting cognitive performance (49, 50, 91). There are two possible explanations for this discrepancy. First, while previous studies have often demonstrated a superiority of task and naturalistic viewing over resting state in predicting fluid intelligence or WM (49, 50), there are fewer reports of FC predicting EM (e.g., (92, 93)) and, to our knowledge, no study, has compared rest and movie-watching. While we acknowledge that the resting state represents a complex amalgamation of cognitive, emotional, and perceptual processes (94), the good prediction power of the resting state may arise from the presence of mind wandering during rest, which is strongly related to EM (95, 96). EM plays a crucial role in generating mental content during mind wandering, especially episodes characterized by distinct times and locations (59, 97). Second, most previous studies utilized connectome predictive modeling (CPM), which relies on a linear relationship between FC and behavioral data. In contrast, deep learning makes no assumption about the nature of the relationship, and thus, a deep learning model may outperform CPM. Indeed, the predictive power in the current study is stronger than for CPM-based predictions reported before. For example, across various studies, the correlation between predicted and actual fluid intelligence typically hovers around 0.25 (98–100), while an average correlation of approximately 0.11 was reported across 58 different measures covering various domains (101). However, in the current study, we found that deep learning yields a stronger association (resting state prediction of EM: .31< r <.49; movie-watching prediction of WM: .42< r <.57), even across data-sets (cross-validation of EM: r =.24; cross-validation of WM: r =.47, between FC and cognitive data).

In contrast to the EM prediction, both n-back and movie-watching datasets yielded significant predictions of offline WM performance. Importantly, the models derived from movie-watching and n-back outperformed the resting state in WM prediction. These differences in model accuracy when predicting the same target behaviors (i.e., WM) suggest the presence of trait-state interactions. Specifically, movies and n-back enhance individual differences in WM-relevant connection. Indeed, we found that several WM-related networks (102–105), including the fronto-parietal, the salience, and the dorsal/ventral attention networks contribute to prediction. Additionally, previous research showed that movie-watching alters the propagation of activity across cortical pathways (106), particularly within and between regions involved in audiovisual processing and attention. These alterations lead to a less segregated and more integrated network organization (107). Similarly, the n-back task has been associated with increased integration of task-positive cortico-cortical connectivity (105, 108) and striato-cortical connectivity (103). Taken together, our results confirm previous findings that movie watching paradigm is a great condition for studying individual differences in various cognitive domains. Nonetheless, if movie watching paradigm is not feasible/ available, the resting state still provides a robust means of studying individual differences, particularly in self-referential domains, such as EM.

Our study introduces a novel deep neural network architecture that features dense connections and incorporates an attentional mechanism. We found that the current method outperformed widely used approaches, such as kernel regression (110) and connectome predictive modeling (17), in predicting cognitive data. Our findings contrast with previous work showing similar performance between kernel regression and various DMN architectures for predicting behavioral and demographic variables (29). Unlike the BrainNet convolutional neural network, which focuses on staged transformations, our densely connected model promotes extensive feature reuse, possibly leading to more robust feature extraction. That is, the model explicitly uses the Enhanced Residual Block (ERB) and High-Frequency Attention Block (HFAB), enhancing the focus on relevant features and potentially boosting performance. The architecture includes multiple layers and blocks, balancing depth and computational efficiency.

Applying two different parcellation methods (86, 111) to the DyNAMiC data indicated that parcellation resolution does not significantly impact model performance (see Figs. S1-S2 and Tables S1-S2). However, improvements in registration and parcellation, such as individualized parcellations and areal features for alignment, may enhance the delineation of functionally homogeneous regions (112), improving the predictive performance of deep learning models. Therefore, the impact of alignment and parcellation on predictive model performance remains a key area for future research. Additionally, the performance of our approach still needs to be examined on a large-scale publicly available neuroimaging dataset for predicting other types of cognitive data.

We found a significant link between the brain-cognition prediction gap, life-style, and risk factors for vascular disease, such that individuals with a negative gap exhibited lower levels of physical activity and higher cardiovascular risk scores compared to those with a positive gap. This finding was consistent across both age-heterogeneous and age-homogenous older samples. Brain-cognition prediction gap could serve as a biomarker for identifying individuals at risk (e.g., individuals with a negative gap). Previous studies suggest that the brain age prediction gap is associated with cognitive aging (62), some aspects of physiological aging (61), as well as aging in other organs (113), and even mortality in older age (61). However, a recent study revealed that brain age accounts for only a small portion of cognitive decline compared to chronological age (65), suggesting that cognitive prediction might be more informative. Our findings build upon this concept by extending the brain-age gap to behavioral variables, demonstrating that the brain-cognition gap could provide insights regarding physical activity status, education, and cardio vascular risk – key factors contributing to cognitive reserve (114–117).

Critically, we found that D1DRs and D2DRs were strongly associated with the brain-cognition gap, such that lower dopamine receptor levels were associated with greater gaps. More specifically, in two independent samples, we discovered greater correspondence (i.e., near zero in the gap) between brain function and cognition in individuals with higher D1DR/D2DR whereas lower correspondence (i.e., significantly different gap from zero) was found in individuals with lower D1DR/D2DR availability. Previous computational models proposed that DA modulates neuronal gain, which improves SNR in neural processing, contributing to more coherent activity across large-scale networks (e.g., balanced integration and segregation (83)). Past studies also showed that lower D1DR contributed to more BOLD variability in the subcortical area (81) and less functional segregation of the striatum (118) and the large-scale networks in aging (83), possibly due to increased noise (lower SNR). In support of this notion, we found for the first time that regional variability, estimated using entropy, mediated the impact of DA on the brain-cognition gap. Although the cross-sectional nature of our data warrants caution, this novel finding suggests that lower DA integrity relates to BOLD variability, which in turn is associated with a larger brain-cognition gap.

In summary, our findings reveal that while tasks like movie-watching excel in predicting WM performance, there are features during rest that predict internally-oriented mind-wandering-type tasks, such as EM. Additionally, individuals whose brains predict poorer cognitive performance (i.e., negative gap) exhibit lower physical activity and higher cardiovascular risk compared to those whose brains predict higher cognitive function than their actual performance (i.e., positive gap). This finding suggests that our prediction model offers a novel marker to identify individuals at risk of compromised brain maintenance. Furthermore, individuals with lower DA showed less accurate cognitive prediction (larger brain-cognition gap) due to increased BOLD variability and less unique and cohesive FC.

Materials and methods

Participants

All participants provided written informed consent and studies were conducted in accordance with the Declaration of Helsinki and approved by the Regional Ethical Board and the local Radiation Safety Committee (reference numbers: 2012-57-31M; 2017-404-32M).

This study used data from DyNAMiC (54), which is a longitudinal study with a focus on changes in the brain connectome and the D1DR system. At baseline,180 participants (20-79 years, 50% female) across the adult lifespan underwent all tests between 2017 and 2020 (54) (Fig. 5). Rigorous exclusion criteria were used to recruit a sample without neurological conditions and medical treatments affecting brain functioning and cognition. Exclusion criteria included brain injury or neurological disorder, dementia, neurodevelopmental disorder, psychiatric diagnosis, psychopharmacological treatment, history of severe head trauma, substance abuse or dependence, and illicit drug use. Individuals with other chronic or severe medical conditions (e.g., cancer, diabetes, and Parkinson’s disease) were also excluded. Here, we only use data from the baseline measurement.

Figure 5.

We used a separate sample as a testing dataset from the COBRA study (85) (Fig. 5). COBRA is a longitudinal aging study in which 181 healthy individuals (64-68 years, 45% female) underwent baseline assessments of brain, cognition, health, and lifestyle during 2012–2014 (85). Exclusion criteria at baseline included traumatic brain injury, stroke, dementia, intellectual disability, epilepsy, psychiatric and neurological disorders, diabetes, and cancer medications, severe visual or auditory impairment, claustrophobia, and poor Swedish language skills. In the current study, we used data from 177 subjects from COBRA who underwent both MRI and PET examinations at baseline (78, 85, 102, 103, 119).

Cognitive Measures

The same cognitive test battery was used in DyNAMiC and COBRA (Nevalainen et al 2015; Nordin et al, 2022), and assessed two cognitive domains: episodic memory (EM) and working memory (WM). Each domain was assessed using three separate tests, including letter-, number-, and figure-based material, respectively. Participants completed all tasks on a computer and responded by either typing in words or numbers; using the computer mouse; or pressing keys marked by different colors. Each test included initial practice runs, after which testing followed across several trials. For each of the three tests within a domain, scores were summarized across trials for a measure of overall performance. Finally, these sum scores were z-standardized and averaged to form one composite score per cognitive domain (T score: mean =50; sd =10) representing within-domain performance (e.g., EM or WM performance). Missing values were replaced by the average of the available observed scores.

Episodic Memory (EM)

Tests of EM included word recall, number-word recall, and object-location recall. In word recall, participants viewed 16 Swedish concrete nouns, successively appearing on the computer screen. Each word was presented for 6 s, with an inter-stimulus interval (ISI) of 1 s. Directly following encoding, participants reported as many words as they could using the keyboard. Two trials were completed, with a combined maximum score of 32. In number-word recall, participants encoded pairs of 2-digit numbers and concrete plural nouns (e.g., 46 dogs). During encoding, eight number-word pairs were presented, each for 6 s, with an ISI of 1 s. Directly after encoding, nouns were presented again, in re-arranged order, and participants had to report the 2-digit number associated with each presented noun (e.g., How many dogs?). This test included two trials with a total combined maximum score of 16. The third test assessed object-location memory. Participants viewed a 6 × 6 square grid in which 12 objects were consecutively shown at distinct locations. Each object-position pairing was displayed for 8 s, with an ISI of 1 s. Directly following encoding, all objects were simultaneously displayed next to the grid for the participant to place in their correct position within the grid. If unable to recall the correct position of an object, participants had to guess to the best of their ability. Two trials of this test were completed, making the total combined maximum score 24.

Working Memory (WM)

Working memory was examined with three tests letter updating, number updating, and spatial updating. These tests differed from the working memory n-back task performed during fMRI scanning. In letter-updating, capital letters (A–D) were consecutively presented on the computer screen, with participants instructed to always keep the three final letters in memory. Each letter was presented for 1 s, with an ISI of 0.5 s. When prompted, at any given moment, participants provided their responses by typing in three letters using the keyboard and provided a guessing-based answer if they failed to remember the correct letters. Four practice trials were followed by 16 test trials consisting of either 7-, 9-, 11- or 13-letter sequences. The combined maximum number of correct answers across trials was 48 (16 trials × 3 reported letters = 48). The number-updating test followed a columnized 3-back design. Three boxes were presented next to each other on the screen throughout the task, in which a single digit (1–9) was sequentially presented from left to right for 1.5 s with an ISI of 0.5 s. During an ongoing sequence, participants had to judge whether the number currently presented in a specific box matched the last number presented in the same box (i.e., appearing three numbers before). For each presented number they responded yes/no by pressing assigned keys (“yes” = right index finger; “no” = left index finger). Two practice trials were followed by four test trials each consisting of 30 numbers. The combined maximum number of correct answers across trials (after discarding responses to the first three numbers in every trial – as these were not preceded by any numbers to be matched with) was 108 (27 numbers × 4 trials). In spatial-updating, participants viewed three 3 × 3 square grids presented next to each other on the computer screen. At the start of each trial, a blue circular object was displayed at a random location within each grid. After 4 s, the circular objects disappeared, leaving the grids empty. An arrow then appeared below each grid, indicating that the circular object in the corresponding grid was to be mentally moved one step in the direction of the arrow. The arrows appeared stepwise from left to right grids, each presented for 2.5 s (ISI = 0.5 s). Prompted by three new arrows, participants mentally moved the circular objects one more time, resulting in each circular object having moved two steps from its original location at the end of each trial. Participants indicated which square the circular object in each grid had ended up in using the computer mouse. If unsure, they were instructed to guess. The test was performed across 10 test trials, preceded by five practice trials. The combined maximum number of correct location indications was 30.

Measure of Physical Activity and Cardiovascular Disease Risk

Physical Activity

An extensive battery of self-rating questionnaires was administered in DyNAMiC and COBRA (54, 85). Participants were asked to indicate the frequency (number of hours during a typical summer week; options: 1-14 h with 1-h increments, or 15+ hrs) and the intensity (how physically demanding on a scale from 1 =“not at all” to 5 =“extremely”) by which they typically engage in a selection of activities relevant to life in northern Sweden. These included 15 specific activities. For the purpose of the present study, we focused on physical activities, and in particular on those activities that are purely physical and that individuals are sufficiently engaged in (i.e., physically demanding >= 2). Each of these activities was performed by at least 20% of the participants at least once a week; e.g., walking, bicycling, jogging, strength training, household tasks, and daily work-related activities. We computed physical activity frequency (sum hrs/week) to generate physical activity scores accordingly.

Cardiovascular Disease risk

The risk of cardiovascular disease was determined via a multivariable score, according to the algorithm developed in the Framingham Heart Study (120). Variables include age, sex, hypertension diagnosis, systolic blood pressure, body-mass index, smoking, and diabetes mellitus. The risk estimates were derived using an algorithm proposed by D’Agostino et al. (120), which employs Cox proportional-hazard regression models to predict the probability of developing any form of cardiovascular disease within 10 years:

With S₀(t) being the baseline survival at follow-up t (t =10 years), βi the estimated regression coefficient, X_i the log-transformed value of the ith risk factor, X_i the corresponding mean, and m the number of risk factors included. Baseline survival, means, and regression coefficients were taken from the original algorithm, with DyNAMiC and COBRA participants’ risk variables inserted to compute the final scores. Risk score calculators can be found at framinghamheartstudy.org.

Image Acquisitions

Structural, functional, and neurochemical brain measures were acquired using MRI and PET at Umeå University Hospital in northern Sweden. For both DyNAMiC and COBRA, all MRI data were collected using a 3T Discovery MR750 MRI system (General Electric, Healthcare, Illinois, USA) equipped with a 32-channel phased-array head coil. PET was conducted in 3D mode with a Discovery PET/CT 690 (General Electric, WI, United States) to assess whole-brain D1DR with [¹¹C]SCH23390) and D2DR with [¹¹C]raclopride at rest in DyNAMiC and COBRA, respectively. Comprehensive descriptions of MRI, PET, and cognitive testing protocols are given elsewhere (54, 85). In this study, we primarily focus on those data directly pertinent to the current investigation.

Functional MRI

For the DyNAMiC dataset, high-resolution anatomical T1-weighted images were collected using a 3-dimensional (3D) fast spoiled gradient-echo sequence with acquisition parameters of 176 sagittal slices, thickness =1 mm, TR =8.2 ms, TE =3.2 ms, flip angle =12°, and a field of view (FOV) =250×250 mm. Whole-brain functional images were acquired during resting state, naturalistic viewing, and an n-back WM task. Functional images were acquired using a T2*-weighted single-shot echo-planar imaging (EPI) sequence, with 330 volumes collected over 12 min. The sequence provided 37 axial slices, slice thickness =3.4 mm, 0.5 mm spacing, TR =2,000 ms, TE =30 ms, flip angle =80°, and FOV =250×250 mm. Ten dummy scans were collected at the start of the sequence. During the resting state, participants were instructed to keep their eyes open and focus on a white fixation cross on a black background displayed on a computer screen through a tilted mirror attached to the head coil. WM was assessed in the scanner (12-min) with a numerical n-back task which consisted of blocks of 1-back, 2-back, and 3-back (102, 103). During movie-watching, participants viewed and listened to a 12-minute video consisting of selected and chronologically ordered sections from the Swedish movie Cockpit (121). Participants were instructed to view the movie attentively and answered a short multiple-choice questionnaire about the movie after the scanning session.

In COBRA, data were collected using identical scans for resting state and n-back WM. However, the resting state scan was shorter, lasting only 6 minutes (73, 103).

Functional Connectivity Analysis

Functional data from all conditions (i.e., rest, movie, n-back) were pre-processed using the Statistical Parametric Mapping software package (SPM12). The functional images were first corrected for slice-timing differences and in-scanner motion followed by registration to anatomical T1 images. Distortion correction was performed using subject-specific and T1 co-registered field maps. The functional time series were subsequently demeaned and detrended followed by simultaneous nuisance regression and temporal high-pass filtering (threshold at 0.009 Hz) to not re-introduce nuisance signals (122). The nuisance regression model included mean cerebrospinal and white-matter signal and their derivatives, 24-motion parameters (123), a binary vector flagging motion-contaminated volumes exceeding framewise displacement (FD) of 0.2 mm (124), in addition to an 18-parameter RETRICOR model (125, 126) of cardiac pulsation (up to third-order harmonics), respiration (up to fourth-order harmonics), and first-order cardio-respiratory interactions estimated using the PhysIO Toolbox v.5.0 (127). Regression models for n-back included an additional set of finite impulse response (FIR) task regressors (128) to avoid false positive connectivity due to task-evoked activations (129). The FIR regression approach involved fitting mean cross-block responses for each time point within a time-locked window of equal duration to each task block (27 blocks of 20 s), extended by an additional 18 s following each block to account for the duration of the hemodynamic response function (HRF). Given that this approach linearly fits a set of binary task regressors for each time point, it is nearly identical to subtracting the mean task response, with the difference being that FIR regression is better able to handle overlapping task responses and differences in the shape of the HRF (129). The nuisance-regressed images were subsequently normalized to sample-specific group templates (DARTEL) (130) for each dataset respectively, followed by spatial smoothing using a 6-mm FWHM Gaussian kernel to mitigate DARTEL-induced aliasing and affine-transformed to stereotactic MNI space (ICBM152NLin2009) (131, 132). Functional images from both DyNAMiC and COBRA were preprocessed according to the steps described above with the only exception that the RETRICOR parameters were excluded from the regression models in COBRA due to technical issues related to the respiration and cardiac traces.

Graph Construction

For all fMRI conditions, functional time series were averaged from 273 cortical and subcortical regions, represented by 5-mm radius spheres, based on a widely employed FC parcellation (86). In addition to 264 regions reported in the Power parcellation, we added 9 additional regions including some subcortical regions, such as putamen, caudate, and anterior and posterior hippocampus, identified using independent component analysis (8, 133). These regions were categorized into 14 resting state networks according to a consensus partition (86). To mitigate sampling from non-gray matter voxels, each parcel underwent erosion by a permissive gray matter mask (eroding voxels <.1% threshold). The averaged time series were then subjected to Pearson’s correlations, followed by Fisher’s r-to-z transformation, resulting in the creation of a 273×273 adjacency matrix for each participant, with coefficients along the main diagonals set to zero.

To further investigate the impact of network parcellation, we replicated our prediction analysis (Supplementary Material, Figs S1-S2, and Table S1-S2) using Schaefer parcellation (111), which entails 300 cortical and subcortical regions.

Deep Neural Network Model

Based on convolutional neural networks, deep learning is an advanced form of artificial intelligence that uses multiple layers of “hidden” neural networks. Deep learning methodologies are capable of automatically identifying complex patterns and representations directly from raw data using these multi-layered networks, thus eliminating the need for explicit feature engineering or manual intervention (134). The success of the training and learning phases depends on the model’s ability to process high-dimensional input data, extracting meaningful features from complex data. This is done while managing the number of trainable parameters, which are crucial for automated feature learning during the construction of the model.

In this study, the inputs to our deep learning models were subject-specific FC maps with a matrix size of 273×273 (e.g., Fig. 6a). We generated different versions of each FC map by replacing portions of networks. For example, we relocated the DMN network toward the last nodes, which were assigned as DAN, cerebellar, subcortical, and uncertain in Figure 6a. This approach allowed us to create a diverse set of FC matrices for each individual, each reflecting a different composition of edges and neighboring while maintaining the linear relationships exonerated in the original data. Consequently, we augmented the dataset by producing a total of 3,600 FC maps from the initial set of FC maps.

Figure 6.

Following the DenseNet framework (87), we incorporated the Enhanced Residual Block (ERB) and High-Frequency Attention Block (HFAB) into the dense layers (Fig. 6b), termed DenselyAttention, to facilitate feature reuse in each layer. DenseNet (87) diverges from traditional methods like deepening layers or widening network structure by focusing on feature reuse and bypass settings. This results in fewer parameters than similar dense networks such as ResNet (135), enhances feature reuse, improves feature propagation, makes training easier, and reduces issues of gradient vanishing and model degradation.

The architecture of DenseNet is characterized by its dense connectivity pattern, which entails direct connections from each layer to all subsequent layers within its dense block (as illustrated in Fig. 6b). This design ensures that every layer has access to the feature maps generated by preceding layers, thereby facilitating a seamless and efficient gradient flow throughout the network. In essence, the knowledge acquired at each layer is propagated forward, enabling the model to effectively capture intricate patterns and dependencies within the data, ultimately enhancing its ability to learn and generalize (87). Additionally, dense blocks provide a regularizing effect, reducing overfitting, particularly on tasks with smaller training datasets (87). This is suitable for this study which includes relatively small samples. Each sequence combines operations of batch normalization (BN), rectified linear unit (ReLU), and a 3×3 convolution (Conv). Batch normalization can effectively prevent overfitting, as described by equation 1:

Where μ_β, mean; σ_β, standard deviation; ɛ, random noise; α and β are adaptable variables in training. We utilized the Rectified Linear Unit (ReLU) activation function (136), which activates neurons by directly outputting the input if it is positive, or zero otherwise, as outlined in equation 2.

Where x is the input to a neuron. ReLU benefits the performance of networks with dense layers by decreasing the computation and selectively optimizing parameters.

Four dense blocks facilitate a stepwise down-sampling in the network. These blocks are connected with transition layers, which consist of a 1×1 convolutional layer followed by a 2×2 average pooling layer.

Additionally, ERB and HFAB pairs are introduced for targeted high-frequency features and residual block enhancement. The ERB-HFAB pairs are stacked sequentially at the beginning of each dense block loop (Fig. 6b), with ERB and HFAB having 16 feature maps.

High-Frequency Attention Block (HFAB)

Our approach to attentional mechanisms, specifically the HFAB, introduces a sequential attention branch inspired by edge detection. This branch rescales each position based on its neighboring pixels, efficiently focusing on high-frequency areas. The HFAB employs a 3×3 convolution to enhance both the receptive field and computational efficiency. Batch normalization is seamlessly integrated into the attention branch, introducing global statistics during inference without additional computational cost.

Enhanced Residual Block (ERB)

We present ERB as an alternative to the traditional residual block. As illustrated in Figure 6b, ERB comprises a re-parameterization block (RepBlock) and a ReLU. In the training stage, the RepBlock utilizes a 1×1 convolution to either expand or contract the number of feature maps, employing a 3×3 convolution to extract features in a higher-dimensional space. Furthermore, two skip connections are integrated to mitigate training complexities. During inference, all linear transformations can be unified, facilitating the conversion of each RepBlock into a singular 3×3 convolution. Essentially, ERB capitalizes on the advantages of residual learning.

We implemented model training using Tensorflow 2.11.0 (137) and Keras 2.11.0 (138) as programming interfaces and trained on a fifth-generation MacBook Pro (Apple M1 MAX silicon chips, 10-core CPU, 24-core GPU, 16-core Neural Engine, 64 GB memory). For regression tasks, selecting an appropriate loss function is crucial for guiding the optimization process and ensuring accurate predictions. In this study, we opted for the mean squared error (MSE) loss function due to its suitability for regression problems. To minimize the loss function, we trained the network using the stochastic gradient descent (SGD) optimizer with a learning rate of 8e^-5 and a Nesterov momentum of 0.9 (139). The number of epochs was set to 100, and the batch size was set to 74. We also added dropout (140) after each convolutional layer, except the first one, with a rate of 0.15.

We conducted training on distinct, identical datasets extracted from DyNAMiC, each comprising FC maps generated from 120 subjects. To prepare each training dataset, we randomly shuffled data for training and validation patches, allocating 80% for training and 20% for validation. For testing, we utilized all FC maps of 60 subjects from the same sample (the age-heterogeneous DyNAMiC study) enabling us to assess model performance through three-fold cross-validation (Fig. 5). Each cross-validation fold was a new training with unseen validation set to the model. Based on its performance on the testing dataset, we selected and employed the winning model for all subsequent analyses. This model, which demonstrated the best performance on the testing dataset, underwent cross-validation in an independent sample from the age-homogeneous COBRA study.

To explore and visually represent the crucial features of the deep learning models contributing to the prediction of cognitive scores, we used the Grad-CAM (Gradient-weighted Class Activation Mapping) technique (141). This method interprets the model’s decisions by highlighting the regions of the input image with the most significant impact on the model’s output. Grad-CAM conducts a backward pass to compute the gradients of the target class score with respect to the feature maps of the final convolutional layer. We present the average heatmaps calculated for all input data to the model (141).

Positron Emission Tomography (PET)

The scanning sessions started with a 5-minute low-dose helical CT scan (20 mA, 120 kV, 0.8 s per revolution), obtained for attenuation correction. During scanning, a thermoplastic mask was attached to the bed surface to minimize head movement.

In DyNAMiC, a 60-minute scan was performed following 350 MBq (337 ± 27 MBq) in list-mode format. Offline re-binning of list-mode data was conducted to achieve a total of 49 frames with increasing length. In COBRA, a 55-min, 18-frame dynamic PET scan was acquired during rest following intravenous bolus injection of approximately 250 MBq 11C-raclopride (264 ± 19 MBq). For both studies, attenuation- and decay-corrected images (47 slices, field of view = 25 cm, 256×256-pixel transaxial images, voxel size = 0.977× 0.97×3.27 mm³) were reconstructed with the iterative VUE Point HD-SharpIR algorithm (GE; 6 iterations, 24 subsets, 3.0 mm post filtering; full-width-at-half-maximum (FWHM): 3.2 mm). The estimation of receptor availability, or binding potential relative to non-displaceable binding (BP_ND), was carried out following previously described procedures with the cerebellum as a reference region (74). PET images were motion-corrected and co-registered with the structural T1-weighted images from the corresponding session using Statistical Parametric Mapping software (SPM12, Wellcome Department of Imaging Science, Functional Imaging Laboratory, London, UK). Motion-corrected PET data were resliced to match the spatial dimensions of MR data (1 mm³ isotropic, 256×256×256). The mean of the first five frames was used as a source for co-registration. In DyNAMiC, frame-to-frame head motion correction, with translations ranging from 0.23 to 4.22 mm (mean ± sd = 0.95 ± 0.54 mm), revealed a trend-level difference across age-groups (age <40 and age ≥ 40 years), as determined by Student’s t-test (t =2.0, p =.047; mean ± sd for younger individuals = 1.07 ± 0.52, mean ± sd for older individuals =0.90 ± 0.55). Partial-volume-effect (PVE) correction was achieved using the symmetric geometric transfer matrix (SGTM) method for regional correction, implemented in FreeSurfer (142). An estimated point-spread-function of 2.5 mm FWHM was utilized. Regional estimates of BP_ND were calculated within the automated FreeSurfer segmentations employing the simplified reference tissue model (SRTM (143)). In the current study, we focused on the striatal BP_ND, calculated as an average of BP_ND across the caudate and putamen.

The direct impact of dopamine on brain-cognition gaps through mediation analysis

To evaluate whether functional variability mediates the relationship between D1DR and prediction gap connectivity, we conducted additional mediation analyses. We first computed entropy, which estimates within-region signal variability (84), and then averaged this measure across all regions in the parcellation. Following the mediation analysis framework proposed by Baron and Kenny (144), our goal was to determine whether the association between D1/D2 receptors and brain-cognition gaps is mediated by regional variability (entropy) or if the indirect effect exceeds the direct association between D1/D2 receptors and brain-cognition gaps. To assess the statistical significance of this mediation effect, we employed the bootstrapping method as outlined by Preacher and Hayes (145).

Statistical significance analysis

Statistical analyses were carried out using SPSS (IBM Corp., V24.0.0, Armonk, NY, USA), MATLAB (The MathWorks Inc., V9.13.0 (R2022b), Natick, MA, USA), and GraphPad Prism (GraphPad Software, Inc. V5.01, CA, USA). We performed partial correlations between predicted and actual scores, as well as linear regression analyses. To investigate the relationship between generated gap variables and DA receptor availability, we controlled for age (in DyNAMiC) using partial correlation. The Mann–Whitney U test was used to calculate the mean differences in prediction accuracy. The level of statistical significance was set at p ≤.05.

Supplementary Materials

Figure S1.

Table S1.

Figure S2.

Table S2.

Data availability

The scripts used for developing the model are available at https://github.com/MorEsm/AI-based-Prediction-of-Cognitive-Function. (We are working on this channel to provide open-source scripts).

Acknowledgements

This work was funded by the Swedish Research Council (grant number 2016-01936 to A.S.), Knut and Alice Wallenberg Foundation (Wallenberg Fellow grant to A.S.), bank of Sweden (RJ, grant number P20-0515 to A.S.), StratNeuro grant at Karolinska Institute (A.S.). Morteza Esmaeili and Erin B. Bjørkeli were supported by the Southern Eastern Norway Regional Health Authority (Helse Sør-Øst RHF, HSØ, ME: grant number HSØ 2018047; E.B.B. HSØ 2021023).

Additional information

Author Contributions

M.E., E.B.B., and A.S. (and COBRA PIs) study design, method development, data measurement, and analysis. All authors participated in manuscript writing or editing.