Abstract
For decades, neuroscientists have been on a quest to search for a biomarker that can help capture age-related cognitive decline. One well-known candidate is Brain Age, or a predicted value based on machine-learning models built to predict chronological age from brain MRI data. Here we aim to formally evaluate the utility of Brain Age as a biomarker for capturing cognitive decline. Using 504 aging participants (36-100 years old) from the Human Connectome Project in Aging, we created 26 age-prediction models for Brain Age based on different combinations of MRI modalities. We first tested how much Brain Age from these age-prediction models added to what we had already known from a person’s chronological age in capturing cognitive decline. Based on the commonality analyses, we found a large degree of overlap between Brain Age and chronological age, so much so that, at best, Brain Age could uniquely add only around 1.6% in explaining variation in cognitive decline. Next, the age-prediction models that performed better at predicting chronological age did NOT necessarily create better Brain Age for capturing cognitive decline over and above chronological age. Instead, better-performing age-prediction models created Brain Age that overlapped larger with chronological age, up to around 29% out of 32%, in explaining cognitive decline, thus not improving the models’ ability to capture cognitive decline. Lastly, unlike Brain Age, Brain Cognition, or a predicted value based on machine-learning models built to predict cognitive abilities from brain MRI data, provided much higher unique effects. Brain Cognition added over 11% to explain variation in cognitive decline beyond chronological age, leading to around a 1/3-time improvement of the total variation explained. Accordingly, while demonstrating the limited utility of Brain Age, we provided a solution to improve our ability to use brain MRI data as a biomarker for cognitive decline.
eLife assessment
This useful manuscript challenges the utility of current paradigms for estimating brain-age with magnetic resonance imaging measures, but presents inadequate evidence to support the suggestion that an alternative approach focused on predicting cognition is more useful. The paper would benefit from a clearer explication of the methods and a more critical evaluation of the conceptual basis of the different models. This work will be of interest to researchers working on brain-age and related models.
Introduction
Older adults often experience declines in several cognitive abilities such as memory, attention and processing speed, collectively known as fluid cognition (cognitionfluid) (Salthouse, 2019; Weintraub et al., 2014). Having objective biomarkers to capture these declines in cognitionfluid would give researchers and clinicians a tool to detect early cognitive impairments, monitor treatment/intervention efficacy and forecast cognitive prognosis (Frisoni et al., 2017). Over the past decade, Brain Age (Franke et al., 2010) has emerged as a potential biomarker to capture declines in cognitionfluid (Cole, 2020; Cole et al., 2018; Liem et al., 2017; Richard et al., 2018; Wrigglesworth et al., 2022; see review Boyle et al., 2021). Yet, to justify the use of Brain Age as an informative biomarker for cognitionfluid, we still need to address many outstanding issues.
First, to what extent does having information on Brain Age improve our ability to capture declines in cognitionfluid beyond knowing a person’s chronological age? To compute Brain Age, researchers first built a prediction model that predicts the chronological age based on a person’s brain neuroimaging data (Baecker et al., 2021). They then apply this prediction model to an unseen individual, not part of the model-building process. Brain Age is the predicted value of this model. Accordingly, by design, Brain Age is tightly close to chronological age. Because chronological age usually has a strong relationship with cognitionfluid, to begin with, it is unclear how much Brain Age adds to what is already captured by chronological age.
Note researchers often subtract chronological age from Brain Age, creating an index known as Brain Age Gap (Franke & Gaser, 2019). A higher value of Brain Age Gap is thought to reflect accelerated/premature aging. Yet, given that Brain Age Gap is calculated based on both Brain Age and chronological age, Brain Age Gap still depends on chronological age (Butler et al., 2021). If, for instance, Brain Age was based on prediction models with poor performance and made a prediction that everyone was 50 years old, individual differences in Brain Age Gap would then depend solely on chronological age (i.e., 50 minus chronological age). In addition to the dependency on chronological age, Brain Age is often overestimated in younger, but underestimated in older, individuals (Le et al., 2018). There are many adjustments proposed to correct for this estimation bias, but the outcomes tend to be similar, if not identical, to each other (Beheshti et al., 2019; de Lange & Cole, 2020; Liang et al., 2019; Smith et al., 2019). These adjustments can be applied to Brain Age and Brain Age Gap, creating Corrected Brain Age and Corrected Brain Age Gap, respectively. Corrected Brain Age Gap in particular is viewed as being able to control for both age dependency and estimation biases (Butler et al., 2021). Here we tested the ability of different Brain Age calculations in capturing cognitionfluid, over and above chronological age.
Second, do better-performing age-prediction models correspond to the improvement in an ability to capture cognitionfluid? Over the past decades, there has been a race to improve the performance of age-prediction models to be better at predicting chronological age, for instance, by combining different MRI/neuroimaging modalities features (Cole, 2020; Engemann et al., 2020; Liem et al., 2017) or by applying more sophisticated machinelearning algorithms (Baecker et al., 2021; Jonsson et al., 2019; Zhao & Zhao, 2021). However, the improvement in predicting chronological age may not necessarily make Brain Age to be better at capturing cognitionfluid. If, for instance, the age-prediction model had the perfect performance, Brain Age Gap would be exactly zero and would have no utility in capturing cognitionfluid beyond chronological age. Few, if any, studies have examined the performance of age-prediction models as a function of their ability to capture cognitionfluid.
Third and finally, can we further improve our ability to capture the decline in cognitionfluid by using, not only Brain Age and chronological age, but also another biomarker, Brain Cognition? Analogous to Brain Age, Brain Cognition is defined as a predicted value from prediction models that predict cognitionfluid based on a person’s brain data (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Rasero et al., 2021; Sripada et al., 2020; Tetereva et al., 2022; for review, see Vieira et al., 2022). Age-related cognitive decline is not only related to the changes in age, but also to the changes in cognition. While Brain Age is about the former, Brain Cognition is about the letter. Brain Age and Brain Cognition are found to capture some overlapped variation in cognitive abilities (Jiang et al., 2022). Despite the overlap, there might still be unique effects contributed by each biomarker that, when combined, may collectively help capture cognitionfluid.
Our study set out to test the utility of Brain Age as a biomarker for capturing decline in cognitionfluid. Using aging participants (36-100 years old) from the Human Connectome Project in Aging, we computed different Brain Age indices (including Brain Age, Brain Age Gap, Corrected Brain Age and Corrected Brain Age Gap) and Brain Cognition from prediction models based on different sets of MRI features. These MRI features covered task, resting-state and structural MRI, creating 26 prediction models in total. We, then, tested the biomarkers’ ability to explain cognitionfluid in unseen participants. To test the ability of Brain Age by itself, we applied simple regression models with each Brain Age index as a sole regressor to explain Cognitionfluid. Next, to test the unique effects of Brain Age in explaining Cognitionfluid beyond chronological age, we applied multiple regression models with both each Brain Age index and chronological age as regressors to explain Cognitionfluid. To reveal how much chronological age and Brain Age had in common in explaining Cognitionfluid (i.e., common effects), we then applied the commonality analysis (Nimon et al., 2008) to these multiple regression models. Additionally, given that certain sets of MRI features led to prediction models that were better at predicting chronological age, we also examined if these better-performing age-prediction models improved the ability of Brain Age in explaining cognitionfluid. Finally, to test whether Brain Cognition could further improve the ability to capture cognitionfluid, beyond Brain Age and chronological age, we added Brain Cognition to the multiple regression models and tested its unique and common effects in explaining Cognitionfluid.
Results
Relationship between chronological age and Cognitionfluid
Figure 1a shows the negative relationship between chronological age and Cognitionfluid (r(502) = −.57, p < .001, R2 = .32). Older individuals tended to have a lower Cognitionfluid score.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig1.tif/full/max/0/default.jpg)
Relationship between chronological age and Cognitionfluid (a) and predictive performance of prediction models using Brain MRI from different sets of MRI features to predict chronological age (b) and Cognitionfluid (c).
Note we only provided the scatter plots between observed and predicted values in the test sets from the best prediction models for each target here. See Supplementary Figures 1 and 2 for the scatter plots from other prediction models.
Predictive performance of prediction models for Brain Age and Brain Cognition
Figure 1b and 1c show the predictive performance of different sets of brain MRI features in predicting chronological age and Cognitionfluid, respectively. For age prediction, the top-four similarly performing models were ‘stacked’ models that included multiple sets of brain MRI features: “Stacked: All excluding Task Contrast”, “Non Task”, “All excluding Task FC” and “All” (R2 > .76, r >.83, MAE < 65 months). For Cognitionfluid prediction, the top-performing model was “Stacked: All” (R2 = .393, r = .627, MAE = 7.7 points). The best set of features across age and Cognitionfluid prediction was cortical thickness. Across sets of MRI features, the age-prediction models tended to provide higher R2 and r than the Cognitionfluid-prediction models. Figure 2 shows the feature importance of prediction models based on each of the 18 sets of features. Figure 3 shows the feature importance of the eight stacked prediction models.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig2.tif/full/max/0/default.jpg)
Feature importance of prediction models based on each of the 18 sets of features.
We calculated feature importance by, first, standardising Elastic Net weights across brain features of each set of features from each test fold. We then plotted the averaged weights across the five test folds for each of the set of features. For functional connectivity (FC), we, first, multiplied the absolute PCA scores (extracted from the ‘components_’ attribute of ‘sklearn.decomposition.PCA’) with Elastic Net weights and, then, summed the multiplied values across the 75 components, leaving 71,631 ROI-pair indices. Thereafter, we standardised these indices from each test fold and averaged them across the five test folds. Finally, given that the 71,631 ROI-pair indices were based on correlations among 379 ROIs, we averaged the ROI-pair indices from each ROI and plotted them. Accordingly, our FC plots showed the contribution of each seeding area.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig3.tif/full/max/0/default.jpg)
Feature importance of the eight stacked prediction models.
Here we plotted the Elastic weights, standardised across predicted values from different sets of features and averaged across the five test folds.
Simple regression: Using each Brain Age index to explain Cognitionfluid
Figure 4a shows variation in Cognitionfluid explained by Brain Age Indices when having each Brain Age index as the sole regressor in simple regression models. Brain Age and Corrected Brain Age created from higher-performing age-prediction models explained a higher amount of variation in Cognitionfluid. However, Brain Age Gap created from the lower-performing age-prediction models explained a higher amount of variation in Cognitionfluid. For instance, the top performing age-prediction model, “Stacked: All excluding Task Contrast”, generated Brain Age and Corrected Brain Age that explained the highest amount of variation in Cognitionfluid, but, at the same time, produced Brain Age Gap that explained the least amount of variation in Cognitionfluid.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig4.tif/full/max/0/default.jpg)
Simple regression: using each Brain Age index or Brain Cognition to explain Cognitionfluid.
4a shows variation in Cognitionfluid explained by each Brain Age index as a function of the predictive performance of age-prediction models. 4b plots variation in Cognitionfluid explained by Brain Age indices and Brain Cognition.
On the contrary, an amount of variation in Cognitionfluid explained by Corrected Brain Age Gap was relatively small (maximum at R2=.041) across age-prediction models and did not relate to the predictive performance of the age-prediction models. Figure 4b shows variation in Cognitionfluid explained by Brain Cognition, as compared to Brain Age indices. Brain Cognition appeared to explain a higher amount of variation in Cognitionfluid than any Brain Age indices, especially for top-performing age/cognition-prediction models (e.g., Stacked: All).
Multiple regression: Using chronological age and each Brain Age index to explain Cognitionfluid
Figure 5 shows the commonality analysis of multiple regression models, having both chronological age and each Brain Age index as the regressors for Cognitionfluid. We found R2 for these models at M = .326 (SD = 005). The unique effects of Brain Age indices were all relatively small (maximum at ΔR2Brain Age index = .0161, with statistically significant at p-value < .05 in 10 out of 26 models) across the four Brain Age indices and across different ageprediction models.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig5.tif/full/max/0/default.jpg)
Commonality analysis of a multiple regression model, having both chronological age and each Brain Age index as the regressors for Cognitionfluid.
However, it is clear that different Brain Age indices led to different levels of the unique effects of chronological age and the common effects between chronological age and Brain Age indices. For the top-performing age-prediction models (e.g., Stacked: All excluding Task Contrast), the unique effects of chronological age were low for Brain Age and Corrected Brain Age, but high for Brain Age Gap. On the contrary, the lower-performing age-prediction models provided high common effects for Brain Age and Corrected Brain Age, but low for Brain Age Gap. Nonetheless, for Corrected Brain Age Gap, the unique effects of chronological age were much higher than the common effects across all age-prediction models.
Multiple regression: Using chronological age, each Brain Age index and Brain Cognition to explain Cognitionfluid
Figure 6 shows the commonality analysis of multiple regression models, having chronological age, each Brain Age index and Brain Cognition as the regressors for Cognitionfluid. We found R2 for these models at M=.385 (SD=.042). As before, the unique effects of Brain Age indices were all relatively small across the four Brain Age indices and across different prediction models. On the contrary, the unique effects of Brain Cognition appeared much larger (maximum at ΔR2cognition = .1183, statistically significant p-value at .05 in 24 out of 26 models).
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_fig6.tif/full/max/0/default.jpg)
Commonality analysis of a multiple regression model, having chronological age and each Brain Age index and Brain Cognition as the regressors for Cognitionfluid.
For top-performing age/cognition-prediction models (e.g., Stacked All), the largest proportion of Cognitionfluid was attributed to a) the common effects among the three for Brain Age and Corrected Brain Age and b) the common effects between chronological age and Brain Cognition for Brain Age Gap and Corrected Brain Age Gap.
Discussion
To demonstrate the utility of Brain Age as a biomarker for cognitionfluid, we investigated three unsettling issues. First, how much does Brain Age add to what is already captured by chronological age? The short answer is very little. Second, do better-performing age-prediction models improve the ability of Brain Age to capture cognitionfluid? Unfortunately, the answer is no. Third, do we have a solution that can improve our ability to capture cognitionfluid from brain MRI? The answer is, fortunately, yes. Using Brain Cognition as a biomarker, along with chronological age, seemed to capture a higher amount of variation in cognitionfluid than only using Brain Age.
First, Brain Age itself did not add much more information to help us capture cognitionfluid than what we had already known from a person’s chronological age. This can clearly be seen from the small unique effects of Brain Age indices in the multiple regression models having Brain Age and chronological age as the regressors. While the unique effects of some Brain Age indices from certain age-prediction models were statistically significant, there were all relatively small. Without Brain Age indices, chronological age by itself already explained around 32% of variation in cognitionfluid. Including Brain Age indices only added around 1.6% at best.
Investigating the simple regression models and the commonality analysis between each Brain Age index and chronological age provided additional insights. In the simple regression models, higher-performing age-prediction models, such as stacked models, created Brain Age and Corrected Brain Age that captured a higher amount of variation in Cognitionfluid. Because both Brain Age and Corrected Brain Age from higher-performing age-prediction models were closer to the real chronological age of participants, their ability to capture cognitionfluid mirrored the ability of chronological age. The commonality analysis confirmed this by showing higher common effects between (Corrected) Brain Age and chronological age from higher-performing age-prediction models. In contrast, lower-performing (as opposed to higher-performing) age-prediction models, such as CARIT NoGo-Go, created Brain Age Gap that explained a higher amount of variation in Cognitionfluid. Brain Age Gap was a result of subtracting a real chronological age from Brain Age. And when Brain Age was a poor indicator of the real chronological age, the ability of Brain Age Gap is driven more by the real chronological age (Butler et al., 2021). The commonality analysis confirmed this by showing higher common effects, therefore more similarity in variance, between Brain Age Gap and chronological age from lower-performing, than higher-performing, age-prediction models.
Corrected Brain Age Gap, on the other hand, showed weak effects in the simple regression models across all age-prediction models (max at around 4.1% of variation explained). Corrected Brain Age Gap was the only index among the four that appeared to deconfound the influences of chronological age on the relationship between brain aging and cognitionfluid (Butler et al., 2021). This can be seen in the small common effects between Corrected Brain Age Gap and chronological age in the multiple regression models with chronological age and each Brain Age index as regressors. Note while these common effects between Corrected Brain Age Gap and chronological age were small, most were not zero (max at around 3.3% of variation explained). This means that the correction done to deconfound the influences of chronological age on Corrected Brain Age Gap (de Lange & Cole, 2020) may not be perfect. Perhaps this is because the estimation of the influences of chronological age was done in the training set, which might not fully be applicable to the test sets. Still, weak effects of Corrected Brain Age Gap in the simple regression indicate that, after controlling for the influences of chronological age, this Brain Age index could only account for a small amount of variation in cognitionfluid. In other words, the weak effects of Corrected Brain Age Gap shown by the simple regression are consistent with the small unique effects across the four Brain Age indices shown by the multiple regression models having a Brain Age index and chronological age as regressors.
Second, the predictive performance of age-prediction models did not correspond to the ability of Brain Age to capture cognitionfluid over and above chronological age. For instance, while the best-performing age-prediction model was “Stacked: All excluding Task Contrast” (R2=.775), the unique effects of Brain Age indices from this model in the two-regressor multiple regressions were weak (ΔR2Brain Age index ≤.0048) and not statistically significant. The highest unique effects of Brain Age indices in the two-regressor multiple regression models were from the FACENAME: Distractor model (ΔR2Brain age index ≤.0135, p < .05) that had a poorer performance in predicting chronological age (R2=.204). Accordingly, a race to improve the performance of age-prediction models (Baecker et al., 2021) does not necessarily enhance the utility of Brain Age indices as a biomarker for cognitionfluid. This calls for a new paradigm. Future research should aim to build prediction models for Brain Age indices that are not necessarily good at predicting age, but at capturing phenotypes of interest, such as cognitionfluid and beyond.
Third, while Brain Age indices may not add much to capture cognitionfluid, over and above chronological age, Brain Cognition did as shown by its unique effects. Adding Brain Cognition as a regressor along with chronological age and a Brain Age index allowed us to explain as much as 11% more of the variation in cognitionfluid (i.e., around 1/3 times better than the multiple regression models without Brain Cognition). Based on the commonality analyses, the multiple regression models that explained higher variation were driven predominantly by the common effects involving Brain Cognition. This overlapped variation was consistent with a recent study (Jiang et al., 2022). Altogether, while both the effects of Brain Age and Brain Cognition overlapped with chronological age, Brain Cognition offered additional unique effects that helped improve the ability to capture cognitionfluid.
What does it mean then for researchers/clinicians who would like to use Brain Age as a biomarker? First, they have to be aware of the overlap in variation between Brain Age and chronological age and should focus on the contribution of Brain Age over and above chronological age. Using Brain Age Gap will not fix this. Similar to a previous recommendation (Butler et al., 2021), we suggest focusing on Corrected Brain Age Gap or, better, unique effects of Brain Age indices in multiple regressions. In the case of cognitionfluid, the unique effects might be too small to be clinically meaningful. While our emphasis was on cognitionfluid, we suspect that the unique effects of Brain Age might also be small for other phenotypes, especially if those phenotypes vary with age (e.g., neurodegenerative risks). Next, researchers/clinicians should not select the MRI modalities or machine-learning algorithms based on age-prediction performance, but rather select them based on the unique effects of Brain Age indices. Finally, beyond Brain Age, researchers/clinicians should explore the unique effects of other biomarkers that may tap closer into the phenotypes of interest. One straightforward approach tested here is to build the model using the phenotype of interest as a target (i.e., cognitionfluid). This approach significantly improved our ability to capture cognitionfluid in the current study.
Methods and Materials
Dataset
We used the Human Connectome Project in Aging (HCP-A) (Bookheimer et al., 2019) Release 2.0 (24-February-2021). HCP-A’s ‘typical-aging’ participants (36-100 years old) may have prevalent health conditions (e.g., hypertension and different forms of vascular risks) but did not have identified pathological causes of cognitive decline (e.g., stroke and clinical dementia). In this Release, HCP-A provided data from 725 participants. HCP-A offered quality control flags, and here, we removed participants with the flag ‘A’ anatomical anomalies or ‘B’ segmentation and surface (n= 117). Following further removal of participants with missing values in any of MRI modalities (n=15) or cognitive measurements (n= 111), we ultimately included 504 individuals (293 females, M= 57.83 (SD=14.25) years old) in our analyses. For ethical procedures including informed consent, please see Bookheimer and colleagues’ (2019).
Sets of brain MRI features
HCP-A provides details of parameters for brain MRI elsewhere (Bookheimer et al., 2019; Harms et al., 2018). Here we used MRI data that were pre-processed by the HCP-A with recommended methods, including the MSMALL alignment (Glasser et al., 2016; Robinson et al., 2018) and ICA-FIX (Glasser et al., 2016) for functional MRI. We used multiple brain MRI modalities, covering task functional MRI (task fMRI), resting-state functional MRI (rsfMRI) and structural MRI (sMRI), and organised them into 19 sets of features.
Sets of Features 1-10: Task fMRI contrast (Task Contrast)
Task contrasts reflect fMRI activation relevant to events in each task. Bookheimer and colleagues (2019) provided detailed information about the fMRI in HCP-A. Here we focused on the pre-processed task fMRI files with a suffix, “_PA_Atlas_MSMAll_hp0_clean.dtserĩes.nii.” To extract Task Contrasts, we regressed the fMRI time series on the convolved task events using a double-gamma canonical hemodynamic response function via FMRIB Software Library (FSL)’s FMRI Expert Analysis Tool (FEAT) (Woolrich et al., 2001). We then parcellated the contrast ‘cope’ files, using the Glasser atlas (Gordon et al., 2016) for cortical surface regions and the Freesurfer’s automatic segmentation (aseg) (Fischl et al., 2002) for subcortical regions. This resulted in 379 regions, which was, in turn, the number of features for each Task Contrast set of features.
HCP-A collected fMRI data from three tasks: Face Name (Sperling et al., 2001), Conditioned Approach Response Inhibition Task (CARIT) (Somerville et al., 2018) and VISual MOTOR (VISMOTOR) (Ances et al., 2009). First, the Face Name task (Sperling et al., 2001) taps into episodic memory. The task had three blocks. In the encoding block [Encoding], participants were asked to memorise the names of faces shown. These faces were then shown again in the recall block [Recall] when the participants were asked if they could remember the names of the previously shown faces. There was also the distractor block [Distractor] occurring between the encoding and recall blocks. Here participants were distracted by a Go/NoGo task. We computed six contrasts for this Face Name task: [Encode], [Recall], [Distractor], [Encode vs. Distractor], [Recall vs. Distractor] and [Encode vs. Recall].
Second, the CARIT task (Somerville et al., 2018) was adapted from the classic Go/NoGo task and taps into inhibitory control. Participants were asked to press a button to all [Go] but not to two [NoGo] shapes. We computed three contrasts for the CARIT task: [NoGo], [Go] and [NoGo vs. Go].
Third, the VISMOTOR task (Ances et al., 2009) was designed to test simple activation of the motor and visual cortices. Participants saw a checkerboard with a red square either on the left or right. They needed to press a corresponding key to indicate the location of the red square. We computed just one contrast for the VISMOTOR task: [Vismotor], which indicates the presence of the checkerboard vs. baseline.
Sets of Features 11-13: Task fMRI functional connectivity (Task FC)
Task FC reflects functional connectivity (FC) among the brain regions during each task, which is considered an important source of individual differences (Elliott et al., 2019; Fair et al., 2007; Gratton et al., 2018). Unlike Task Contrasts, here we treated the double-gamma, convolved task events as regressors of no interest and focused on the residuals of the regression from each task (Fair et al., 2007). Using the same atlases as Task Contrast (Fischl et al., 2002; Glasser et al., 2016), we computed Pearson’s correlations of each pair of 379 regions, resulting in a table of 71,631 non-overlapping FC indices for each task. We then applied r-to-z transformation and principal component analysis (PCA) of 75 components (Rasero et al., 2021; Sripada et al., 2019, 2020). Note to avoid data leakage, we conducted the PCA on each training set and applied its definition to the corresponding test set. Accordingly, there were three sets of 75 features for Task FC, one for each task.
Set of Features 14: Resting-state functional MRI functional connectivity (Rest FC)
Similar to Task FC, Rest FC reflects functional connectivity (FC) among the brain regions, except that Rest FC occurred during the resting (as opposed to task-performing) period. HCP-A collected Rest FC from four 6.42-min (488 frames) runs across two days, leading to 26-min long data (Harms et al., 2018). On each day, the study scanned two runs of Rest FC, starting with anterior-to-posterior (AP) and then with posterior-to-anterior (PA) phase encoding polarity. We used the “rfMRI_REST_Atlas_MSMAll_hp0_clean.dscalar.nii” file that was pre-processed and concatenated across the four runs. We applied the same computations (i.e., parcellation, Pearson’s correlations, r-to-z transformation and PCA) with the Task FC.
Sets of Features 15-18: Structural MRI (sMRI)
sMRI reflects individual differences in brain anatomy. The HCP-A used an established preprocessing pipeline for sMRI (Glasser et al., 2013). We focused on four sets of features: cortical thickness, cortical surface area, subcortical volume and total brain volume. For cortical thickness and cortical surface area, we used Destrieux’s atlas (Destrieux et al., 2010; Fischl, 2012) from FreeSurfer’s “aparc.stats” file, resulting in 148 regions for each set of features. For subcortical volume, we used the aseg atlas (Fischl et al., 2002) from FreeSurfer’s “aseg.stats” file, resulting in 19 regions. For total brain volume, we had five FreeSurfer-based features: “FS_IntraCranial_Vol” or estimated intra-cranial volume, “FS_TotCort_GM_Vol” or total cortical grey matter volume, “FS_Tot_WM_Vol” or total cortical white matter volume, “FS_SubCort_GM_Vol” or total subcortical grey matter volume and “FS_BrainSegVol_eTIV_Ratio” or ratio of brain segmentation volume to estimated total intracranial volume.
Cognitive abilities: Cognitionfluid
We measured Cognitionfluid via the NIH Toolbox (Weintraub et al., 2014), using the “fluidcogcomp_unadj” variable. Cognitionfluid summarises scores from five tests assessed outside of the MRI: Dimensional Change Card Sort, Flanker Inhibitory Control and Attention, Picture Sequence Memory, List Sorting Working Memory and Pattern Comparison Processing Speed.
Prediction models for Brain Age and Brain Cognition
To compute Brain Age and Brain Cognition, we ran two separate prediction models. These prediction models either had chronological age or Cognitionfluid as the target and standardised brain MRI as the features (Denissen et al., 2022). We used nested cross-validation (CV) to build these models. We first split the data into five folds (i.e., outer CV). One of these outer-CV folds was treated as a test set, and the rest was treated as a training set, which was further divided into five folds (i.e., inner CV). We used the inner CV to tune for hyperparameters of the models and the outer CV to evaluate the predictive performance of the models. We controlled for the potential influences of biological sex on the brain features by first residualising biological sex from brain features in the training set. We then applied the regression of this residualisation based on the training set to the test set. We also standardised the brain features in the training set and then used the mean and standard deviation of the training set to standardise the test set.
To demonstrate the predictive performance, we assessed the similarity between the observed values and the predicted values of each model across test sets, using Pearson’s r, coefficient of determination (R2) and mean absolute error (MAE). Note that for R2, we used the sum of squares definition (i.e., R2 = 1 – (sum of squares residuals/total sum of squares)) per a previous recommendation (Poldrack et al., 2020). We considered the predicted values from the test sets of models predicting age or Cognitionfluid, as Brain Age and Brain Cognition, respectively.
In addition to using each of the 18 sets of features in separate prediction models, we drew information across these sets via stacking. Specifically, we computed predicted values from each of the 18 sets of features in the training sets. We then treated different combinations of these predicted values as features to predict the targets in separate “stacked” models. We specified eight stacked models: “All” (i.e., including all 18 sets of features), “All excluding Task FC”, “All excluding Task Contrast”, “Non-Task” (i.e., including only Rest FC and sMRI), “Resting and Task FC”, “Task Contrast and FC”, “Task Contrast” and “Task FC”. Accordingly, in total, there were 26 prediction models for Brain Age and Brain Cognition.
For the machine learning algorithm, we used Elastic Net (Zou & Hastie, 2005) as implemented in sklearn (Pedregosa et al., 2011). Not only does Elastic Net provide a relatively good predictive performance for fMRI (Dubois et al., 2018; Pat, Wang, Anney, et al., 2022; Pat, Wang, Bartonicek, et al., 2022; Tetereva et al., 2022), but it also offers easy-to-interpret feature importance (Molnar, 2019). In our grid search, we tuned two Elastic Net hyperparameters: a using 70 numbers in log space, ranging from .1 and 100, and ℓ1-ratio using 25 numbers in linear space, ranging from 0 and 1.
Brain Age calculations: Brain Age, Brain Age Gap, Corrected Brain Age and Corrected Brain Age Gap
In addition to Brain Age, which is the predicted value from the models predicting chronological age in the test sets, we calculated three other indices to reflect the estimation of brain aging. First, Brain Age Gap reflects the difference between the age predicted by brain MRI and the actual, chronological age. Here we simply subtracted the chronological age from Brain Age.
Next, to correct for the estimation biases (i.e., overestimation in younger, but underestimation in older, individuals (Le et al., 2018)), we applied de Lange and Cole’s (2020) 5th equation. Specifically, we first fit a regression line predicting the Brain Age from a chronological age in a training set. We then used the slope and intercept of this regression line to adjust Brain Age in a test set, resulting in Corrected Brain Age.
Lastly, we computed Corrected Brain Age Gap by subtracting the chronological age from the Corrected Brain Age (Butler et al., 2021; Le et al., 2018).
The ability of Brain Age indices to capture Cognitionfluid
We first combined Brain Age, Brain Cognition, chronological age and Cognitionfluid across test sets into one table. We then conducted three sets of regression analyses to demonstrate the ability of different Brain Age indices, calculated from 26 different prediction models based on different sets of brain MRI features, to capture Cognitionfluid.
1. Simple Regression: Using each Brain Age index to explain Cognitionfluid
Here using simple regression, we simply had each Brain Age index as the sole regressor for Cognitionfluid:
where i is the index for the four Brain Age indices. Because different Brain Age indices differ in the adjustments applied, this simple regression could reveal the extent to which each adjustment influences variation in Cognitionfluid explained. Additionally, Brain Age calculated from 26 different prediction models would have different levels of predictive performance in predicting chronological age. Accordingly, this simple regression could also reveal if Brain Age from a better-performing age-prediction model was able to capture more variation in Cognitionfluid.
In addition to Brain Age indices, we also used simple regression to test how well Brain Cognition as a sole regressor explains Cognitionfluid:
This allows us to compare the ability of Brain Age Indices vs. Brain Cognition as a sole regressor for predicting Cognitionfluid.
2. Multiple Regression: Using chronological age and each Brain Age index to explain Cognitionfluid
Here using multiple regression, we had both chronological age and each Brain Age index as the regressors for Cognitionfluid:
Having chronological age in the same regression model as a Brain Age index allowed us to control for the effects of chronological age on the Brain Age index, thereby, revealing the unique effects of the Brain Age index (Butler et al., 2021; Le et al., 2018). To formally determine the unique effects of a Brain Age index on Cognitionfluid along with the effects it shared with chronological age (i.e., common effects), we applied the commonality analysis (Nimon et al., 2008). For the unique effects, we computed ΔR2. ΔR2 is the increase in R2 when having an additional regressor in the regression model:
We determined the statistical significance of ΔR2 by:
where FChange is the F-ratio (with the degree of freedom of kChange and N – k2 – 1), N is the number of observations, 2 is the model with more regressors, k is the number of regressors, kChange is the difference between the number of regressors.
As for the common effects between chronological age and each Brain Age index, we used the below calculation:
These common effects indicate the extent to which variation in Cognitionfluid explained by each Brain Age index was shared with chronological age. Note the common effects can be negative, especially with a high multicollinearity (Ray-Mukherjee et al., 2014). To deal with this, we treated negative common effects as zero (Frederick, 1999) and then scaled variation explained by other effects to be proportional to the total effects of the full regression model.
3. Multiple Regression: Using chronological age, each Brain Age index and Brain Cognition to explain Cognitionfluid
Similar to the above multiple regression model, we had chronological age, each Brain Age index and Brain Cognition as the regressors for Cognitionfluid:
Applying the commonality analysis here allowed us, first, to investigate the addictive, unique effects of Brain Cognition, over and above chronological age and Brain Age indices. More importantly, the commonality analysis also enabled us to test the common, shared effects that Brain Cognition had with chronological age and Brain Age indices in explaining Cognitionfluid. We calculated the commonality analysis as follows (Nimon et al., 2017):
Conflict of interest statement
The authors declare no competing interests.
Acknowledgements
Data were provided by the Human Connectome Project in Aging. Research reported in this publication was supported by the National Institute On Aging of the National Institutes of Health under Award Number U01AG052564 and by funds provided by the McDonnell Center for Systems Neuroscience at Washington University in St. Louis. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The author(s) wish to acknowledge the use of New Zealand eScience Infrastructure (NeSI) high performance computing facilities, consulting support and/or training services as part of this research. New Zealand’s national facilities are provided by NeSI and funded jointly by NeSI’s collaborator institutions and through the Ministry of Business, Innovation & Employment’s Research Infrastructure programme. URL https://www.nesi.org.nz. A.T. and N.P. were supported by Health Research Council Funding (21/618) and by the University of Otago.
Code Accessibility
The shell and Python scripts used in the analyses are made available here: https://github.com/HAM-lab-Otago-University/HCP-Aging_commonality
Supplementary
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_figS1.tif/full/max/0/default.jpg)
The scatter plots between observed and predicted values in the test sets from age-prediction models.
![](https://prod--epp.elifesciences.org/iiif/2/87297%2Fv1%2Fcontent%2F522374v2_figS2.tif/full/max/0/default.jpg)
The scatter plots between observed and predicted values in the test sets from cognition-prediction models
References
- Effects of aging on cerebral blood flow, oxygen metabolism, and blood oxygenation level dependent responses to visual stimulationHuman Brain Mapping 30:1120–1132https://doi.org/10.1002/hbm.20574
- Machine learning for brain age prediction: Introduction to methods and clinical applicationsEBioMedicine 72https://doi.org/10.1016/j.ebiom.2021.103600
- Bias-adjustment in neuroimagingbased brain age frameworks: A robust schemeNeuroImage: Clinical 24https://doi.org/10.1016/j.nicl.2019.102063
- The Lifespan Human Connectome Project in Aging: An overviewNeuroImage 185:335–348https://doi.org/10.1016/j.neuroimage.2018.10.009
- Brain-predicted age difference score is related to specific cognitive functions: A multi-site replication analysisBrain Imaging and Behavior 15:327–345https://doi.org/10.1007/s11682-020-00260-3
- Pitfalls in brain age analysesHuman Brain Mapping 42:4092–4101https://doi.org/10.1002/hbm.25533
- Multimodality neuroimaging brain-age in UK biobank: Relationship to biomedical, lifestyle, and cognitive factorsNeurobiology of Aging 92:34–42https://doi.org/10.1016/j.neurobiolaging.2020.03.014
- Brain age predicts mortalityMolecular Psychiatry 23https://doi.org/10.1038/mp.2017.62
- Commentary: Correction procedures in brain-age predictionNeuroImage: Clinical 26https://doi.org/10.1016/j.nicl.2020.102229
- Brain age as a surrogate marker for cognitive performance in multiple sclerosisEuropean Journal of Neurology 29:3039–3049https://doi.org/10.1111/ene.15473
- Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclatureNeuroImage 53:1–15https://doi.org/10.1016/j.neuroimage.2010.06.010
- A distributed brain network predicts general intelligence from resting-state human neuroimaging dataPhilosophical Transactions of the Royal Society B: Biological Sciences 373https://doi.org/10.1098/rstb.2017.0284
- General functional connectivity: Shared features of resting-state and task fMRI drive reliable and heritable individual differences in functional brain networksNeuroImage 189:516–532https://doi.org/10.1016/j.neuroimage.2019.01.068
- Combining magnetoencephalography with magnetic resonance imaging enhances learning of surrogate-biomarkersELife 9https://doi.org/10.7554/eLife.54055
- A method for using blocked and event-related fMRI data to study “resting state” functional connectivityNeuroImage 35:396–405https://doi.org/10.1016/j.neuroimage.2006.11.051
- FreeSurferNeuroImage 62:774–781https://doi.org/10.1016/j.neuroimage.2012.01.021
- Whole Brain SegmentationNeuron 33:341–355https://doi.org/10.1016/S0896-6273(02)00569-X
- Ten Years of BrainAGE as a Neuroimaging Biomarker of Brain Aging: What Insights Have We Gained?Frontiers in Neurology 10https://doi.org/10.3389/fneur.2019.00789
- Estimating the age of healthy subjects from T1-weighted MRI scans using kernel methods: Exploring the influence of various parametersNeuroImage 50:883–892https://doi.org/10.1016/j.neuroimage.2010.01.005
- Partitioning variance in the multivariate case: A step-by-step guide to canonical commonality analysisAdvances in Social Science Methodology 5:305–318
- Strategic roadmap for an early diagnosis of Alzheimer’s disease based on biomarkersThe Lancet Neurology 16:661–676https://doi.org/10.1016/S1474-4422(17)30159-X
- The Human Connectome Project’s neuroimaging approachNature Neuroscience 19:1175–1187https://doi.org/10.1038/nn.4361
- The minimal preprocessing pipelines for the Human Connectome ProjectNeuroImage 80:105–124https://doi.org/10.1016/j.neuroimage.2013.04.127
- Generation and Evaluation of a Cortical Area Parcellation from Resting-State CorrelationsCerebral Cortex 26:288–303https://doi.org/10.1093/cercor/bhu239
- Functional Brain Networks Are Dominated by Stable Group and Individual Factors, Not Cognitive or Daily VariationNeuron 98:439–452https://doi.org/10.1016/j.neuron.2018.03.035
- Extending the Human Connectome Project across ages: Imaging protocols for the Lifespan Development and Aging projectsNeuroImage 183:972–984https://doi.org/10.1016/j.neuroimage.2018.09.060
- A Neuroimaging Signature of Cognitive Aging from Whole-Brain Functional ConnectivityAdvanced Science 9https://doi.org/10.1002/advs.202201621
- Brain age prediction using deep learning uncovers associated sequence variantsNature Communications 10https://doi.org/10.1038/s41467-019-13163-9
- A Nonlinear Simulation Framework Supports Adjusting for Age When Analyzing BrainAGE
- Investigating systematic bias in brain age estimation with application to post-traumatic stress disordersHuman Brain Mapping 40:3143–3152https://doi.org/10.1002/hbm.24588
- Predicting brain-age from multimodal imaging data captures cognitive impairmentNeuroImage 148:179–188https://doi.org/10.1016/j.neuroimage.2016.11.005
- Molnar, C. (2019). Interpretable Machine Learning. A Guide for Making Black Box Models Explainable. https://christophm.github.io/interpretable-ml-book/
- An R package to compute commonality coefficients in the multiple regression case: An introduction to the package and a practical exampleBehavior Research Methods 40:457–466https://doi.org/10.3758/BRM.40.2.457
- Erratum to: An R package to compute commonality coefficients in the multiple regression case: An introduction to the package and a practical exampleBehavior Research Methods 49:2275–2275https://doi.org/10.3758/s13428-017-0853-2
- Longitudinally stable, brain-based predictive models mediate the relationships between childhood cognition and socio-demographic, psychological and genetic factorshttps://doi.org/10.1002/hbm.26027
- Explainable machine learning approach to predict and explain the relationship between task-based fMRI and individual differences in cognitionhttps://doi.org/10.1093/cercor/bhac235
- Scikit-learn: Machine Learning in PythonJournal of Machine Learning Research 12:2825–2830
- Establishment of Best Practices for Evidence for Prediction: A ReviewJAMA Psychiatry 77:534–540https://doi.org/10.1001/jamapsychiatry.2019.3671
- Integrating across neuroimaging modalities boosts prediction accuracy of cognitive abilityPLOS Computational Biology 17https://doi.org/10.1371/journal.pcbi.1008347
- Using commonality analysis in multiple regressions: A tool to decompose regression effects in the face of multicollinearityMethods in Ecology and Evolution 5:320–328https://doi.org/10.1111/2041-210X.12166
- Assessing distinct patterns of cognitive aging using tissuespecific brain age prediction based on diffusion tensor imaging and brain morphometryPeerJ 6https://doi.org/10.7717/peerj.5908
- Multimodal surface matching with higher-order smoothness constraintsNeuroImage 167:453–465https://doi.org/10.1016/j.neuroimage.2017.10.037
- Trajectories of normal cognitive agingPsychology and Aging 34:17–24https://doi.org/10.1037/pag0000288
- Estimation of brain age delta from brain imagingNeuroImage 200:528–539https://doi.org/10.1016/j.neuroimage.2019.06.017
- The Lifespan Human Connectome Project in Development: A large-scale study of brain connectivity development in 5–21 year oldsNeuroImage 183:456–468https://doi.org/10.1016/j.neuroimage.2018.08.050
- Encoding novel face-name associations: A functional MRI studyHuman Brain Mapping 14:129–139https://doi.org/10.1002/hbm.1047
- Basic Units of Inter-Individual Variation in Resting State ConnectomesScientific Reports 9https://doi.org/10.1038/s41598-018-38406-5
- Toward a “treadmill test” for cognition: Improved prediction of general cognitive ability from the task activated brainHuman Brain Mapping 41:3186–3197https://doi.org/10.1002/hbm.25007
- Capturing brain-cognition relationship: Integrating task-based fMRI across tasks markedly boosts prediction and test-retest reliabilityNeuroImage 263https://doi.org/10.1016/j.neuroimage.2022.119588
- On the prediction of human intelligence from neuroimaging: A systematic review of methods and reportingIntelligence 93https://doi.org/10.1016/j.intell.2022.101654
- The Cognition Battery of the NIH Toolbox for Assessment of Neurological and Behavioral Function: Validation in an Adult SampleJournal of the International Neuropsychological Society 20:567–578https://doi.org/10.1017/S1355617714000320
- Temporal Autocorrelation in Univariate Linear Modeling of FMRI DataNeuroImage 14:1370–1386https://doi.org/10.1006/nimg.2001.0931
- Brain-predicted age difference is associated with cognitive processing in later-lifeNeurobiology of Aging 109:195–203https://doi.org/10.1016/j.neurobiolaging.2021.10.007
- Deep learning of brain magnetic resonance images: A brief reviewMethods 192:131–140https://doi.org/10.1016/j.ymeth.2020.09.007
- Regularization and variable selection via the elastic netJournal of the Royal Statistical Society: Series B (Statistical Methodology) 67:301–320https://doi.org/10.1111/j.1467-9868.2005.00503.x
Article and author information
Version history
Copyright
© 2023, Alina Tetereva & Narun Pat
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
- views
- 1,338
- downloads
- 62
- citations
- 0
Views, downloads and citations are aggregated across all versions of this paper published by eLife.