Decoding the brain state-dependent relationship between pupil dynamics and resting state fMRI signal fluctuation
- Translational Neuroimaging and Neural Control Group, High Field Magnetic Resonance Department, Max Planck Institute for Biological Cybernetics, Germany
- Graduate Training Centre of Neuroscience, International Max Planck Research School, University of Tuebingen, Germany
- Institute of Neuroscience and Medicine 4, Medical Imaging Physics, Forschungszentrum Jülich, Germany
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, United States
Figures
Figure 1
Variability of the pupil–fMRI linkage.
(A) Pupil–fMRI correlation map created by correlating the two modalities’ concatenated signals from all trials. (B) Selected individual-trial correlations maps. (C) Histogram of spatial correlations …
-
Figure 1—source data 1
The mean correlation map (A), all individual correlation maps (B), and the spatial correlation values (C) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig1-data1-v2.zip
Figure 2 with 6 supplements
Clustering of trials with distinct pupil–fMRI correlation patterns.
(A) Schematic of the clustering procedure. UMAP is used to reduce the dimensionality of all individual-trial correlation maps to 72 dimensions. A 2D UMAP-projection of the real data is shown. Each …
-
Figure 2—source data 1
Cluster trial labels, individual silhouette scores (B), mean cluster PSDs (C), and cluster-specific correlation maps (D) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-data1-v2.zip
Figure 2—figure supplement 1
Cluster reproducibility across 100 repetitions with random UMAP and GMM initializations.
(A) Matrix displaying the ratio of cluster membership labels matching the most common cluster membership assignment across the 100 repetitions. (B) Matrix displaying the mean spatial correlation …
-
Figure 2—figure supplement 1—source data 1
The label match ratios (A) and map similarity values (B) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp1-data1-v2.zip
Figure 2—figure supplement 2
Cluster-specific pupil fluctuation features.
(A) PSDs of all trials divided based on their cluster memberships. Clusters 1 and 3 show specific peak frequencies. Cluster 4 shows the largest PSDs, hence the largest pupil size fluctuations. …
-
Figure 2—figure supplement 2—source data 1
All PSDs (A) are available in the source data file.
All pupil signals (B) are available online (see Materials and methods).
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp2-data1-v2.zip
Figure 2—figure supplement 3
Clustering reproducibility across 100 clustering repetitions based on HRF-convolved pupil signals.
(A) HRFs with different peak times were used to create the convolved and lagged pupil signals. (B) Mean match ratio of cluster membership labels created using the convolved signals and those …
-
Figure 2—figure supplement 3—source data 1
The HRF kernels (A), cluster membership label match ratios (B), and map similarity values (C) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp3-data1-v2.zip
Figure 2—figure supplement 4
Cluster reproducibility across 100 repetitions of split-halves clustering.
(A) Matrix displaying the ratio of split-halves cluster membership labels matching the cluster membership assignment based on all trials across 100 repetitions. (B) Matrix displaying the mean …
-
Figure 2—figure supplement 4—source data 1
The label match ratios (A) and map similarity values (B) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp4-data1-v2.zip
Figure 2—figure supplement 5
Cluster reproducibility across 100 sets of artificially generated surrogates with values and spatial autocorrelations matching those of real maps.
(A) A real pupil–fMRI correlation map and three example surrogate maps. (B) Matrix displaying the ratio of cluster membership labels generated based on the surrogate maps and those based on real …
-
Figure 2—figure supplement 5—source data 1
Ten example surrogate sets (i.e. 740 maps total) (A), label match ratios (B), and map similarity values (C) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp5-data1-v2.zip
Figure 2—figure supplement 6
Mean silhouette scores based on 100 clustering repetitions performed on shorter trials.
Based on the silhouette score criterion, the n = 4 clusters result should be chosen even when dividing the trials into three parts.
-
Figure 2—figure supplement 6—source data 1
Individual silhouette scores are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig2-figsupp6-data1-v2.zip
Figure 3 with 3 supplements
Decoding pupil dynamics based on fMRI signals.
(A) Schematic of the decoding procedure. PCA was applied to fMRI data. The PCA time courses were fed into either linear regression or GRU decoders, which generated pupil signal predictions. The …
-
Figure 3—source data 1
Prediction scores (B) and all predicted time courses (C) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig3-data1-v2.zip
Figure 3—figure supplement 1
Prediction score dependence on train-test split trial selection, number of PCA components, and temporal shifts between pupil and fMRI signals.
(A) Mean test and train set prediction scores across 100 random train-test splits. The red dot shows values described in the manuscript. (B) The influence of temporal shifts and different component …
-
Figure 3—figure supplement 1—source data 1
Individual prediction scores across all trial mixes (A) and the mean prediction scores (BC) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig3-figsupp1-data1-v2.zip
Figure 3—figure supplement 2
PCA decoupling of pupil-related fMRI activity from other signal sources.
(A) fMRI variance explained by individual PCA components. Components are ordered by variance explained. Black dot marks the component with the most explained pupil variance. (B) Pupil variance …
-
Figure 3—figure supplement 2—source data 1
The variance explained values (AB) and linear regression weights (C) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig3-figsupp2-data1-v2.zip
Figure 3—figure supplement 3
Similarity of GRU and linear regression prediction maps.
Spatial maps were generated by integrating PCA spatial maps with either linear regression weights or average GRU gradients. The maps highlight the same areas. This observation coupled with the …
-
Figure 3—figure supplement 3—source data 1
The maps are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig3-figsupp3-data1-v2.zip
Figure 4
Localization of pupil dynamics-related information content across the brain.
The spatial map highlights regions from which pupil-related information was decoded. It was created by integrating PCA spatial maps with weights of the trained linear regression model. The map …
-
Figure 4—source data 1
The masked map is available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig4-data1-v2.zip
Figure 5 with 4 supplements
Characterization of brain state-specific pupil–fMRI relationships.
(A) Pupil information content maps generated by integrating PCA spatial maps with weights of linear regression models trained on cluster-specific trials. In all clusters, negative weights were found …
-
Figure 5—source data 1
The unmasked cluster maps (A) and the masked maps based on randomization tests with a different random seed (B) are available in the source data file.
- https://cdn.elifesciences.org/articles/68980/elife-68980-fig5-data1-v2.zip
Figure 5—figure supplement 1
The spatial map based on cluster 1 trials highlights regions from which pupil-related information was decoded.
Masked regions (white) did not pass the false discovery rate corrected significance threshold (p=0.01). Abbreviations: Ce – cerebellum, CP – caudate-putamen, ECx – entorhinal cortex, Hp – …
Figure 5—figure supplement 2
The spatial map based on cluster 2 trials highlights regions from which pupil-related information was decoded.
Masked regions (white) did not pass the false discovery rate corrected significance threshold (p=0.01). Abbreviations: B9 – B9 serotonergic cells, Ce – cerebellum, CgCx – cingulate cortex, ECx – …
Figure 5—figure supplement 3
The spatial map based on cluster 3 trials highlights regions from which pupil-related information was decoded.
Masked regions (white) did not pass the false discovery rate corrected significance threshold (p=0.01). Abbreviations: Ce – cerebellum, CgCx – cingulate cortex, CP – caudate-putamen, ECx – …
Figure 5—figure supplement 4
The spatial map based on cluster 4 trials highlights regions from which pupil-related information was decoded.
Masked regions (white) did not pass the false discovery rate corrected significance threshold (p=0.01). Abbreviations: Ce – cerebellum, CgCx – cingulate cortex, Hp – hippocampus, LH – lateral …
Tables
Table 1
Optimized GRU hyperparameters.
| Parameter name | Description | Range | Final value |
|---|---|---|---|
| Number of layers | Multiple recurrent layers could be stacked on top of each other. | [1; 3] | 1 |
| Hidden size | Hidden state vector size. | [10; 500] | 300 |
| Learning rate | The rate at which network weights were updated during training. | [10–6; 1] | 0.0023 |
| L2 | Strength of the L2 weight regularization. | [0; 10] | 0.0052 |
| Gradient clipping | Gradient clipping (Pascanu et al., 2013) limits the gradient magnitude at a specified maximum value. | [yes; no] | Yes |
| Max. gradient | Value at which the gradients are clipped. | [0.1, 2] | 1 |
| Dropout | During training, a percentage of units could be set to 0 for regularization purposes (Srivastava et al., 2014). | [0; 0.2] | 0 |
| Residual connection | Feeding the input directly to the linear decoder bypassing the RNN’s computation. | [yes; no] | No |
| Batch size | The number of training trials fed into the network before each weight update. | [3; 20] | 12 |
