Across many scientific disciplines, datasets are rapidly increasing in size and scope. These resources have kickstarted a new era of data-driven scientific discovery (Richards et al., 2019; Jumper et al., 2021; Iten et al., 2020; Ravuri et al., 2021; Schawinski et al., 2018; D’Isanto and Polsterer, 2018). In visual neuroscience, recent efforts to sample individual brains at unprecedented scale and depth have yielded high-resolution functional magnetic resonance imaging (fMRI) datasets in which subjects view thousands of distinct images over several dozen hours of scanning (see Naselaris et al., 2021 for a review). These exciting “condition-rich” datasets are large enough to propel the development of computational models of how humans process complex naturalistic stimuli. For example, resources such as the Natural Scenes Dataset (NSD, Allen et al., 2022), BOLD5000 (Chang et al., 2019), and THINGS (Hebart et al., 2019) may be useful for advancing our ability to characterize the tuning (Bao et al., 2020; Li and Bonner, 2022; Long et al., 2018; Kriegeskorte and Wei, 2021; Popham et al., 2021), topography (Blauch et al., 2022; Doshi and Konkle, 2021; Zhang et al., 2021; Lee et al., 2020), and computations (Yamins et al., 2014; DiCarlo et al., 2012; Freeman et al., 2013; Marques et al., 2021; Horikawa and Kamitani, 2017) performed in visual cortex.

The potential of large-scale datasets to reveal general principles of neural function depends critically on signal-to-noise ratio (SNR), which refers to one’s ability to reliably measure distinct neural signatures associated with different stimuli or experimental conditions. Diverse sources of noise affect fMRI data, and these noise sources limit the robustness and interpretability of data analyses (Liu, 2016; Kay et al., 2013). For example, subject head motion, scanner instabilities, physiological noise, and thermal noise all contribute unwanted variability to fMRI data. Noise is especially problematic in studies that sample a large number of conditions, since the number of repetitions of each condition is typically limited, resulting in noisy responses even after trial-averaging.

The approach we have developed to mitigate the effects of noise comes in the context of general linear model (GLM) analysis of fMRI time-series data (Dale, 1999; Monti, 2011). We assume that the goal of the GLM analysis is to estimate beta weights representing the blood oxygenation level dependent (BOLD) response amplitude evoked by different experimental conditions. In this context, we define noise as variability observed across repeated instances of a given condition. Therefore, methods that decrease such variability are desirable. Our approach seeks to maximize data quality at the level of individual voxels in individual subjects (as opposed to data quality assessed only at the region or group level), and seeks to obtain response estimates for single trials. These desiderata are powerful; if achieved, they can flexibly support a wide range of subsequent analyses including relating brain responses to trial-wise behavioral measures and pooling data across trials, brain regions, and/or subjects.

To realize these goals, we introduce GLMsingle, a user-friendly software toolbox (with both MATLAB and Python implementations) that performs single-trial BOLD response estimation. Given fMRI time-series data and a design matrix indicating the onsets of experimental conditions, GLMsingle implements a set of optimizations that target three aspects of the GLM framework (Figure 1):

1. The choice of hemodynamic response function (HRF) to convolve with the design matrix

2. The inclusion of nuisance regressors that account for components of the data that are thought to be noise

3. The use of regularization to improve the accuracy of the final beta estimates

Figure 1

Download asset Open asset

Importantly, to enable fluid application to even the largest fMRI datasets, GLMsingle is fully automated (no manual setting of parameters) and can be executed efficiently even when gigabytes of fMRI data are passed as input.

We previously used the GLMsingle algorithm to estimate BOLD responses in the NSD dataset (Allen et al., 2022). While the optimizations implemented in GLMsingle had a positive impact on data quality, it was not apparent whether the improvements would generalize to other datasets. The goal of this paper is to provide a standalone description of GLMsingle and to rigorously assess performance not only on NSD, but also on BOLD5000 (Chang et al., 2019), a distinct large-scale fMRI dataset acquired with different subjects, at different field strength, and with a different experimental design (see Materials and methods). In both datasets, we show that the optimizations implemented in GLMsingle dramatically improve the reliability of GLM beta estimates. We provide further evidence of the general utility of GLMsingle by also evaluating its performance on the music-listening experiment from StudyForrest (Hanke et al., 2015). This dataset differs in a number of respects from NSD and BOLD5000: it reflects a non-visual modality, and contains fewer experimental conditions, longer condition durations, a greater number of repetitions per condition, and a jittered inter-stimulus interval. The performance improvements found in this third dataset suggest that GLMsingle may be applicable to a wide range of fMRI tasks and study designs.

We also study the effect of these optimizations on downstream analyses that are of particular relevance to systems and cognitive neuroscience, including representational similarity analysis (RSA; Kriegeskorte et al., 2008) and multivoxel pattern analysis (MVPA; Haxby et al., 2001; Norman et al., 2006; Poldrack et al., 2011). In all analyses, we observe improvements in key outcome metrics, suggesting that GLMsingle meaningfully improves the ability of researchers to gain insight into neural representation and computation. Our findings demonstrate that GLMsingle affords the neuroimaging community a clear opportunity for improved data quality. Online materials (code, documentation, example scripts) pertaining to GLMsingle are available at glmsingle.org.

To assess the impact of GLMsingle, we evaluate four different types of single-trial response estimates (henceforth, beta versions). The first arises from a baseline procedure that reflects a typical GLM approach for fMRI analysis (beta version $b1$), and each subsequent beta version ($b2$-$b4$) incorporates an additional strategy for optimizing model fits and mitigating the effects of noise. The final beta version ($b4$) contains the complete set of optimizations provided by the GLMsingle toolbox. The GLMsingle algorithm consists of the following steps:

1. A baseline single-trial GLM is used to model each stimulus trial separately using a canonical HRF. This provides a useful baseline for comparison ($b1$: AssumeHRF).

2. An optimal HRF is identified for each voxel (Allen et al., 2022) by iteratively fitting a set of GLMs, each time using a different HRF from a library of 20 HRFs. For each voxel, we identify the HRF that provides the best fit to the data (highest variance explained), and inherit the single-trial betas associated with that HRF ($b2$: FitHRF).

3. GLMdenoise (Kay et al., 2013; Charest et al., 2018) is used to determine nuisance regressors to include in the model. Principal components analysis is applied to time-series data from a pool of noise voxels (see Materials and methods for details), and the top principal components are added one at a time to the GLM until cross-validated variance explained is maximized on-average across voxels ($b3$: FitHRF + GLMdenoise).

4. With the nuisance regressors determined, fractional ridge regression (Rokem and Kay, 2020) is used to regularize the single-trial betas, using a custom amount of regularization for each voxel, determined via cross-validation ($b4$: FitHRF + GLMdenoise + RR).

GLMsingle improves the reliability of beta estimates

We first examined the effect of GLMsingle on the test-retest reliability of voxels across relevant regions of visual cortex in NSD and BOLD5000 (Figure 2). For NSD, we analyzed the first 10 scan sessions of data from each of the first 4 subjects (each scan session consisted of 12 x 5.0 min runs). Each scan session was analyzed separately, and each scan session contained, on average, 35 conditions with 3 trials each, 107 conditions with 2 trials each, and 431 conditions with 1 trial each. For BOLD5000, we analyzed the complete dataset, consisting of subjects CSI1-4. These data were collected over 15 scan sessions (each scan session consisted of either 9 or 10 runs lasting 6 min and 28 s each). Groups of 5 scan sessions were analyzed jointly, and each group resulted in, on average, 0 conditions with 4 trials each, 9 conditions with 3 trials each, 40 conditions with 2 trials each, and 1642 conditions with 1 trial each. Subject CSI4 completed only 9 of 15 experimental sessions, and their data were analyzed in two partitions of 5 and 4 sessions each. Our reliability procedure measures the consistency of a voxel’s response profile (using Pearson $r$) over repeated presentations of the same stimuli, revealing areas of the brain containing stable BOLD responses. This straightforward approach enables direct comparison of data quality between different beta versions.

Figure 2 with 1 supplement see all

Download asset Open asset

We directly compared the $b1$ and $b4$ beta versions for each subject within a liberal mask of visual cortex (nsdgeneral ROI), finding widespread increases in reliability when comparing GLMsingle to baseline (Figure 2a). The positive effect is nearly uniform across voxels in NSD. In BOLD5000, as in NSD, we see aggregate benefits when comparing $b1$ and $b4$, though results for individual voxels are more variable. A likely explanation for this is that reliability metrics are inherently noisier due to the smaller number of repeated stimuli in BOLD5000. In addition, the magnitude of the benefits of b4 over b1 are somewhat smaller in BOLD5000 compared to NSD. This is likely true for a number of reasons, including the generally lower SNR in BOLD5000 (as a data-driven technique, GLMsingle is more effective at higher levels of SNR) and the relatively long inter-stimulus interval in BOLD5000 (ridge regression is expected to have smaller impact when there is less overlap of the BOLD response across successive trials).

To summarize the impact of GLMsingle in NSD and BOLD5000, we compared the performance of b1-b4 for individual subjects, across different voxel reliability thresholds (Figure 2b). We find that all subjects show clear improvement from $b1$ to $b4$ and that the improvement in reliability due to GLMsingle tends to increase when examining voxels that respond more reliably to experimental stimuli. Furthermore, examining reliability in intermediate beta versions ($b2$ and $b3$) – which implement HRF optimization and GLMdenoise, respectively – reveals that each successive stage of processing in GLMsingle tends to confer added benefit to voxel reliability compared to baseline ($b1$).

To better understand the nature of the HRF fitting procedure, we examined selected HRFs across visually-responsive cortex of a representative subject (Figure 2—figure supplement 1). In active voxels, we observe a structured, low-frequency spatial gradient in HRF indices (Figure 2—figure supplement 1b). Moreover, identified HRF indices are highly consistent from session to session (Figure 2—figure supplement 1c). This provides evidence that, beyond merely capturing subject-specific deviation from the assumed HRF, the FitHRF procedure confers benefit by capturing voxel-wise differences in HRF shapes. We note that there are strong biophysical reasons to expect voxel-wise differences in HRF shapes, related to variation in the brain’s vasculature (Kay et al., 2020).

We next compared GLMsingle to Least-Squares Separate (LSS), a popular technique for robust signal estimation in rapid event-related designs (Mumford et al., 2012; Mumford et al., 2014; Abdulrahman and Henson, 2016). The LSS procedure fits a separate GLM for each stimulus, where the trial of interest is modeled as one regressor, and all other (non-target) trials are collapsed into a second regressor. LSS provides a useful point of comparison for ridge regression, as both strategies seek to mitigate the instabilities in GLM estimation that can arise from having correlated single-trial predictors. To directly compare GLMsingle to LSS, we computed auxiliary GLMsingle beta versions that do not incorporate GLMdenoise. This allows us to isolate the effect of the GLM estimation procedure (i.e. LSS vs. fractional ridge regression).

For both the case of an assumed HRF and the case of voxel-wise tailored HRFs, we find that fractional ridge regression yields more reliable signal estimates than LSS (Figure 3). These improvements are most pronounced in the most reliable voxels (Figure 3c). LSS can be viewed as applying heavy regularization uniformly across voxels, while our ridge regression approach is more flexible, tailoring the degree of regularization to the SNR of each voxel. Heavy regularization may actually degrade the quality of signal estimates in reliable voxels, and our approach avoids this possibility.

Figure 3

Download asset Open asset

We then performed a complete assessment of all auxiliary beta versions and the primary versions ($b1$-$b4$), in order to determine whether any other analysis strategy could achieve parity with $b4$ in the quality of GLM outputs. Reassuringly, when summarizing the relative quality of all 8 beta versions over a range of reliability thresholds, we observe superior performance from $b4$, the default output of GLMsingle (Figure 3a). For these and subsequent findings, it is important to note that differences between beta versions should be interpreted with caution at very high reliability thresholds, as voxel counts may be quite low for certain subjects.

GLMsingle relies on an internal cross-validation procedure through which key hyperparameters (the number of noise regressors and the voxel-wise levels of ridge regression regularization) are optimized to maximize the consistency of responses across condition repetitions. This raises a possible concern that our reliability estimates (e.g. Figure 2) are somewhat optimistic. As a strict assessment of reliability, we repeated the reliability quantification for each of the 8 beta versions, this time computing test-retest correlation values using only beta responses obtained from completely separate data partitions. We find that results are broadly unchanged using this more stringent evaluation procedure (Figure 3b).

As a further test of the general applicability of GLMsingle, we repeated the above procedures using data from the music-listening component of StudyForrest (Hanke et al., 2015). This dataset measures brain responses as subjects listen to 25 distinct 6-s musical clips from 5 genres, with 8 trials per condition for each of 16 subjects. Each condition occurred once per functional run, and each subject completed one session of data consisting of 8 runs. This dataset differs from NSD and BOLD5000 in several key respects: the scale (there are far fewer trials), the task modality (auditory, as opposed to visual), and the use of a jittered inter-stimulus interval (the delay between trials is variable between 4 and 8 s). As in NSD and BOLD5000, we observe substantial improvements in reliability through the application of GLMsingle (Figure 4a–c), these improvements are consistent across subjects (Figure 4d), and each individual component of GLMsingle confers added benefit in reliability compared to the baseline GLM. These findings, arising from a smaller scale dataset that may more closely resemble a typical fMRI study, suggest the general applicability of GLMsingle to a wide range of datasets.

Figure 4

Download asset Open asset

GLMsingle helps disentangle neural responses to neighboring trials

Thus far, we have established that GLMsingle provides BOLD response estimates that have substantially improved reliability compared to a baseline GLM. In the remainder of this paper, we explore whether these improvements have tangible consequences for downstream analyses relevant to cognitive and systems neuroscience. We first examine whether GLMsingle is able to more effectively disentangle neural responses to proximal stimuli, as inaccurate single-trial GLM estimation may manifest as high similarity (temporal autocorrelation) between beta maps from nearby trials. We computed dataset-averaged temporal similarity matrices, revealing the degree of temporal autocorrelation in each beta version (Figure 5). Temporal autocorrelation manifests as non-zero correlation values off the diagonal of the temporal similarity matrices, and is presumably undesirable.

Figure 5

Download asset Open asset

In a baseline GLM that uses a canonical HRF and ordinary least-squares (OLS) fitting ($b1$), we observe striking patterns of temporal autocorrelation extending several dozen trials forward in time. This is true in both NSD, which has a rapid event-related design (a new stimulus presented every 4 s), as well as in BOLD5000, where stimuli are spaced 10 s apart to alleviate issues relating to signal overlap. To quantify these effects, we compute mean temporal autocorrelation as a function of time post-stimulus for each beta version. In NSD, for the baseline GLM ($b1$), positive correlations are as high as $r=0.5$ for consecutive trials, and gradually reduce to around $r=0$ after around 100 s (Figure 5a). In BOLD5000, $b1$ autocorrelation peaks as high as around $r=0.4$ for consecutive trials, requiring nearly 160 s to reduce to $r=0$ (Figure 5b). We speculate that the relatively long timescale of the correlations reflects the long timescale of hemodynamic responses (the post-undershoot can extend for 30 s or longer) and/or the slow nature of (low-frequency) physiological noise related to cardiac and respiratory variation. Notably, mean beta maps from successive trials in NSD are anticorrelated for $b1$, a known artifact of OLS fitting in the case of high multicollinearity between GLM predictors (Mumford et al., 2014; Soch et al., 2020).

When applying GLMsingle, these patterns of temporal autocorrelation change dramatically. In NSD $b4$, autocorrelation drops to $r=0$ much more rapidly than in $b1$, and in BOLD5000, beta maps from successive trials in $b4$ are now nearly uncorrelated on average. This is an expected outcome, since the stimuli in NSD and BOLD5000 are ordered pseudorandomly. In both datasets, an intermediate beta version ($b2$) containing only HRF optimization confers marginal benefit over $b1$, but the most dramatic improvements come from the addition of both GLMdenoise and fractional ridge regression ($b4$). Overall, these results demonstrate the utility of GLMsingle for disentangling neural responses to nearby stimuli in event-related designs, even when events are presented relatively slowly (as in BOLD5000).

GLMsingle improves between-subject representational similarity across datasets

Large-scale datasets such as NSD and BOLD5000 are well-suited for representational analyses (e.g. RSA) that compare evoked neural response patterns between individual subjects, across different experimental modalities, and against computational models (e.g. deep neural networks, see Kriegeskorte, 2015; Serre, 2019 for review.) In almost all such studies, representational analyses presume that the same set of stimuli will evoke reasonably similar responses across subjects. As such, given the ubiquity of noise in fMRI, it is reasonable to expect that improving the accuracy of single-trial response estimates should yield representations that are more similar across individuals.

To compare representations between subjects, we used the approach of RSA (Kriegeskorte et al., 2008). First, we isolated stimuli that overlap between BOLD5000 and the subset of NSD analyzed for this manuscript (the first 10 sessions from each subject). Using these 241 stimuli, we constructed representational dissimilarity matrices (RDMs) using repetition-averaged betas from each individual, and then correlated all pairs of subject RDMs within and between datasets. Note that GLMsingle is not designed to enhance or optimize cross-subject representational similarity; as such, it is informative to examine RSA correlations between subjects as a way of assessing methods for denoising (Charest et al., 2018). Strikingly, in comparing beta versions $b1$ and $b4$, we observe a consistent strengthening of RDM correspondence (Figure 6b). This trend held within NSD, within BOLD5000, and when comparing the RDMs of subject pairs between the two datasets. The latter result is especially notable given the many methodological differences between NSD and BOLD5000: fMRI data were collected at different sites on different scanners, at different field strengths (7T vs. 3T), with different behavioral tasks, and with different inter-stimulus intervals (4 s vs. 10 s).

Figure 6

Download asset Open asset

These results indicate that GLMsingle, through its multifaceted approach to mitigating the effects of noise, helps reveal meaningful shared variance in neural responses across individuals who viewed the same stimuli. The GLMsingle toolbox may therefore be a key resource for future fMRI studies seeking to stitch together data across subjects from different sites or cohorts.

GLMsingle enables fine-grained image-level MVPA decoding

As a final analysis, we assessed the effect of GLMsingle on the results of multivoxel pattern analysis (MVPA). In a “one-vs.-many” classification paradigm, we trained linear SVM models for each subject to predict image identity from neural response patterns. The baseline GLM ($b1$) classification accuracy was slightly above chance on average for the subjects in NSD and BOLD5000 when including all visual cortex voxels (Figure 7a, blue traces). Performing the same MVPA procedure using GLMsingle betas ($b4$), we observe that mean accuracy approximately triples in NSD and doubles in BOLD5000 (Figure 7a, red traces). Moreover, in both datasets, we observe a substantial increase in classification accuracies with increasing voxel reliability threshold, with the most dramatic improvements achieved using $b4$ in NSD (Figure 7a, left panel, right-most bins).

Figure 7

Download asset Open asset

The level of performance that GLMsingle facilitates on this challenging multi-way decoding task highlights the ability of the technique to accurately identify and model the stable structure contained in noisy fMRI time-series. To illustrate this point, we performed 2D multidimensional scaling (MDS, Borg and Groenen, 2005) using the NSD betas that were included in MVPA. Comparing results between beta versions $b1$ and $b4$, we observe improved clarity of an animacy division in the representational space of an example subject (Figure 7b).

As scientific datasets grow in scale and scope, new techniques for data processing will help to unlock their potential. This is especially true in human neuroscience where data remain both expensive and time-consuming to collect (Naselaris et al., 2021). This paper has introduced GLMsingle, a publicly available toolbox for analyzing fMRI time-series data that leverages data-driven techniques to improve the accuracy of single-trial fMRI response estimates. We have tested GLMsingle extensively using NSD and BOLD5000, two of the largest fMRI datasets that densely sample responses within individuals. For both datasets, analyses of the response estimates provided by GLMsingle indicate substantial improvements in several key metrics of interest to neuroscientists: (i) enhanced test-retest reliability of voxel response profiles, a straightforward metric of data quality; (ii) reduced temporal autocorrelation, a common fMRI effect that is presumably undesirable and especially prominent in rapid event-related designs; (iii) increased representational similarity across subjects both within and across datasets; and (iv) improved multivariate pattern classification performance when discriminating responses evoked by individual images.

The experimental data used in this paper are all previously published and available online at https://naturalscenesdataset.org, https://doi.org/10.18112/openneuro.ds001499.v1.3.1, and https://doi.org/10.18112/openneuro.ds000113.v1.3.0. The GLMsingle toolbox is available on GitHub at https://github.com/cvnlab/GLMsingle, (copy archived at swh:1:rev:3a1a580eae6bc7e221cbe01101be3c669687dc6a). Code used for the analyses demonstrated in this paper is publicly available on GitHub at https://github.com/jacob-prince/GLMsingle_paper, (copy archived at swh:1:rev:08cdd25e4fd96f545056e516ec1910ab749e1887). Source data files linked to the results plotted in Figures 2-7 and Figure 2—figure supplement 1 are available in an OSF repository at https://osf.io/d4p98.

1. 1. Kay K
   2. Kurzawski J
   3. Charest I
   4. Prince JS

   (2022) Open Science Framework

   GLMsingle.

   https://doi.org/10.17605/OSF.IO/D4P98

1. 1. Hanke M
   2. Baumgartner FJ
   3. Ibe P
   4. Kaule FR
   5. Pollmann S
   6. Speck O
   7. Zinke W
   8. Stadler J

   (2018) OpenNeuro

   Forrest Gump.

   https://doi.org/10.18112/openneuro.ds000113.v1.3.0
2. 1. Chang N
   2. Pyles J
   3. Prince J
   4. Marcus A
   5. Gupta A
   6. Tarr M
   7. Aminoff E

   (2019) OpenNeuro

   BOLD5000.

   https://doi.org/10.18112/openneuro.ds001499.v1.3.1
3. 1. Allen EJ
   2. St-Yves G
   3. Wu Y
   4. Breedlove JL
   5. Prince JS
   6. Dowdle LT
   7. Nau M
   8. Caron B
   9. Pestilli F
   10. Charest I
   11. Hutchinson JB
   12. Naselaris T
   13. Kay K

   (2021) University of Minnesota

   ID naturalscenesdataset. Natural Scenes Dataset.

   https://naturalscenesdataset.org/

1. 1. Abdulrahman H
   2. Henson RN

   (2016) Effect of trial-to-trial variability on optimal event-related fmri design: implications for beta-series correlation and multi-voxel pattern analysis

   NeuroImage 125:756–766.

   https://doi.org/10.1016/j.neuroimage.2015.11.009

   • PubMed
   • Google Scholar
2. 1. Allen EJ
   2. St-Yves G
   3. Wu Y
   4. Breedlove JL
   5. Prince JS
   6. Dowdle LT
   7. Nau M
   8. Caron B
   9. Pestilli F
   10. Charest I
   11. Hutchinson JB
   12. Naselaris T
   13. Kay K

   (2022) A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence

   Nature Neuroscience 25:116–126.

   https://doi.org/10.1038/s41593-021-00962-x

   • PubMed
   • Google Scholar
3. Conference

   1. Bai B
   2. Kantor P

   (2007) A shape-based finite impulse response model for functional brain images

   2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro. pp. 440–443.

   https://doi.org/10.1109/ISBI.2007.356883

   • Google Scholar
4. 1. Bao P
   2. She L
   3. McGill M
   4. Tsao DY

   (2020) A map of object space in primate inferotemporal cortex

   Nature 583:103–108.

   https://doi.org/10.1038/s41586-020-2350-5

   • PubMed
   • Google Scholar
5. 1. Blauch NM
   2. Behrmann M
   3. Plaut DC

   (2022) A connectivity-constrained computational account of topographic organization in primate high-level visual cortex

   PNAS 119:e2112566119.

   https://doi.org/10.1073/pnas.2112566119

   • Google Scholar
6. Book

   1. Borg I
   2. Groenen PJ

   (2005) Modern Multidimensional Scaling: Theory and Applications

   Springer Science & Business Media.

   https://doi.org/10.18637/jss.v014.b04

   • Google Scholar
7. 1. Chang N
   2. Pyles JA
   3. Marcus A
   4. Gupta A
   5. Tarr MJ
   6. Aminoff EM

   (2019) BOLD5000, a public fmri dataset while viewing 5000 visual images

   Scientific Data 6:49.

   https://doi.org/10.1038/s41597-019-0052-3

   • PubMed
   • Google Scholar
8. 1. Charest I
   2. Kriegeskorte N
   3. Kay KN

   (2018) GLMdenoise improves multivariate pattern analysis of fMRI data

   NeuroImage 183:606–616.

   https://doi.org/10.1016/j.neuroimage.2018.08.064

   • PubMed
   • Google Scholar
9. 1. Dale AM

   (1999) Optimal experimental design for event-related fmri

   Human Brain Mapping 8:109–114.

   https://doi.org/10.1002/(SICI)1097-0193(1999)8:2/3

   • PubMed
   • Google Scholar
10. Conference

    1. Deng J
    2. Dong W
    3. Socher R
    4. Li LJ
    5. Li K
    6. Fei-Fei L

    (2009) Imagenet: A large-scale hierarchical image database

    2009 IEEE conference on computer vision and pattern recognition. pp. 248–255.

    https://doi.org/10.1109/CVPR.2009.5206848

    • Google Scholar
11. 1. DiCarlo JJ
    2. Zoccolan D
    3. Rust NC

    (2012) How does the brain solve visual object recognition?

    Neuron 73:415–434.

    https://doi.org/10.1016/j.neuron.2012.01.010

    • PubMed
    • Google Scholar
12. 1. Diedrichsen J
    2. Kriegeskorte N

    (2017) Representational models: a common framework for understanding encoding, pattern-component, and representational-similarity analysis

    PLOS Computational Biology 13:e1005508.

    https://doi.org/10.1371/journal.pcbi.1005508

    • PubMed
    • Google Scholar
13. 1. D’Isanto A
    2. Polsterer KL

    (2018) Photometric redshift estimation via deep learning-generalized and pre-classification-less, image based, fully probabilistic redshifts

    Astronomy & Astrophysics 609:A111.

    https://doi.org/10.1051/0004-6361/201731326

    • Google Scholar
14. 1. Doshi F
    2. Konkle T

    (2021) Organizational motifs of cortical responses to objects emerge in topographic projections of deep neural networks

    Journal of Vision 21:2226.

    https://doi.org/10.1167/jov.21.9.2226

    • Google Scholar
15. 1. Esteban O
    2. Markiewicz CJ
    3. Blair RW
    4. Moodie CA
    5. Isik AI
    6. Erramuzpe A
    7. Kent JD
    8. Goncalves M
    9. DuPre E
    10. Snyder M
    11. Oya H
    12. Ghosh SS
    13. Wright J
    14. Durnez J
    15. Poldrack RA
    16. Gorgolewski KJ

    (2019) FMRIPrep: a robust preprocessing pipeline for functional MRI

    Nature Methods 16:111–116.

    https://doi.org/10.1038/s41592-018-0235-4

    • PubMed
    • Google Scholar
16. 1. Freeman J
    2. Ziemba CM
    3. Heeger DJ
    4. Simoncelli EP
    5. Movshon JA

    (2013) A functional and perceptual signature of the second visual area in primates

    Nature Neuroscience 16:974–981.

    https://doi.org/10.1038/nn.3402

    • PubMed
    • Google Scholar
17. 1. Handwerker DA
    2. Ollinger JM
    3. D’Esposito M

    (2004) Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses

    NeuroImage 21:1639–1651.

    https://doi.org/10.1016/j.neuroimage.2003.11.029

    • PubMed
    • Google Scholar
18. 1. Handwerker DA
    2. Gonzalez-Castillo J
    3. D’Esposito M
    4. Bandettini PA

    (2012) The continuing challenge of understanding and modeling hemodynamic variation in fMRI

    NeuroImage 62:1017–1023.

    https://doi.org/10.1016/j.neuroimage.2012.02.015

    • PubMed
    • Google Scholar
19. 1. Hanke M
    2. Dinga R
    3. Häusler C
    4. Guntupalli JS
    5. Casey M
    6. Kaule FR
    7. Stadler J

    (2015) High-resolution 7-tesla fmri data on the perception of musical genres – an extension to the studyforrest dataset

    F1000Research 4:174.

    https://doi.org/10.12688/f1000research.6679.1

    • Google Scholar
20. 1. Haxby JV
    2. Gobbini MI
    3. Furey ML
    4. Ishai A
    5. Schouten JL
    6. Pietrini P

    (2001) Distributed and overlapping representations of faces and objects in ventral temporal cortex

    Science 293:2425–2430.

    https://doi.org/10.1126/science.1063736

    • PubMed
    • Google Scholar
21. 1. Hebart MN
    2. Dickter AH
    3. Kidder A
    4. Kwok WY
    5. Corriveau A
    6. Van Wicklin C
    7. Baker CI

    (2019) Things: a database of 1,854 object concepts and more than 26,000 naturalistic object images

    PLOS ONE 14:e0223792.

    https://doi.org/10.1371/journal.pone.0223792

    • PubMed
    • Google Scholar
22. 1. Hoerl AE
    2. Kennard RW

    (1970) Ridge regression: biased estimation for nonorthogonal problems

    Technometrics 12:55–67.

    https://doi.org/10.1080/00401706.1970.10488634

    • Google Scholar
23. 1. Horikawa T
    2. Kamitani Y

    (2017) Generic decoding of seen and imagined objects using hierarchical visual features

    Nature Communications 8:15037.

    https://doi.org/10.1038/ncomms15037

    • PubMed
    • Google Scholar
24. 1. Iten R
    2. Metger T
    3. Wilming H
    4. Del Rio L
    5. Renner R

    (2020) Discovering physical concepts with neural networks

    Physical Review Letters 124:010508.

    https://doi.org/10.1103/PhysRevLett.124.010508

    • PubMed
    • Google Scholar
25. 1. Jumper J
    2. Evans R
    3. Pritzel A
    4. Green T
    5. Figurnov M
    6. Ronneberger O
    7. Tunyasuvunakool K
    8. Bates R
    9. Žídek A
    10. Potapenko A
    11. Bridgland A
    12. Meyer C
    13. Kohl SAA
    14. Ballard AJ
    15. Cowie A
    16. Romera-Paredes B
    17. Nikolov S
    18. Jain R
    19. Adler J
    20. Back T
    21. Petersen S
    22. Reiman D
    23. Clancy E
    24. Zielinski M
    25. Steinegger M
    26. Pacholska M
    27. Berghammer T
    28. Bodenstein S
    29. Silver D
    30. Vinyals O
    31. Senior AW
    32. Kavukcuoglu K
    33. Kohli P
    34. Hassabis D

    (2021) Highly accurate protein structure prediction with alphafold

    Nature 596:583–589.

    https://doi.org/10.1038/s41586-021-03819-2

    • PubMed
    • Google Scholar
26. 1. Kaniuth P
    2. Hebart MN

    (2022) Feature-reweighted representational similarity analysis: A method for improving the fit between computational models, brains, and behavior

    NeuroImage 257:119294.

    https://doi.org/10.1016/j.neuroimage.2022.119294

    • Google Scholar
27. Report

    1. Kay KN
    2. David SV
    3. Prenger RJ
    4. Hansen KA
    5. Gallant JL

    (2008)

    Modeling low-frequency fluctuation and hemodynamic response timecourse in event-related fmri Technical report

    Wiley Online Library.

    • Google Scholar
28. 1. Kay KN
    2. Rokem A
    3. Winawer J
    4. Dougherty RF
    5. Wandell BA

    (2013) GLMdenoise: a fast, automated technique for denoising task-based fmri data

    Frontiers in Neuroscience 7:247.

    https://doi.org/10.3389/fnins.2013.00247

    • PubMed
    • Google Scholar
29. 1. Kay K
    2. Jamison KW
    3. Zhang RY
    4. Uğurbil K

    (2020) A temporal decomposition method for identifying venous effects in task-based fMRI

    Nature Methods 17:1033–1039.

    https://doi.org/10.1038/s41592-020-0941-6

    • PubMed
    • Google Scholar
30. 1. Kay K

    (2022) The risk of bias in denoising methods: examples from neuroimaging

    PLOS ONE 17:e0270895.

    https://doi.org/10.1371/journal.pone.0270895

    • Google Scholar
31. 1. Kriegeskorte N
    2. Mur M
    3. Bandettini PA

    (2008) Representational similarity analysis-connecting the branches of systems neuroscience

    Frontiers in Systems Neuroscience 2:4.

    https://doi.org/10.3389/neuro.06.004.2008

    • PubMed
    • Google Scholar
32. 1. Kriegeskorte N

    (2015) Deep neural networks: a new framework for modeling biological vision and brain information processing

    Annual Review of Vision Science 1:417–446.

    https://doi.org/10.1146/annurev-vision-082114-035447

    • PubMed
    • Google Scholar
33. 1. Kriegeskorte N
    2. Wei XX

    (2021) Neural tuning and representational geometry

    Nature Reviews Neuroscience 22:703–718.

    https://doi.org/10.1038/s41583-021-00502-3

    • Google Scholar
34. Conference

    1. Kumar A
    2. Feng L

    (2014)

    Efficient regularization of temporal autocorrelation estimates in fmri data

    In The 15th International Conference on Biomedical Engineering. pp. 88–91.

    • Google Scholar
35. Preprint

    1. Lee H
    2. Margalit E
    3. Jozwik KM
    4. Cohen MA
    5. Kanwisher N
    6. Yamins DL
    7. DiCarlo JJ

    (2020) Topographic Deep Artificial Neural Networks Reproduce the Hallmarks of the Primate Inferior Temporal Cortex Face Processing Network

    bioRxiv.

    https://doi.org/10.1101/2020.07.09.185116

    • Google Scholar
36. Preprint

    1. Li DSP
    2. Bonner MF

    (2022) Emergent Selectivity for Scenes, Object Properties, and Contour Statistics in Feedforward Models of Scene-Preferring Cortex

    bioRxiv.

    https://doi.org/10.1101/2021.09.24.461733

    • Google Scholar
37. Conference

    1. Lin TY
    2. Maire M
    3. Belongie S
    4. Hays J
    5. Perona P
    6. Ramanan D
    7. Dollár P
    8. Zitnick CL

    (2014)

    Microsoft coco: Common objects in context

    In European conference on computer vision. pp. 740–755.

    • Google Scholar
38. 1. Liu TT
    2. Frank LR
    3. Wong EC
    4. Buxton RB

    (2001) Detection power, estimation efficiency, and predictability in event-related fMRI

    NeuroImage 13:759–773.

    https://doi.org/10.1006/nimg.2000.0728

    • PubMed
    • Google Scholar
39. 1. Liu TT

    (2016) Noise contributions to the fMRI signal: an overview

    NeuroImage 143:141–151.

    https://doi.org/10.1016/j.neuroimage.2016.09.008

    • PubMed
    • Google Scholar
40. 1. Long B
    2. Yu CP
    3. Konkle T

    (2018) Mid-level visual features underlie the high-level categorical organization of the ventral stream

    PNAS 115:E9015–E9024.

    https://doi.org/10.1073/pnas.1719616115

    • PubMed
    • Google Scholar
41. Preprint

    1. Marques T
    2. Schrimpf M
    3. DiCarlo JJ

    (2021) Multi-Scale Hierarchical Neural Network Models That Bridge from Single Neurons in the Primate Primary Visual Cortex to Object Recognition Behavior

    bioRxiv.

    https://doi.org/10.1101/2021.03.01.433495

    • Google Scholar
42. 1. Monti MM

    (2011) Statistical analysis of fMRI time-series: a critical review of the GLM approach

    Frontiers in Human Neuroscience 5:28.

    https://doi.org/10.3389/fnhum.2011.00028

    • PubMed
    • Google Scholar
43. 1. Mumford JA
    2. Turner BO
    3. Ashby FG
    4. Poldrack RA

    (2012) Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses

    NeuroImage 59:2636–2643.

    https://doi.org/10.1016/j.neuroimage.2011.08.076

    • PubMed
    • Google Scholar
44. 1. Mumford JA
    2. Davis T
    3. Poldrack RA

    (2014) The impact of study design on pattern estimation for single-trial multivariate pattern analysis

    NeuroImage 103:130–138.

    https://doi.org/10.1016/j.neuroimage.2014.09.026

    • PubMed
    • Google Scholar
45. 1. Naselaris T
    2. Kay KN
    3. Nishimoto S
    4. Gallant JL

    (2011) Encoding and decoding in fMRI

    NeuroImage 56:400–410.

    https://doi.org/10.1016/j.neuroimage.2010.07.073

    • PubMed
    • Google Scholar
46. 1. Naselaris T
    2. Allen E
    3. Kay K

    (2021) Extensive sampling for complete models of individual brains

    Current Opinion in Behavioral Sciences 40:45–51.

    https://doi.org/10.1016/j.cobeha.2020.12.008

    • Google Scholar
47. 1. Nili H
    2. Wingfield C
    3. Walther A
    4. Su L
    5. Marslen-Wilson W
    6. Kriegeskorte N

    (2014) A toolbox for representational similarity analysis

    PLOS Computational Biology 10:e1003553.

    https://doi.org/10.1371/journal.pcbi.1003553

    • PubMed
    • Google Scholar
48. 1. Norman KA
    2. Polyn SM
    3. Detre GJ
    4. Haxby JV

    (2006) Beyond mind-reading: multi-voxel pattern analysis of fMRI data

    Trends in Cognitive Sciences 10:424–430.

    https://doi.org/10.1016/j.tics.2006.07.005

    • PubMed
    • Google Scholar
49. 1. Olszowy W
    2. Aston J
    3. Rua C
    4. Williams GB

    (2019) Accurate autocorrelation modeling substantially improves fMRI reliability

    Nature Communications 10:1–11.

    https://doi.org/10.1038/s41467-019-09230-w

    • PubMed
    • Google Scholar
50. Book

    1. Poldrack RA
    2. Mumford JA
    3. Nichols TE

    (2011) Handbook of Functional MRI Data Analysis

    Cambridge University Press.

    https://doi.org/10.1017/CBO9780511895029

    • Google Scholar
51. 1. Popham SF
    2. Huth AG
    3. Bilenko NY
    4. Deniz F
    5. Gao JS
    6. Nunez-Elizalde AO
    7. Gallant JL

    (2021) Visual and linguistic semantic representations are aligned at the border of human visual cortex

    Nature Neuroscience 24:1628–1636.

    https://doi.org/10.1038/s41593-021-00921-6

    • PubMed
    • Google Scholar
52. 1. Ravuri S
    2. Lenc K
    3. Willson M
    4. Kangin D
    5. Lam R
    6. Mirowski P
    7. Fitzsimons M
    8. Athanassiadou M
    9. Kashem S
    10. Madge S
    11. Prudden R
    12. Mandhane A
    13. Clark A
    14. Brock A
    15. Simonyan K
    16. Hadsell R
    17. Robinson N
    18. Clancy E
    19. Arribas A
    20. Mohamed S

    (2021) Skilful precipitation nowcasting using deep generative models of radar

    Nature 597:672–677.

    https://doi.org/10.1038/s41586-021-03854-z

    • PubMed
    • Google Scholar
53. 1. Richards BA
    2. Lillicrap TP
    3. Beaudoin P
    4. Bengio Y
    5. Bogacz R
    6. Christensen A
    7. Clopath C
    8. Costa RP
    9. de Berker A
    10. Ganguli S
    11. Gillon CJ
    12. Hafner D
    13. Kepecs A
    14. Kriegeskorte N
    15. Latham P
    16. Lindsay GW
    17. Miller KD
    18. Naud R
    19. Pack CC
    20. Poirazi P
    21. Roelfsema P
    22. Sacramento J
    23. Saxe A
    24. Scellier B
    25. Schapiro AC
    26. Senn W
    27. Wayne G
    28. Yamins D
    29. Zenke F
    30. Zylberberg J
    31. Therien D
    32. Kording KP

    (2019) A deep learning framework for neuroscience

    Nature Neuroscience 22:1761–1770.

    https://doi.org/10.1038/s41593-019-0520-2

    • PubMed
    • Google Scholar
54. 1. Rokem A
    2. Kay K

    (2020) Fractional ridge regression: a fast, interpretable reparameterization of ridge regression

    GigaScience 9:giaa133.

    https://doi.org/10.1093/gigascience/giaa133

    • PubMed
    • Google Scholar
55. 1. Schawinski K
    2. Turp MD
    3. Zhang C

    (2018) Exploring galaxy evolution with generative models

    Astronomy & Astrophysics 616:L16.

    https://doi.org/10.1051/0004-6361/201833800

    • PubMed
    • Google Scholar
56. 1. Serre T

    (2019) Deep learning: the good, the bad, and the ugly

    Annual Review of Vision Science 5:399–426.

    https://doi.org/10.1146/annurev-vision-091718-014951

    • PubMed
    • Google Scholar
57. 1. Soch J
    2. Allefeld C
    3. Haynes JD

    (2020) Inverse transformed encoding models-a solution to the problem of correlated trial-by-trial parameter estimates in fmri decoding

    NeuroImage 209:116449.

    https://doi.org/10.1016/j.neuroimage.2019.116449

    • PubMed
    • Google Scholar
58. Website

    1. Wilson M

    (2018) What is smart HDRhdr? explaining Aapple’s new camera tech | trusted reviews

    Accessed March 30, 2021.

    https://www.trustedreviews.com/news/what-is-smart-hdr-3565603
59. 1. Woolrich MW
    2. Ripley BD
    3. Brady M
    4. Smith SM

    (2001) Temporal autocorrelation in univariate linear modeling of fMRI data

    NeuroImage 14:1370–1386.

    https://doi.org/10.1006/nimg.2001.0931

    • PubMed
    • Google Scholar
60. Conference

    1. Xiao J
    2. Hays J
    3. Ehinger KA
    4. Oliva A
    5. Torralba A

    (2010) SUNSun database: Large-scale scene recognition from abbey to zoo

    IEEE computer society conference on computer vision and pattern recognition.

    https://doi.org/10.1109/CVPR.2010.5539970

    • Google Scholar
61. 1. Yamins DLK
    2. Hong H
    3. Cadieu CF
    4. Solomon EA
    5. Seibert D
    6. DiCarlo JJ

    (2014) Performance-optimized hierarchical models predict neural responses in higher visual cortex

    PNAS 111:8619–8624.

    https://doi.org/10.1073/pnas.1403112111

    • PubMed
    • Google Scholar
62. Preprint

    1. Zhang Y
    2. Zhou K
    3. Bao P
    4. Liu J

    (2021) Principles Governing the Topological Organization of Object Selectivities in Ventral Temporal Cortex

    bioRxiv.

    https://doi.org/10.1101/2021.09.15.460220

    • Google Scholar

Author details

1. Jacob S Prince

   Department of Psychology, Harvard University, Cambridge, United States

   Contribution

   Conceptualization, Resources, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   For correspondence

   jacob.samuel.prince@gmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-6169-9503

2. Ian Charest

   1. Center for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
   2. cerebrUM, Département de Psychologie, Université de Montréal, Montréal, Canada

   Contribution

   Resources, Software, Validation, Methodology

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-3939-3003

3. Jan W Kurzawski

   Department of Psychology, New York University, New York, United States

   Contribution

   Resources, Software, Visualization, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2781-1236

4. John A Pyles

   Center for Human Neuroscience, Department of Psychology, University of Washington, Seattle, United States

   Contribution

   Conceptualization, Resources, Data curation, Supervision

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7627-0504

5. Michael J Tarr

   Department of Psychology, Neuroscience Institute, Carnegie Mellon University, Pittsburgh, United States

   Contribution

   Conceptualization, Data curation, Supervision, Project administration, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4724-1744

6. Kendrick N Kay

   Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, United States

   Contribution

   Conceptualization, Resources, Data curation, Software, Formal analysis, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing – original draft, Project administration, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-6604-9155

• 5,557

  views

• 587

  downloads

• 60

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.