Assessing reliability in neuroimaging research through intra-class effect decomposition (ICED)

  1. Andreas M Brandmaier  Is a corresponding author
  2. Elisabeth Wenger
  3. Nils C Bodammer
  4. Simone Kühn
  5. Naftali Raz
  6. Ulman Lindenberger
  1. Max Planck Institute for Human Development, Germany
  2. University Clinic Hamburg-Eppendorf, Germany

Abstract

Magnetic resonance imaging has become an indispensable tool for studying associations between structural and functional properties of the brain and behavior in humans. However, generally recognized standards for assessing and reporting the reliability of these techniques are still lacking. Here, we introduce a new approach for assessing and reporting reliability, termed intra-class effect decomposition (ICED). ICED uses structural equation modeling of data from a repeated-measures design to decompose reliability into orthogonal sources of measurement error that are associated with different characteristics of the measurements, for example, session, day, or scanning site. This allows researchers to describe the magnitude of different error components, make inferences about error sources, and inform them in planning future studies. We apply ICED to published measurements of myelin content and resting state functional connectivity. These examples illustrate how longitudinal data can be leveraged separately or conjointly with cross-sectional data to obtain more precise estimates of reliability.

Data availability

The dataset on myelin water fraction measurements is freely available at https://osf.io/t68my/ and the link-wise resting state functional connectivity data is available at https://osf.io/8n24x/.

The following previously published data sets were used
    1. Arshad M
    2. Stanley J A
    3. Raz
    4. N
    (2018) Reliability of Myelin Water Fraction in ALIC
    Publicly available at Open Science Framework.

Article and author information

Author details

  1. Andreas M Brandmaier

    Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
    For correspondence
    brandmaier@mpib-berlin.mpg.de
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-8765-6982
  2. Elisabeth Wenger

    Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
  3. Nils C Bodammer

    Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
  4. Simone Kühn

    Clinic and Policlinic for Psychiatry and Psychotherapy, University Clinic Hamburg-Eppendorf, Hamburg, Germany
    Competing interests
    The authors declare that no competing interests exist.
  5. Naftali Raz

    Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.
  6. Ulman Lindenberger

    Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
    Competing interests
    The authors declare that no competing interests exist.

Funding

National Institutes of Health (R01-AG011230)

  • Naftali Raz

Horizon 2020 Framework Programme (732592)

  • Andreas M Brandmaier
  • Simone Kühn
  • Ulman Lindenberger

Max-Planck-Gesellschaft (Open-access funding)

  • Andreas M Brandmaier
  • Elisabeth Wenger
  • Nils C Bodammer
  • Naftali Raz
  • Ulman Lindenberger

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

© 2018, Brandmaier et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,480
    views
  • 280
    downloads
  • 53
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Andreas M Brandmaier
  2. Elisabeth Wenger
  3. Nils C Bodammer
  4. Simone Kühn
  5. Naftali Raz
  6. Ulman Lindenberger
(2018)
Assessing reliability in neuroimaging research through intra-class effect decomposition (ICED)
eLife 7:e35718.
https://doi.org/10.7554/eLife.35718

Share this article

https://doi.org/10.7554/eLife.35718

Further reading

    1. Neuroscience
    Jean-François Brunet
    Review Article

    Historically, the creation of the parasympathetic division of the autonomic nervous system of the vertebrates is inextricably linked to the unification of the cranial and sacral autonomic outflows. There is an intriguing disproportion between the entrenchment of the notion of a ‘cranio-sacral’ pathway, which informs every textbook schematic of the autonomic nervous system since the early XXth century, and the wobbliness of its two roots: an anatomical detail overinterpreted by Walter Holbrook Gaskell (the ‘gap’ between the lumbar and sacral outflows), on which John Newport Langley grafted a piece of physiology (a supposed antagonism of these two outflows on external genitals), repeatedly questioned since, to little avail. I retrace the birth of a flawed scientific concept (the cranio-sacral outflow) and the way in which it ossified instead of dissipated. Then, I suggest that the critique of the ‘cranio-sacral outflow’ invites, in turn, a radical deconstruction of the very notion of a ‘parasympathetic’ outflow, and a more realistic description of the autonomic nervous system.

    1. Neuroscience
    Baher A Ibrahim, Yoshitaka Shinagawa ... Daniel A Llano
    Research Article

    To navigate real-world listening conditions, the auditory system relies on the integration of multiple sources of information. However, to avoid inappropriate cross-talk between inputs, highly connected neural systems need to strike a balance between integration and segregation. Here, we develop a novel approach to examine how repeated neurochemical modules in the mouse inferior colliculus lateral cortex (LC) allow controlled integration of its multimodal inputs. The LC had been impossible to study via imaging because it is buried in a sulcus. Therefore, we coupled two-photon microscopy with the use of a microprism to reveal the first-ever sagittal views of the LC to examine neuronal responses with respect to its neurochemical motifs under anesthetized and awake conditions. This approach revealed marked differences in the acoustic response properties of LC and neighboring non-lemniscal portions of the inferior colliculus. In addition, we observed that the module and matrix cellular motifs of the LC displayed distinct somatosensory and auditory responses. Specifically, neurons in modules demonstrated primarily offset responses to acoustic stimuli with enhancement in responses to bimodal stimuli, whereas matrix neurons showed onset response to acoustic stimuli and suppressed responses to bimodal stimulation. Thus, this new approach revealed that the repeated structural motifs of the LC permit functional integration of multimodal inputs while retaining distinct response properties.