A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies

  1. Victoria T Hunniford
  2. Agnes Grudniewicz
  3. Dean A Fergusson
  4. Joshua Montroy
  5. Emma Grigor
  6. Casey Lansdell
  7. Manoj M Lalu  Is a corresponding author
  8. on behalf of the Canadian Critical Care Translational Biology Group
  1. Ottawa Hospital Research Institute, Canada
  2. University of Ottawa, Canada

Abstract

Background: Multicentric approaches are widely used in clinical trials to assess generalizability of findings, however they are novel in laboratory-based experimentation. It is unclear how multilaboratory studies may differ in conduct and results from single lab studies. Here we synthesized characteristics of these studies and quantitatively compared their outcomes to those generated by single laboratory studies.

Methods: MEDLINE and Embase were systematically searched. Screening and data extractions were completed in duplicate by independent reviewers. Multilaboratory studies investigating interventions using in vivo animal models were included. Study characteristics were extracted. Systematic searches were then performed to identify single center studies matched by intervention and disease. Difference in standardized mean differences (DSMD) was then calculated across studies to assess differences in effect estimates based on study design (>0 indicates larger effects in single center studies).

Results: Sixteen multilaboratory studies met inclusion criteria and were matched to 100 single center studies. The multicenter study design was applied across a diverse range of diseases, including traumatic brain injury, myocardial infarction, and diabetes. The median number of centers was 4 (range 2-6) and the median sample size was 111 (range 23-384) with rodents most frequently used. Multicenter studies adhered to practices that reduce risk of bias significantly more often than single center studies. Multicenter studies also demonstrated significantly smaller effect sizes than single center studies (DSMD 0.72 [95% confidence interval 0.43-1]).

Conclusion: Multilaboratory studies demonstrate trends that have been well recognized in clinical research (i.e. smaller treatment effects with multicentric evaluation and greater rigour in study design). This approach may provide a method to robustly assess interventions and generalizability of findings between laboratories.

Funding: uOttawa Junior Clinical Research Chair; The Ottawa Hospital Anesthesia Alternate Funds Association; Canadian Anesthesia Research Foundation; Government of Ontario Queen Elizabeth II Graduate Scholarship in Science and Technology.

Clinical trial registration: PROSPERO CRD4201809398.

Data availability

The protocol for the effect size comparison was developed a priori and posted on Open Science Framework (https://osf.io/awvs9/).Supplementary documents contains the search strategies, risk of bias assessments, reporting checklists, quality scores, effect sizes, effect size ratios, and standardized mean differences to generate the figures and tables.

Article and author information

Author details

  1. Victoria T Hunniford

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  2. Agnes Grudniewicz

    Telfer School of Management, University of Ottawa, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  3. Dean A Fergusson

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  4. Joshua Montroy

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  5. Emma Grigor

    Faculty of Medicine, University of Ottawa, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  6. Casey Lansdell

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  7. Manoj M Lalu

    Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Canada
    For correspondence
    mlalu@toh.ca
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0322-382X

Funding

QEII Scholarship (Graduate Student Scholarship)

  • Victoria T Hunniford

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

© 2023, Hunniford et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 594
    views
  • 89
    downloads
  • 2
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Victoria T Hunniford
  2. Agnes Grudniewicz
  3. Dean A Fergusson
  4. Joshua Montroy
  5. Emma Grigor
  6. Casey Lansdell
  7. Manoj M Lalu
  8. on behalf of the Canadian Critical Care Translational Biology Group
(2023)
A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies
eLife 12:e76300.
https://doi.org/10.7554/eLife.76300

Share this article

https://doi.org/10.7554/eLife.76300

Further reading

    1. Epidemiology and Global Health
    2. Genetics and Genomics
    Wei Q Deng, Nathan Cawte ... Sonia S Anand
    Research Article

    Background:

    Maternal smoking has been linked to adverse health outcomes in newborns but the extent to which it impacts newborn health has not been quantified through an aggregated cord blood DNA methylation (DNAm) score. Here, we examine the feasibility of using cord blood DNAm scores leveraging large external studies as discovery samples to capture the epigenetic signature of maternal smoking and its influence on newborns in White European and South Asian populations.

    Methods:

    We first examined the association between individual CpGs and cigarette smoking during pregnancy, and smoking exposure in two White European birth cohorts (n=744). Leveraging established CpGs for maternal smoking, we constructed a cord blood epigenetic score of maternal smoking that was validated in one of the European-origin cohorts (n=347). This score was then tested for association with smoking status, secondary smoking exposure during pregnancy, and health outcomes in offspring measured after birth in an independent White European (n=397) and a South Asian birth cohort (n=504).

    Results:

    Several previously reported genes for maternal smoking were supported, with the strongest and most consistent association signal from the GFI1 gene (6 CpGs with p<5 × 10-5). The epigenetic maternal smoking score was strongly associated with smoking status during pregnancy (OR = 1.09 [1.07, 1.10], p=5.5 × 10-33) and more hours of self-reported smoking exposure per week (1.93 [1.27, 2.58], p=7.8 × 10-9) in White Europeans. However, it was not associated with self-reported exposure (p>0.05) among South Asians, likely due to a lack of smoking in this group. The same score was consistently associated with a smaller birth size (–0.37±0.12 cm, p=0.0023) in the South Asian cohort and a lower birth weight (–0.043±0.013 kg, p=0.0011) in the combined cohorts.

    Conclusions:

    This cord blood epigenetic score can help identify babies exposed to maternal smoking and assess its long-term impact on growth. Notably, these results indicate a consistent association between the DNAm signature of maternal smoking and a small body size and low birth weight in newborns, in both White European mothers who exhibited some amount of smoking and in South Asian mothers who themselves were not active smokers.

    Funding:

    This study was funded by the Canadian Institutes of Health Research Metabolomics Team Grant: MWG-146332.

    1. Epidemiology and Global Health
    2. Microbiology and Infectious Disease
    Patrick E Brown, Sze Hang Fu ... Ab-C Study Collaborators
    Research Article Updated

    Background:

    Few national-level studies have evaluated the impact of ‘hybrid’ immunity (vaccination coupled with recovery from infection) from the Omicron variants of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).

    Methods:

    From May 2020 to December 2022, we conducted serial assessments (each of ~4000–9000 adults) examining SARS-CoV-2 antibodies within a mostly representative Canadian cohort drawn from a national online polling platform. Adults, most of whom were vaccinated, reported viral test-confirmed infections and mailed self-collected dried blood spots (DBSs) to a central lab. Samples underwent highly sensitive and specific antibody assays to spike and nucleocapsid protein antigens, the latter triggered only by infection. We estimated cumulative SARS-CoV-2 incidence prior to the Omicron period and during the BA.1/1.1 and BA.2/5 waves. We assessed changes in antibody levels and in age-specific active immunity levels.

    Results:

    Spike levels were higher in infected than in uninfected adults, regardless of vaccination doses. Among adults vaccinated at least thrice and infected more than 6 months earlier, spike levels fell notably and continuously for the 9-month post-vaccination. In contrast, among adults infected within 6 months, spike levels declined gradually. Declines were similar by sex, age group, and ethnicity. Recent vaccination attenuated declines in spike levels from older infections. In a convenience sample, spike antibody and cellular responses were correlated. Near the end of 2022, about 35% of adults above age 60 had their last vaccine dose more than 6 months ago, and about 25% remained uninfected. The cumulative incidence of SARS-CoV-2 infection rose from 13% (95% confidence interval 11–14%) before omicron to 78% (76–80%) by December 2022, equating to 25 million infected adults cumulatively. However, the coronavirus disease 2019 (COVID-19) weekly death rate during the BA.2/5 waves was less than half of that during the BA.1/1.1 wave, implying a protective role for hybrid immunity.

    Conclusions:

    Strategies to maintain population-level hybrid immunity require up-to-date vaccination coverage, including among those recovering from infection. Population-based, self-collected DBSs are a practicable biological surveillance platform.

    Funding:

    Funding was provided by the COVID-19 Immunity Task Force, Canadian Institutes of Health Research, Pfizer Global Medical Grants, and St. Michael’s Hospital Foundation. PJ and ACG are funded by the Canada Research Chairs Program.