A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies

  1. Victoria T Hunniford
  2. Agnes Grudniewicz
  3. Dean A Fergusson
  4. Joshua Montroy
  5. Emma Grigor
  6. Casey Lansdell
  7. Manoj M Lalu  Is a corresponding author
  8. on behalf of the Canadian Critical Care Translational Biology Group
  1. Ottawa Hospital Research Institute, Canada
  2. University of Ottawa, Canada

Abstract

Background: Multicentric approaches are widely used in clinical trials to assess generalizability of findings, however they are novel in laboratory-based experimentation. It is unclear how multilaboratory studies may differ in conduct and results from single lab studies. Here we synthesized characteristics of these studies and quantitatively compared their outcomes to those generated by single laboratory studies.

Methods: MEDLINE and Embase were systematically searched. Screening and data extractions were completed in duplicate by independent reviewers. Multilaboratory studies investigating interventions using in vivo animal models were included. Study characteristics were extracted. Systematic searches were then performed to identify single center studies matched by intervention and disease. Difference in standardized mean differences (DSMD) was then calculated across studies to assess differences in effect estimates based on study design (>0 indicates larger effects in single center studies).

Results: Sixteen multilaboratory studies met inclusion criteria and were matched to 100 single center studies. The multicenter study design was applied across a diverse range of diseases, including traumatic brain injury, myocardial infarction, and diabetes. The median number of centers was 4 (range 2-6) and the median sample size was 111 (range 23-384) with rodents most frequently used. Multicenter studies adhered to practices that reduce risk of bias significantly more often than single center studies. Multicenter studies also demonstrated significantly smaller effect sizes than single center studies (DSMD 0.72 [95% confidence interval 0.43-1]).

Conclusion: Multilaboratory studies demonstrate trends that have been well recognized in clinical research (i.e. smaller treatment effects with multicentric evaluation and greater rigour in study design). This approach may provide a method to robustly assess interventions and generalizability of findings between laboratories.

Funding: uOttawa Junior Clinical Research Chair; The Ottawa Hospital Anesthesia Alternate Funds Association; Canadian Anesthesia Research Foundation; Government of Ontario Queen Elizabeth II Graduate Scholarship in Science and Technology.

Clinical trial registration: PROSPERO CRD4201809398.

Data availability

The protocol for the effect size comparison was developed a priori and posted on Open Science Framework (https://osf.io/awvs9/).Supplementary documents contains the search strategies, risk of bias assessments, reporting checklists, quality scores, effect sizes, effect size ratios, and standardized mean differences to generate the figures and tables.

Article and author information

Author details

  1. Victoria T Hunniford

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  2. Agnes Grudniewicz

    Telfer School of Management, University of Ottawa, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  3. Dean A Fergusson

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  4. Joshua Montroy

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  5. Emma Grigor

    Faculty of Medicine, University of Ottawa, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  6. Casey Lansdell

    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
    Competing interests
    The authors declare that no competing interests exist.
  7. Manoj M Lalu

    Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Canada
    For correspondence
    mlalu@toh.ca
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-0322-382X

Funding

QEII Scholarship (Graduate Student Scholarship)

  • Victoria T Hunniford

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Carlos Isales, Augusta University, United States

Publication history

  1. Received: December 12, 2021
  2. Accepted: March 8, 2023
  3. Accepted Manuscript published: March 9, 2023 (version 1)

Copyright

© 2023, Hunniford et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 70
    Page views
  • 16
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Victoria T Hunniford
  2. Agnes Grudniewicz
  3. Dean A Fergusson
  4. Joshua Montroy
  5. Emma Grigor
  6. Casey Lansdell
  7. Manoj M Lalu
  8. on behalf of the Canadian Critical Care Translational Biology Group
(2023)
A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies
eLife 12:e76300.
https://doi.org/10.7554/eLife.76300

Further reading

    1. Epidemiology and Global Health
    2. Genetics and Genomics
    Peh Joo Ho, Iain BeeHuat Tan ... Jingmei Li
    Research Article

    Background: To evaluate the utility of polygenic risk scores (PRS) in identifying high-risk individuals, different publicly available PRS for breast (n=85), prostate (n=37), colorectal (n=22) and lung cancers (n=11) were examined in a prospective study of 21,694 Chinese adults.

    Methods: We constructed PRS using weights curated in the online PGS Catalog. PRS performance was evaluated by distribution, discrimination, predictive ability, and calibration. Hazard ratios (HR) and corresponding confidence intervals [CI] of the common cancers after 20 years of follow-up were estimated using Cox proportional hazard models for different levels of PRS.

    Results: A total of 495 breast, 308 prostate, 332 female-colorectal, 409 male-colorectal, 181 female-lung and 381 male-lung incident cancers were identified. The area under receiver operating characteristic curve for the best performing site-specific PRS were 0.61 (PGS000873, breast), 0.70 (PGS00662, prostate), 0.65 (PGS000055, female-colorectal), 0.60 (PGS000734, male-colorectal) and 0.56 (PGS000721, female-lung), and 0.58 (PGS000070, male-lung), respectively. Compared to the middle quintile, individuals in the highest cancer-specific PRS quintile were 64% more likely to develop cancers of the breast, prostate, and colorectal. For lung cancer, the lowest cancer-specific PRS quintile was associated with 28-34% decreased risk compared to the middle quintile. In contrast, the hazard ratios observed for quintiles 4 (female-lung: 0.95 [0.61-1.47]; male-lung: 1.14 [0.82-1.57]) and 5 (female-lung: 0.95 [0.61-1.47]) were not significantly different from that for the middle quintile.

    Conclusions: Site-specific PRSs can stratify the risk of developing breast, prostate, and colorectal cancers in this East Asian population. Appropriate correction factors may be required to improve calibration.

    Funding This work is supported by the National Research Foundation Singapore (NRF-NRFF2017-02), PRECISION Health Research, Singapore (PRECISE) and the Agency for Science, Technology and Research (A*STAR). WP Koh was supported by National Medical Research Council, Singapore (NMRC/CSA/0055/2013). CC Khor was supported by National Research Foundation Singapore (NRF-NRFI2018-01). Rajkumar Dorajoo received a grant from the Agency for Science, Technology and Research Career Development Award (A*STAR CDA - 202D8090), and from Ministry of Health Healthy Longevity Catalyst Award (HLCA20Jan-0022). The Singapore Chinese Health Study was supported by grants from the National Medical Research Council, Singapore (NMRC/CIRG/1456/2016) and the U.S. National Institutes of Health [NIH] (R01 CA144034 and UM1 CA182876).

    1. Epidemiology and Global Health
    2. Immunology and Inflammation
    Zaki A Sherif, Christian R Gomez ... RECOVER Mechanistic Pathway Task Force
    Review Article

    COVID-19, with persistent and new onset of symptoms such as fatigue, post-exertional malaise, and cognitive dysfunction that last for months and impact everyday functioning, is referred to as Long COVID under the general category of post-acute sequelae of SARS-CoV-2 infection (PASC). PASC is highly heterogenous and may be associated with multisystem tissue damage/dysfunction including acute encephalitis, cardiopulmonary syndromes, fibrosis, hepatobiliary damages, gastrointestinal dysregulation, myocardial infarction, neuromuscular syndromes, neuropsychiatric disorders, pulmonary damage, renal failure, stroke, and vascular endothelial dysregulation. A better understanding of the pathophysiologic mechanisms underlying PASC is essential to guide prevention and treatment. This review addresses potential mechanisms and hypotheses that connect SARS-CoV-2 infection to long-term health consequences. Comparisons between PASC and other virus-initiated chronic syndromes such as myalgic encephalomyelitis/chronic fatigue syndrome and postural orthostatic tachycardia syndrome will be addressed. Aligning symptoms with other chronic syndromes and identifying potentially regulated common underlining pathways may be necessary for understanding the true nature of PASC. The discussed contributors to PASC symptoms include sequelae from acute SARS-CoV-2 injury to one or more organs, persistent reservoirs of the replicating virus or its remnants in several tissues, re-activation of latent pathogens such as Epstein–Barr and herpes viruses in COVID-19 immune-dysregulated tissue environment, SARS-CoV-2 interactions with host microbiome/virome communities, clotting/coagulation dysregulation, dysfunctional brainstem/vagus nerve signaling, dysautonomia or autonomic dysfunction, ongoing activity of primed immune cells, and autoimmunity due to molecular mimicry between pathogen and host proteins. The individualized nature of PASC symptoms suggests that different therapeutic approaches may be required to best manage specific patients.