A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies

Abstract
Data availability
Article and author information
Metrics

Abstract

Background: Multicentric approaches are widely used in clinical trials to assess generalizability of findings, however they are novel in laboratory-based experimentation. It is unclear how multilaboratory studies may differ in conduct and results from single lab studies. Here we synthesized characteristics of these studies and quantitatively compared their outcomes to those generated by single laboratory studies.

Methods: MEDLINE and Embase were systematically searched. Screening and data extractions were completed in duplicate by independent reviewers. Multilaboratory studies investigating interventions using in vivo animal models were included. Study characteristics were extracted. Systematic searches were then performed to identify single center studies matched by intervention and disease. Difference in standardized mean differences (DSMD) was then calculated across studies to assess differences in effect estimates based on study design (>0 indicates larger effects in single center studies).

Results: Sixteen multilaboratory studies met inclusion criteria and were matched to 100 single center studies. The multicenter study design was applied across a diverse range of diseases, including traumatic brain injury, myocardial infarction, and diabetes. The median number of centers was 4 (range 2-6) and the median sample size was 111 (range 23-384) with rodents most frequently used. Multicenter studies adhered to practices that reduce risk of bias significantly more often than single center studies. Multicenter studies also demonstrated significantly smaller effect sizes than single center studies (DSMD 0.72 [95% confidence interval 0.43-1]).

Conclusion: Multilaboratory studies demonstrate trends that have been well recognized in clinical research (i.e. smaller treatment effects with multicentric evaluation and greater rigour in study design). This approach may provide a method to robustly assess interventions and generalizability of findings between laboratories.

Funding: uOttawa Junior Clinical Research Chair; The Ottawa Hospital Anesthesia Alternate Funds Association; Canadian Anesthesia Research Foundation; Government of Ontario Queen Elizabeth II Graduate Scholarship in Science and Technology.

Clinical trial registration: PROSPERO CRD4201809398.

Data availability

The protocol for the effect size comparison was developed a priori and posted on Open Science Framework (https://osf.io/awvs9/).Supplementary documents contains the search strategies, risk of bias assessments, reporting checklists, quality scores, effect sizes, effect size ratios, and standardized mean differences to generate the figures and tables.

Article and author information

Author details

Victoria T Hunniford

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Agnes Grudniewicz

Telfer School of Management, University of Ottawa, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Dean A Fergusson

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Joshua Montroy

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Emma Grigor

Faculty of Medicine, University of Ottawa, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Casey Lansdell

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

Competing interests
The authors declare that no competing interests exist.
Manoj M Lalu

Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Canada

For correspondence
mlalu@toh.ca

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0002-0322-382X

Funding

QEII Scholarship (Graduate Student Scholarship)

Victoria T Hunniford

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.