Epidemiological characteristics and prevalence rates of research reproducibility across disciplines: a scoping review of articles published in 2018-2019
Abstract
Introduction: Reproducibility is a central tenant of research. We aimed to synthesize the literature on reproducibility and describe its epidemiological characteristics, including how reproducibility is defined and assessed. We also aimed to determine and compare estimates for reproducibility across different fields.
Methods: We conducted a scoping review to identify English language replication studies published between 2018-2019 in economics, education, psychology, health sciences and biomedicine. We searched Medline, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health Literature – CINAHL, Education Source via EBSCOHost, ERIC, EconPapers, International Bibliography of the Social Sciences (IBSS), and EconLit. Documents retrieved were screened in duplicate against our inclusion criteria. We extracted year of publication, number of authors, country of affiliation of the corresponding author, and whether the study was funded. For the individual replication studies, we recorded whether a registered protocol for the replication study was used, whether there was contact between the reproducing team and the original authors, what study design was used, and what the primary outcome was. Finally, we recorded how reproducibilty was defined by the authors, and whether the assessed study(ies) successfully reproduced based on this definition. Extraction was done by a single reviewer and quality controlled by a second reviewer.
Results: Our search identified 11,224 unique documents, of which 47 were included in this review. Most studies were related to either psychology (48.6%) or health sciences (23.7%). Among these 47 documents, 36 described a single reproducibility study while the remaining 11 reported at least two reproducibility studies in the same paper. Less than the half of the studies referred to a registered protocol. There was variability in the definitions of reproduciblity success. In total, across the 47 documents 177 studies were reported. Based on the definition used by the author of each study, 95 of 177 (53.7%) studies reproduced.
Conclusion: This study gives an overview of research across five disciplines that explicitly set out to reproduce previous research. Such reproducibility studies are extremely scarce, the definition of a successfully reproduced study is ambiguous, and the reproducibility rate is overall modest.
Funding: No external funding was received for this work.
Data availability
Data and materials are available on the Open Science Framework (https://osf.io/wn7gm/).
-
Scoping review data on reproducibility of research in a 2018-2019 multi-discipline sampleOpen Science FrameworkDOI 10.17605/OSF.IO/WN7GM.
Article and author information
Author details
Funding
The authors declare that there was no funding for this work.
Copyright
© 2023, Cobey et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 1,066
- views
-
- 99
- downloads
-
- 5
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Medicine
The zinc transporter Slc30a1 plays an essential role in maintaining cellular zinc homeostasis. Despite this, its functional role in macrophages remains largely unknown. Here, we examine the function of Slc30a1 in host defense using mice models infected with an attenuated stain of Salmonella enterica Typhimurium and primary macrophages infected with the attenuated Salmonella. Bulk transcriptome sequencing in primary macrophages identifies Slc30a1 as a candidate in response to Salmonella infection. Whole-mount immunofluorescence and confocal microscopy imaging of primary macrophage and spleen from Salmonella-infected Slc30a1flag-EGFP mice demonstrate Slc30a1 expression is increased in infected macrophages with localization at the plasma membrane and in the cytosol. Lyz2-Cre-driven Slc30a1 conditional knockout mice (Slc30a1fl/fl;Lyz2-Cre) exhibit increased susceptibility to Salmonella infection compared to control littermates. We demonstrate that Slc30a1-deficient macrophages are defective in intracellular killing, which correlated with reduced activation of nuclear factor kappa B and reduction in nitric oxide (NO) production. Notably, the model exhibits intracellular zinc accumulation, demonstrating that Slc30a1 is required for zinc export. We thus conclude that zinc export enables the efficient NO-mediated antibacterial activity of macrophages to control invading Salmonella.
-
- Chromosomes and Gene Expression
- Medicine
LncRNAs are involved in modulating the individual risk and the severity of progression in metabolic dysfunction-associated fatty liver disease (MASLD), but their precise roles remain largely unknown. This study aimed to investigate the role of lncRNA Snhg3 in the development and progression of MASLD, along with the underlying mechanisms. The result showed that Snhg3 was significantly downregulated in the liver of high-fat diet-induced obesity (DIO) mice. Notably, palmitic acid promoted the expression of Snhg3 and overexpression of Snhg3 increased lipid accumulation in primary hepatocytes. Furthermore, hepatocyte-specific Snhg3 deficiency decreased body and liver weight, alleviated hepatic steatosis and promoted hepatic fatty acid metabolism in DIO mice, whereas overexpression induced the opposite effect. Mechanistically, Snhg3 promoted the expression, stability and nuclear localization of SND1 protein via interacting with SND1, thereby inducing K63-linked ubiquitination modification of SND1. Moreover, Snhg3 decreased the H3K27me3 level and induced SND1-mediated chromatin loose remodeling, thus reducing H3K27me3 enrichment at the Pparg promoter and enhancing PPARγ expression. The administration of PPARγ antagonist T0070907 improved Snhg3-aggravated hepatic steatosis. Our study revealed a new signaling pathway, Snhg3/SND1/H3K27me3/PPARγ, responsible for mice MASLD and indicates that lncRNA-mediated epigenetic modification has a crucial role in the pathology of MASLD.