Meta-Research: Centralized scientific communities are less likely to generate replicable results
- University of Chicago, United States
- Stanford University, United States
- Santa Fe Institute, United States
-
Download
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Meta-Research: Centralized scientific communities are less likely to generate replicable resultseLife 8:e43094.https://doi.org/10.7554/eLife.43094 - Cite
- CommentOpen annotations (there are currently 0 annotations on this page).
Figures
Figure 1 with 2 supplements
Alignment of drug-gene interaction (DGI) claims reported in the literature with DGI claims from high-throughput experiments.
(A) Our analysis of 51,292 DGI claims (see Supplementary file 1) in the literature revealed 60,159 supporting findings (green) and 4253 opposing findings (pink) in aggregate. These DGI claims …
Figure 1—figure supplement 1
Publications that share authors are more likely to agree about the direction of a drug-gene interaction than publications with distinct authors, computed among pairs of papers reporting claims in the sub-corpus of 2493 claims.
(A) Bootstrap samples of agreement (1 = agreement, 0 = disagreement) among pairs of papers that reported findings about the same drug-gene interaction depending on whether the pairs of papers shared …
Figure 1—figure supplement 2
Estimates of probability density functions for variables of interest in our corpus using a normal kernel function.
We used the normal kernel function to estimate probability density functions.
Figure 2 with 2 supplements
Estimates of claim replication as a function of the probability of support in the literature and generalizability across high-throughput experiments.
(A) Posterior distributions of probability of support in the biomedical literature for a sample of seven DGI claims for which there are at least two findings (supporting and/or opposing). Note that …
Figure 2—figure supplement 1
Replication increases with claim’s probability of support in the literature.
(A) Probability of claim replication estimated via a logistic regression model with replication as the response variable and support in the literature and experimental variability as predictors, for …
Figure 2—figure supplement 2
Description of claim types in the whole corpus of 51,292 claims.
(A) Mean number of findings supporting or opposing the direction of the effect for a scientific claim across our typology. (B) Cumulative distribution of the probability of support in the …
Figure 3 with 2 supplements
Exemplary networks comprising social, methodological, and references dependences and centralization patterns in scientific communities.
(A) Multilayer networks for four of the claims shown in Figure 2A. The nodes in each layer are scientific papers. Pairs of papers are connected by an unweighted edge in the top layer if they agree …
Figure 3—figure supplement 1
Pearson correlation coefficients between network indices.
Social, methodological, and prior knowledge independencies are positively correlated with each other and negatively correlated with the network centralization of scientific communities. n = 2493 …
Figure 3—figure supplement 2
Papers and pairs of papers are differentiated by the number of findings they report (in the sub-corpus of 2493 claims).
(A) Distribution of papers against the number of supporting findings they report. (B) Distribution of pairs of papers against the number of supporting findings they report.
Figure 4 with 5 supplements
Predictors of replication success.
(A) Odds ratios derived from logistic regression models with claim replication as the response variable and seven predictors modeled independently (disconnected colored dots; n = 2493) and …
Figure 4—figure supplement 1
Predictors of replication success.
(A) Relative risk (RR) of claim replication derived from the logistic regression models in Figure 4A. (B–H) Predicted probabilities (PP) of claim replication for the logistic regression models in …
Figure 4—figure supplement 2
Claims reported by centralized communities less likely replicate.
(A–C) Logistic interaction model with claim replication as the response variable regressed on support in the literature and author centralization as interacting predictors, controlling for …
Figure 4—figure supplement 3
Claims reported by multiple socially independent teams more likely replicate.
(A–C) Logistic interaction model with claim replication as the response variable regressed on support from the literature and social independence as interacting predictors, controlling for …
Figure 4—figure supplement 4
Estimates of probability density functions for our variables on the sub-corpus of claims with determined direction of the drug-gene effect in CTD and LINCS L1000.
We used the normal kernel function to estimate probability density functions.
Figure 4—figure supplement 5
Support in the literature and decentralization of scientific communities remain strong and significant predictors of claim replication success after we account for multicollinearity.
Odds ratios derived from logistic model with replication as the response variable, including only predictors with relatively low variance inflation factor (VIF <4). Predictors are rescaled for …
Additional files
-
Supplementary file 1
Data about the corpus of 51,292 drug-gene interaction claims.
- https://doi.org/10.7554/eLife.43094.017
-
Supplementary file 2
Data about the sub-corpus of 2493 drug-gene interaction claims for which there are two or more published findings.
- https://doi.org/10.7554/eLife.43094.018
-
Supplementary file 3
Logistic regression models with claim replication R [Replicated = 1, Non-replicated = 0] as response variable and predictors modelled independently.
- https://doi.org/10.7554/eLife.43094.019
-
Supplementary file 4
Logistic regression models with claim replication R [Replicated = 1, Non-replicated = 0] as response variable and predictors modelled simultaneously.
- https://doi.org/10.7554/eLife.43094.020
