1. Cancer Biology

Reproducibility in Cancer Biology: Challenges for assessing replicability in preclinical cancer biology

  1. Timothy M Errington  Is a corresponding author
  2. Alexandria Denis
  3. Nicole Perfito
  4. Elizabeth Iorns
  5. Brian A Nosek
  1. Center for Open Science, United States
  2. Science Exchange, United States
  3. University of Virginia, United States
https://doi.org/10.7554/eLife.67995
  • Download
  • Cite
  • CommentOpen annotations (there are currently 0 annotations on this page).
Share this article
Cite this article
  1. Timothy M Errington
  2. Alexandria Denis
  3. Nicole Perfito
  4. Elizabeth Iorns
  5. Brian A Nosek
(2021)
Reproducibility in Cancer Biology: Challenges for assessing replicability in preclinical cancer biology
eLife 10:e67995.
https://doi.org/10.7554/eLife.67995
  2. Download BibTeX
  3. Download .RIS
4 figures, 1 table and 1 additional file

Figures

Figure 1 with 1 supplement
Download asset Open asset
Barriers to conducting replications – by experiment.

During the design phase of the project the 193 experiments selected for replication were coded according to six criteria: availability and sharing of data; reporting of statistical analysis (i.e., …

Figure 1—figure supplement 1
Download asset Open asset
Barriers to conducting replications – by paper.

This figure is similar to Figure 1, except that the results for the experiments have been combined to give results for the papers that contained the experiments. Experiments from 53 papers were …

Figure 2 with 2 supplements
Download asset Open asset
Relationship between extent of clarification needed and helpfulness of authors.

Fluctuation plots showing the coded ratings for extent of clarifications needed from original authors and the degree to which authors were helpful in providing feedback and materials for designing …

Figure 2—figure supplement 1
Download asset Open asset
Techniques used in the original experiments.

A total of 820 experimental techniques were identified in the 193 experiments selected for replication at the start of the project. These techniques were coded into 48 sub-categories, which were …

Figure 2—figure supplement 2
Download asset Open asset
Relationship between extent of clarification needed or helpfulness with category of techniques.

Fluctuation plots showing the coded ratings for the category of techniques used in the original experiments and the extent of clarification needed from the original authors by paper (A) and by …

Figure 3
Download asset Open asset
Relationship between extent of modifications needed and implementation of modifications.

Fluctuation plots showing the coded ratings for extent of modifications needed in order to conduct the replication experiments, and the extent to which the replication authors were able to implement …

Figure 4
Download asset Open asset
The different phases of the replication process.

Graph showing the number of papers entering each of the six phases of the replication process, and the mean duration of each phase in weeks. 53 papers entered the design phase, which started with …

Tables

Table 1
The 53 papers selected for replication in the RP:CB.
Original paperExperiments selectedRegistered reportExperiments registeredReplication study*Experiments completedData, digital materials, and code
Poliseno et al., 201011Khan et al., 20156Kerwin et al., 20205https://osf.io/yyqas/
Sharma et al., 20108Haven et al., 20168N/A0https://osf.io/xbign/
Gupta et al., 20102N/A0N/A0https://osf.io/4bokd/
Figueroa et al., 20106N/A0N/A0https://osf.io/xdojz/
Ricci-Vitiani et al., 20103Chroscinski et al., 2015b2Errington et al., 2021a1https://osf.io/mpyvx/
Kan et al., 20103Sharma et al., 2016a3Errington et al., 2021a1https://osf.io/jpeqg/
Heidorn et al., 20108Bhargava et al., 2016a5Errington et al., 2021a1https://osf.io/b1aw6/
Hatzivassiliou et al., 20104Bhargava et al., 2016b3Pelech et al., 20212https://osf.io/0hezb/
Vermeulen et al., 20104Evans et al., 2015a3Essex et al., 20193https://osf.io/pgjhx/
Carro et al., 20108N/A0N/A0https://osf.io/mfxpj/
Nazarian et al., 20105N/A0N/A0https://osf.io/679uw/
Johannessen et al., 20105Sharma et al., 2016b5Errington et al., 2021a2https://osf.io/lmhjg/
Poulikakos et al., 20105N/A0N/A0https://osf.io/acpq7/
Sugahara et al., 20104Kandela et al., 2015a3Mantis et al., 20173https://osf.io/xu1g2/
Ward et al., 20103Fiehn et al., 20163Showalter et al., 20173https://osf.io/8l4ea/
Ko et al., 20103N/A0N/A0https://osf.io/udw78/
Zuber et al., 20113N/A0N/A0https://osf.io/devog/
Delmore et al., 20112Kandela et al., 2015b2Aird et al., 20172https://osf.io/7zqxp/
Goetz et al., 20112Fiering et al., 20152Sheen et al., 20192https://osf.io/7yqmp/
Sirota et al., 20111Kandela et al., 2015c1Kandela et al., 20171https://osf.io/hxrmm/
Raj et al., 20114N/A0N/A0https://osf.io/uvapt/
Possemato et al., 20113N/A0N/A0https://osf.io/u1mfn/
Tay et al., 20115Phelps et al., 20165Wang et al., 20204https://osf.io/oblj1/
Xu et al., 20115Evans et al., 2015b5N/A0https://osf.io/kvshc/
DeNicola et al., 20114N/A0N/A0https://osf.io/i0yka/
Zhu et al., 20113N/A0N/A0https://osf.io/oi7jj/
Liu et al., 20114Li et al., 20153Yan et al., 20193https://osf.io/gb7sr/
Dawson et al., 20113Fung et al., 20153Shan et al., 20173https://osf.io/hcqqy/
Qian et al., 20113N/A0N/A0https://osf.io/ckpsn/
Sumazin et al., 20113N/A0N/A0https://osf.io/wcasz/
Chaffer et al., 20112N/A0N/A0https://osf.io/u6m4z/
Opitz et al., 20115N/A0N/A0https://osf.io/o2xpf/
Kang et al., 20112Raouf et al., 20152N/A0https://osf.io/82nfe/
Chen et al., 20122N/A0N/A0https://osf.io/egoni/
Driessens et al., 20122N/A0N/A0https://osf.io/znixv/
Garnett et al., 20123Vanden Heuvel et al., 20163Vanden Heuvel et al., 20183https://osf.io/nbryi/
Schepers et al., 20123N/A0N/A0https://osf.io/1ovqn/
Willingham et al., 20122Chroscinski et al., 2015a1Horrigan and Reproducibility Project: Cancer Biology, 2017a1https://osf.io/9pbos/
Straussman et al., 20124Blum et al., 20144N/A0https://osf.io/p4lzc/
Arthur et al., 20122Eaton et al., 20152Eaton et al., 20182https://osf.io/y4tvd/
Peinado et al., 20123Lesnik et al., 20162Kim et al., 20182https://osf.io/ewqzf/
Malanchi et al., 20113Incardona et al., 20152N/A0https://osf.io/vseix/
Berger et al., 20121Chroscinski et al., 20141Horrigan et al., 2017b1https://osf.io/jvpnw/
Prahallad et al., 20124N/A0N/A0https://osf.io/ecy85/
Wilson et al., 20123Greenfield et al., 20142N/A0https://osf.io/h0pnz/
Lu et al., 20125Richarson et al., 20163Errington et al., 2021a2https://osf.io/vfsbo/
Lin et al., 20122Blum et al., 20152Lewis et al., 20182https://osf.io/mokeb/
Lee et al., 20123N/A0N/A0https://osf.io/i25y8/
Castellarin et al., 20121Repass et al., 20161Repass and Reproducibility Project: Cancer Biology, 20181https://osf.io/v4se2/
Crasta et al., 20123N/A0N/A0https://osf.io/47xy6/
Png et al., 20115N/A0N/A0https://osf.io/tkzme/
Metallo et al., 20115N/A0N/A0https://osf.io/isdbh/
Morin et al., 20101N/A0N/A0https://osf.io/6kuy8/

  1. 193 experiments in 53 papers were selected for replication. The papers are listed in column 1, and the number of experiments selected from each paper is listed in column 2. Registered Reports for 87 experiments from 29 papers were published in eLife. The Registered Reports are listed in column 3, and the number of experiments included in each Registered Report is listed in column 4. 50 experiments from 23 Registered Reports were completed. 17 Replication Studies reporting the results of 41 experiments were published in eLife; the results of another nine experiments from the six remaining Registered Reports were published in an aggregate paper (Errington et al., 2021a). The Replication Studies are listed in column 5, and the number of experiments included in each study is listed in column 6. Column seven contains a link to data, digital materials, and code.

Additional files

Transparent reporting form
https://cdn.elifesciences.org/articles/67995/elife-67995-transrepform1-v1.docx

Download links