Two new studies from the Reproducibility Project: Cancer Biology have been published in eLife. The aim of the project is to assess reproducibility in cancer biology and identify what influences its success or failure in science more generally.
An eLife editorial published alongside the first Replication Studies in January reads:
"We will publish more Replication Studies over the months ahead and, at the conclusion of the project, a meta-analysis of all the studies. While we wait for this, it is important not to overinterpret the results. Already it is clear that nuanced interpretations are necessary, not black and white conclusions about which studies reproduced and which did not. It is also clear that this approach to testing reproducibility remains an experiment, with advantages and disadvantages, including the fact that it sometimes yields results that cannot be interpreted."
It also emphasises:
"It is also important to note that even if all the original studies were reproducible, not all of them would be found to be reproducible, just based on chance. The experiments in the Reproducibility Project are typically powered to have an 80% probability of reproducing something that is true: this means that if we attempt to repeat three experiments from a paper, there is only a ~50% chance that all three experiments will yield significant p values, even if the original study was reproducible. Therefore, we cannot place the bar so high that the replications need to hit a significant p value in every experiment. If a replication reproduces some of the key experiments in the original study, and sees effects that are similar to those seen in the original in other experiments, we need to conclude that it has substantially reproduced the original study."
The assessment of eLife editors is that the two new studies have reproduced important parts of the original papers. To date, including the papers published today, four of the Replication Studies have reproduced important parts of the original papers, one study has not, and another two studies could not be interpreted.
The Replication Studies in the cancer biology project are presented in two distinct phases. First, the project leaders developed protocols for each study and shared those with original authors for informal review, before submitting the protocols as “Registered Reports” to eLife for formal peer review.
Registered Reports are an opportunity to get expert feedback from reviewers to maximise the quality of the experimental design and methodology. Next, Replication Studies are conducted as specified in the Registered Reports and are also subject to eLife peer review.
By the end of the project, around 30 Replication Studies will have been reviewed by eLife.
The project is a collaboration between the Center for Open Science and Science Exchange.
All of the Replication Studies and Registered Reports can be freely accessed online at https://elifesciences.org/collections/reproducibility-project-cancer-biology. Contents, including text, figures and data, are free to reuse under a CC BY 4.0 license.
Notes to Editors
**Please note** eLife will not be press releasing any future Replication Studies. Journalists wishing to be kept informed when we publish a new Replication Study should sign up here.
Media contacts
Emily Packer
eLife
e.packer@elifesciences.org
+441223855373
About
eLife is a unique collaboration between the funders and practitioners of research to improve the way important research is selected, presented and shared. eLife publishes outstanding works across the life sciences and biomedicine — from basic biological research to applied, translational and clinical studies. All papers are selected by active scientists in the research community. Decisions and responses are agreed by the reviewers and consolidated by the Reviewing Editor into a single, clear set of instructions for authors, removing the need for laborious cycles of revision and allowing authors to publish their findings quickly. eLife is supported by the Howard Hughes Medical Institute, the Max Planck Society and the Wellcome Trust. Learn more at elifesciences.org.
