In light of yesterday's media coverage on the first results of the Reproducibility Project: Cancer Biology, we released early copies of the full studies, along with an Editorial and other accompanying articles. These materials have today been published in eLife and are now available to read here.
The aim of the project, which is a collaboration between the Center for Open Science and Science Exchange, is to assess reproducibility in cancer biology, and to identify what influences its success or failure in science more generally.
“The first five papers are part of a substantial effort to study reproducibility in cancer biology in a transparent way,” says eLife Editor-in-Chief Randy Schekman. “We aim to bolster this effort by applying eLife’s rigorous and consultative editorial process to the evaluation and peer review of the resulting papers.”
“This is an experiment. Every approach to assessing reproducibility has strengths and weaknesses. With over 20 further papers to publish, we do not yet know whether the approach taken in this project is the most effective. However, these initial papers will encourage discussion of these issues and the final analyses will highlight factors that influence reproducibility and the best ways to evaluate it,” he says.
Tim Errington from the Center for Open Science says: “These first five papers show that achieving reproducibility is hard and there is room for improvement. We can already see how variations in the availability of critical information and processes can impact how challenging it is to replicate projects and results.”
The assessment of eLife editors is that of the first five Replication Studies, two broadly supported key conclusions from the original studies, two were inconclusive due to technical problems with certain key experiments, and one failed to reproduce the findings of the original study.
Examples of media coverage featuring the first studies can be found below:
- Cancer reproducibility project releases first results (Nature news article)
- Replication studies offer much more than technical details (Nature editorial)
- Researchers struggle to replicate 5 influential cancer experiments from top labs (The Washington Post)
- Rigorous replication effort succeeds for just two of five cancer papers (Science)
- Project to replicate landmark cancer studies meets mixed results (STAT News)
- What Does It Mean When Cancer Findings Can't Be Reproduced? (NPR)
- How Reliable Are Cancer Studies? (The Atlantic)
- An Attempt to Replicate Top Cancer Studies Casts Doubt on Reproducibility Itself (The Chronicle of Higher Education)
- Cancer Biology Reproducibility Project Sees Mixed Results (PBS)
- Early studies raise questions over cancer reproducibility project (Times Higher Education)
- Fighting Cancer’s Crisis of Confidence, One Study at a Time (WIRED)
- Cancer studies get mixed grades on redo tests (Science News)
