Science Forum: How failure to falsify in high-volume science contributes to the replication crisis

  1. Sarah M Rajtmajer
  2. Timothy M Errington
  3. Frank G Hillary  Is a corresponding author
  1. Pennsylvania State University, United States
  2. Center for Open Science, United States

Abstract

The number of scientific papers published every year continues to increase, but scientific knowledge is not progressing at the same rate. Here we argue that a greater emphasis on falsification - the direct testing of strong hypotheses - would lead to faster progress by allowing well-specified hypotheses to be eliminated. We describe an example from neuroscience where there has been little work to directly test two prominent but incompatible hypotheses related to traumatic brain injury. Based on this example, we discuss how building strong hypotheses and then setting out to falsify them can bring greater precision to the clinical neurosciences, and argue that this approach could be beneficial to all areas of science.

Data availability

There are no data associated with this article.

Article and author information

Author details

  1. Sarah M Rajtmajer

    College of Information Sciences and Technology, Pennsylvania State University, University Park, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1464-0848
  2. Timothy M Errington

    Center for Open Science, Charlottesville, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-4959-5143
  3. Frank G Hillary

    Department of Psychology, Pennsylvania State University, University Park, United States
    For correspondence
    fhillary@psu.edu
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1427-0218

Funding

The authors declare that there was no funding for this work.

Reviewing Editor

  1. Peter Rodgers, eLife, United Kingdom

Publication history

  1. Received: March 22, 2022
  2. Accepted: July 28, 2022
  3. Accepted Manuscript published: August 8, 2022 (version 1)
  4. Version of Record published: August 23, 2022 (version 2)

Copyright

© 2022, Rajtmajer et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,844
    Page views
  • 471
    Downloads
  • 3
    Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Crossref, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Sarah M Rajtmajer
  2. Timothy M Errington
  3. Frank G Hillary
(2022)
Science Forum: How failure to falsify in high-volume science contributes to the replication crisis
eLife 11:e78830.
https://doi.org/10.7554/eLife.78830
  1. Further reading

Further reading

    1. Neuroscience
    Silvia Maggi, Rebecca M Hock ... Mark D Humphries
    Research Article

    Investigating how, when, and what subjects learn during decision-making tasks requires tracking their choice strategies on a trial-by-trial basis. Here we present a simple but effective probabilistic approach to tracking choice strategies at trial resolution using Bayesian evidence accumulation. We show this approach identifies both successful learning and the exploratory strategies used in decision tasks performed by humans, non-human primates, rats, and synthetic agents. Both when subjects learn and when rules change the exploratory strategies of win-stay and lose-shift, often considered complementary, are consistently used independently. Indeed, we find the use of lose-shift is strong evidence that subjects have latently learnt the salient features of a new rewarded rule. Our approach can be extended to any discrete choice strategy, and its low computational cost is ideally suited for real-time analysis and closed-loop control.

    1. Computational and Systems Biology
    2. Neuroscience
    Tony Zhang, Matthew Rosenberg ... Markus Meister
    Research Article

    An animal entering a new environment typically faces three challenges: explore the space for resources, memorize their locations, and navigate towards those targets as needed. Here we propose a neural algorithm that can solve all these problems and operates reliably in diverse and complex environments. At its core, the mechanism makes use of a behavioral module common to all motile animals, namely the ability to follow an odor to its source. We show how the brain can learn to generate internal “virtual odors” that guide the animal to any location of interest. This endotaxis algorithm can be implemented with a simple 3-layer neural circuit using only biologically realistic structures and learning rules. Several neural components of this scheme are found in brains from insects to humans. Nature may have evolved a general mechanism for search and navigation on the ancient backbone of chemotaxis.