The Ellipse of Insignificance, a refined fragility indexfor ascertaining robustness of results indichotomous outcome trials

  1. David Robert Grimes  Is a corresponding author
  1. Dublin City University, Ireland

Abstract

There is increasing awareness throughout biomedical science that many results do not withstand the trials of repeat investigation. The growing abundance of medical literature has only increased the urgent need for tools to gauge the robustness and trustworthiness of published science. Dichotomous outcome designs are vital in randomized clinical trials, cohort studies, and observational data for ascertaining differences between experimental and control arms. It has however been shown with tools like the fragility index (FI) that many ostensibly impactful results fail to materialise when even small numbers of patients or subjects in either the control or experimental arms are recoded from event to non-event. Critics of this metric counter that there is no objective means to determine a meaningful FI. As currently used, FI is not multi-dimensional and is computationally expensive. In this work a conceptually similar geometrical approach is introduced, the ellipse of insignificance (EOI). This method yields precise deterministic values for the degree of manipulation or miscoding that can be tolerated simultaneously in both control and experimental arms, allowing for the derivation of objective measures of experimental robustness. More than this, the tool is intimately connected with sensitivity and specificity of the event / non-event tests, and is readily combined with knowledge of test parameters to reject unsound results. The method is outlined here, with illustrative clinical examples.

Data availability

The paper is a modelling study and methodology and contains no data, and code provided in the supplementary material allows reproduction of all methods.

Article and author information

Author details

  1. David Robert Grimes

    Dublin City University, Dublin, Ireland
    For correspondence
    davidrobert.grimes@dcu.ie
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3140-3278

Funding

Wellcome Trust (214461/A/18/Z)

  • David Robert Grimes

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Philip Boonstra, University of Michigan, United States

Publication history

  1. Preprint posted: March 28, 2022 (view preprint)
  2. Received: April 19, 2022
  3. Accepted: September 13, 2022
  4. Accepted Manuscript published: September 20, 2022 (version 1)
  5. Version of Record published: October 21, 2022 (version 2)

Copyright

© 2022, Grimes

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 401
    Page views
  • 70
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. David Robert Grimes
(2022)
The Ellipse of Insignificance, a refined fragility indexfor ascertaining robustness of results indichotomous outcome trials
eLife 11:e79573.
https://doi.org/10.7554/eLife.79573

Further reading

    1. Epidemiology and Global Health
    Mette Hartmann Nonboe, George Napolitano ... Elsebeth Lynge
    Research Article

    Background:

    Denmark was one of the few countries where it was politically decided to continue cancer screening during the COVID-19 pandemic. We assessed the actual population uptake of mammography and cervical screening during this period.

    Methods:

    The first COVID-19 lockdown in Denmark was announced on 11 March 2020. To investigate possible changes in cancer screening activity due to the COVID-19 pandemic, we analysed data from the beginning of 2017 until the end of 2021. A time series analysis was carried out to discover possible trends and outliers in the screening activities in the period 2017–2021. Data on mammography screening and cervical screening were retrieved from governmental pandemic-specific monitoring of health care activities.

    Results:

    A brief drop was seen in screening activity right after the first COVID-19 lockdown, but the activity quickly returned to its previous level. A short-term deficit of 43% [CI –49 to –37] was found for mammography screening. A short-term deficit of 62% [CI –65 to –58] was found for cervical screening. Furthermore, a slight, statistically significant downward trend in cervical screening from 2018 to 2021 was probably unrelated to the pandemic. Other changes, for example, a marked drop in mammography screening towards the end of 2021, also seem unrelated to the pandemic.

    Conclusions:

    Denmark continued cancer screening during the pandemic, but following the first lockdown a temporary drop was seen in breast and cervical screening activity.

    Funding:

    Region Zealand (R22-A597).

    1. Epidemiology and Global Health
    Mark M Dekker, Luc E Coffeng ... Sake J de Vlas
    Research Article Updated

    To curb the initial spread of SARS-CoV-2, many countries relied on nation-wide implementation of non-pharmaceutical intervention measures, resulting in substantial socio-economic impacts. Potentially, subnational implementations might have had less of a societal impact, but comparable epidemiological impact. Here, using the first COVID-19 wave in the Netherlands as a case in point, we address this issue by developing a high-resolution analysis framework that uses a demographically stratified population and a spatially explicit, dynamic, individual contact-pattern based epidemiology, calibrated to hospital admissions data and mobility trends extracted from mobile phone signals and Google. We demonstrate how a subnational approach could achieve similar level of epidemiological control in terms of hospital admissions, while some parts of the country could stay open for a longer period. Our framework is exportable to other countries and settings, and may be used to develop policies on subnational approach as a better strategic choice for controlling future epidemics.