Figures and data in The ellipse of insignificance, a refined fragility index for ascertaining robustness of results in dichotomous outcome trials | eLife

Figures
Tables
Additional files

4 figures, 4 tables and 2 additional files

Figures

Figure 1

Download asset Open asset

Ellipse of insignificance example.

(a) An example ellipse of insignificance for the $a = 50$ , $b = 50$ , $c = 10$ , $d = 90$ at a significance level of $α = 0.05$ . All points bounded by the ellipse depict $(x, y)$ combinations which would not lead to the null being rejected. (b) Relevant vectors for ascertaining misconding thresholds. In this example, the Fewest Experimental/Control Knowingly Uncoded Participants (FECKUP) point is $(x_{e}, y_{e}) = (12.8, 14.4)$ , $f_{m i n} = 19.3$ , and $(x_{i}, y_{i}) = (30.1, 26.3)$ . See text for details.

Figure 2

Download asset Open asset

Application of ellipse of insignificance analysis to existent data.

(a) Ellipses of insignificance analysis for a published study (n = 913) for illustrative example 1 of published data. The shaded region denotes the ellipse of insignificance, the red line shows the Fewest Experimental/Control Knowingly Uncoded Participants (FECKUP) vector (the minimum vector from the origin to the ellipse).

Figure 3

Download asset Open asset

Illustrative example 2.

(a) Ellipses of insignificance analysis for two studies with same $χ^{2}$ statistic. (b) Fewest Experimental/Control Knowingly Uncoded Participants (FECKUP) vectors for both studies. Experiment 1 is given by orange ellipse and red dotted line, Experiment 2 by the blue ellipse and dotted line.

Figure 4

Download asset Open asset

Illustrative example 3.

An ellipse of insignificance (EOI) analysis on the data supplied in the City A/City B screening comparison yields a Fewest Experimental/Control Knowingly Uncoded Participants (FECKUP) vector (in red) of 46.2 subjects, corresponding to a minimum tolerance of 66.5 total subjects after resolving the vector.

As $x_{i} = 73.7$ (shown in green) with $y_{i} = 62.7$ (shown in blue), but as the sensitivity and specificity of the tests used in City A are known, it can be shown that $x_{m} \approx 93$ , exceeding the limits of x_i, placing the point within the ellipse and rendering any seeming significance void. Note that only a part of the EOI (denoted by the blue solid shape) is shown for clarity.

Tables

Table 1

Reported groups and related variables.

	Endpoint positive	Endpoint negative
Experimental group	$a - x$	$b + x$
Control group	$c + y$	$d - y$

Table 2

Ellipse of insignificance (EOI) derived metrics for published data.

EOI statistic ( $α = 0.05$ )	Derived value
Experimental group tolerance x_i	6.9 subjects
Control group tolerance y_i	1.9 subjects
FECKUP vector length	1.9 subjects
Tolerance threshold for error (experimental group) $ϵ_{E}$	0.99%
Tolerance threshold for error (control group) $ϵ_{C}$	0.89%
Absolute tolerance threshold for error (all subjects) $ϵ_{A}$	0.22%

Table 3

Experimental metrics for similar test statistics.

Significance level	Data	$ϵ_{E}$	$ϵ_{C}$	$ϵ_{A}$
$α = 0.05$	Experiment 1	17.7%	18.2%	8.9%
	Experiment 2	11%	9.5%	5.0%
$α = 0.01$	Experiment 1	16.3%	17.0%	8.3%
	Experiment 2	10.4%	8.6%	4.6%
$α = 0.001$	Experiment 1	14.8%	15.5%	7.5%
	Experiment 2	9.8%	7.6%	4.1%
$α = 0.0001$	Experiment 1	13.5%	14.3%	6.9%
	Experiment 2	9.2%	6.8%	3.8%

Table 4

Results of different analysis.

	CIN2 + positive	No CIN2 + detected	Methodology
City A (measured)	113	887	LBC only
City B (measured)	24	976	HPV screening/LBC reflex
True values (both cities)	20	980	N/A

Additional files

MDAR checklist: https://cdn.elifesciences.org/articles/79573/elife-79573-mdarchecklist1-v2.pdf
Download elife-79573-mdarchecklist1-v2.pdf
Source code 1 Sample MATLAB/OCTAVE code for rapid implementation of EOI analysis method outlined.: https://cdn.elifesciences.org/articles/79573/elife-79573-code1-v2.zip
Download elife-79573-code1-v2.zip

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Article PDF

Open citations (links to open the citations from this article in various online reference manager services)

Mendeley

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

David Robert Grimes

(2022)

The ellipse of insignificance, a refined fragility index for ascertaining robustness of results in dichotomous outcome trials

eLife 11:e79573.

https://doi.org/10.7554/eLife.79573

Sign up for email alerts

Privacy notice