Transparency of research practices in cardiovascular literature

Scientists are under pressure to publish impactful research quickly. In this “publish or perish” culture of science, scientists who are the first to publish a discovery in a well-known journal are often rewarded with credit and more funding. But this publication pressure can have unintended consequences and lead to the publication of incomplete or flawed research. To improve the quality and integrity of research, advocates encourage scientists to share their data and how they came to their conclusions to allow others to verify or replicate their work.

Several fields, including cancer research, have launched efforts to improve study quality and transparency. But so far, there has been little analysis about the reproducibility and replicability of cardiovascular disease research. Assessing how much information cardiovascular researchers provide about their methods, materials, and data could help determine the quality of heart disease research studies and boost confidence in the field’s discoveries.

Heckerman et al. found that fewer than 2 percent of heart disease research studies include enough information to be verified by other scientists. In the analysis, Heckerman et al. analyzed 393 cardiovascular disease research studies to determine if the studies provided enough information for others to replicate their work. The amount of information the authors shared varied according to study type. Studies describing the experiences of individuals or small numbers of patients shared the least information about how they came to their conclusions, while larger clinical trials shared more. In some cases, such as when a study used personally identifiable patient information or a drug company’s proprietary data, there may have been reasons to keep the data confidential.

The findings provide valuable information for both cardiovascular scientists and the public. They show a lot of room to improve trust in cardiovascular disease research by ensuring more studies are verifiable. Scientific journals and research funders could incentivize researchers to share more information about their methods and data to increase trust and transparency in the field.

Previous studies have reported on the lack of detailed methodology, data, and code provided that would be necessary for individual studies or study sets to be considered replicable and reproducible (Filazzola and Cahill, 2021). In addition, scientific publications are often behind paywalls that only users affiliated with a larger institution are readily able to access, meaning that research funded by public money is often inaccessible to the general public without paying a fee. Scientific practices have been evolving over time: data from a study can be complex and require a large amount of storage space, methods can be extremely sophisticated and require specialized equipment, and data analysis can require complex algorithms and code. Journals often do not specify requirements for materials sharing, data sharing, analysis code sharing, and methodological information – all of which have been shown to be important to be able to reproduce or replicate a study (Hamra et al., 2019; Munafò et al., 2017). Note that often there may be legitimate reasons for restricting accessibility, such as privacy concerns, restrictive company policies, and costliness of the process.

Several efforts to improve transparency and accessibility are already underway. Among these are the Reproducible Evidence: Practices to Enhance and Achieve Transparency (REPEAT) initiative, a program dedicated to the improvement of transparency and reproducibility of database research in the healthcare sector (Repeat, 2023). REPEAT contributes to Wang et al., 2022, exploring reasons for irreproducibility, responsiveness of corresponding authors, and many more anecdotes in a study attempting to reproduce published results utilizing the same databases as the original investigators. Another example is the Reproducibility Project: Cancer Biology, meant to replicate experiments in cancer biology research papers to analyze their level or replicability (Davis et al., 2014). The Reproducibility Project: Cancer Biology demonstrated their work in preclinical cancer biology research replicability by repeating 50 experiments from 23 papers (Errington et al., 2021b) and 193 experiments from another 53 papers (Errington et al., 2021a), discussing challenges and barriers associated with replicability. In addition, several resources are readily available to aid with the transparency assessment, including but not limited to SciScore (Menke et al., 2022), DataSeer (DataSeer, 2023), and Ripeta (Sumner et al., 2021).

Although some journals and funding agencies have implemented policies to support the public dissemination of research, we know from recent studies in several disciplines that accessible and reproducible research practices are still far from the norm (Borghi and Van Gulick, 2018; Walters et al., 2019; Kemper et al., 2020; Sherry et al., 2020; Smith et al., 2021). However, to date, no study has examined the degree of accessible and reproducible practices in cardiovascular science publications. Within the domain of cardiovascular science research, this type of analysis is extremely important for holding the latest cutting-edge discoveries in cardiovascular science to a high standard because of the impact on human health. We therefore investigate reproducible reporting practices using an adapted previously published screening process (National Academies of Sciences, Engineering, and Medicine, 2019).

With this study, we hoped to identify the prevalence of accessible and reproducible research practices in cardiovascular research. To achieve this aim, we defined what information is necessary to recreate research from a published work (reproducible). We screened randomly selected articles published in 2019 in Circulation, the European Heart Journal, and the Journal of the American College of Cardiology (JACC). We tested, both, that the simple majority, more than half of screened publications, would lack one or more specific criteria that facilitate reproducibility or replicability, and that the vast majority, over 90% of screened publications, would lack one or more specific criteria that facilitate reproducibility or replicability. In addition, we predicted that some study types (e.g. clinical trials) would satisfy significantly more accessibility criteria than other study types. We also predicted that some categories of accessibility criteria would be satisfied significantly more frequently than other categories in both study types and across separate journals (e.g. materials availability vs analysis script availability). We also specified that a lack of specific requirements by journals regarding what information to provide would lead to variability in which accessibility criteria are satisfied.

We screened a similar proportion of 2019 articles from each of the three top cardiovascular journals: Circulation, European Heart Journal, and the Journal of the American College of Cardiology (226, 193, and 220, respectively, Figure 1C). Over half (247 out of 400) of the randomly selected articles in our initial screening were review articles or other article types that contained no empirical data (Figure 1). We ultimately screened a total of 639 articles, of which 393 were empirical research studies. Approximately half of the papers (56%) had easily identifiable study types while the remaining papers required more substantial reading to determine their study type (Figure 1B). 43% (276 out of 639) of the articles did not have a funding statement (Figure 1D). 93% of articles included a COI statement, and 57% stated they had a COI (Figure 1E). Notably, 7% of articles did not have any COI statement at all. Additionally, over 90% of articles were publicly accessible (605 out of 639, Figure 1F).

Figure 1

Download asset Open asset

Cardiovascular research publications rarely make their resources available

Looking only at articles that included empirical research (393 out of 639 total articles), the majority had had no pre-registration statement (286) or no linked and accessible protocol (383; Figure 2A). Of the papers that had a pre-registration statement, nearly all of them had accessible and openable pre-registrations (106 out of 107; Figure 2A). Among articles that had an openable and accessible pre-registration or protocol, almost all articles included methods (106 out of 106, 9 out of 10), approximately half included hypotheses (44 out of 106, 7 out of 10), and less than half included analysis plans (27 out of 106, 5 out of 10; Figure 2A). We define a publication to have all pre-registration aspects if hypotheses, methods, and analysis are pre-registered. Of the 393 screened publications that have empirical research, 53 had 1 out of 3 of the aspects, 35 had 2 out of 4 of the aspects, and 18 studies had all three aspects.

Figure 2

Download asset Open asset

Only study types considered empirical research in which materials are theoretically available are summarized. Only 14% of articles made their materials available (56 out of 393), 31.8% of articles made their data available (125 out of 393), and 10.9% made their analysis scripts available (43 out of 393; Figure 2B). Across the articles that stated materials, data, and analysis scripts were available, it was most common for them only to be made available upon request from the authors (Figure 2B). Across the articles that stated materials, data, and analysis scripts were available, very few studies made the materials and data readily available for downloading or opening (1, 13) and only one study’s analysis scripts were readily downloadable (1 out of 43; Figure 2B). However, the majority of data that was downloadable or openable was also clearly documented (9 out of 13; Figure 2B).

The majority of cardiovascular publications are not readily replicable or reproducible. We predicted that the majority of publications will not provide sufficient resources to replicate or reproduce their studies. We tested both (1) that the simple majority, more than half of screened publications, will be lacking one or more specific replicability or reproducibility criteria, and (2) that the vast majority, over 90% of screened publications, will be lacking one or more specific reproducibility or replicability criteria. We found that 49.6% (195 out of 393) of empirical research studies were not partially reproducible and 49.4% (194 out of 393) were not partially replicable. 2% (7 out of 393) empirical research papers were fully reproducible, but only 5 were fully replicable (Figure 3). Therefore, the simple majority of empirical research articles were partially reproducible or replicable, but the vast majority of empirical research studies were neither fully reproducible nor fully replicable.

Figure 3

Download asset Open asset

Accessibility varies across journals and study types

With the exception of one paper, the entire dataset exhibited low accessibility with scores lower than 0.6; the majority of articles (144 out of 393) had accessibility scores within the 0.20–0.29 range and 18 articles had scores below 0.1. Only one article exhibited an accessibility score of 0.6 or greater (Figure 4A).

Figure 4

Download asset Open asset

We also predicted that the level of accessibility will be inconsistent across several dimensions. Specifically, we predicted that average accessibility score would vary across study types and potentially also across journals, depending on the publisher’s requirements. For example, if clinical trials must be registered and open access, clinical trial publications will satisfy more of the article coding form criteria than other studies. We tested this hypothesis by calculating an accessibility score for every article: the sum of the satisfied screening criteria divided by the total possible satisfiable criteria. We then calculated the average and standard deviation in accessibility scores across study type (Figure 4C). Overall accessibility scores were low across all study types, and standard deviations were large, indicating high variability across individual publications. We found that clinical trials had the highest average accessibility score (0.39), and clinical case studies or series had the lowest accessibility scores (0.08). Average accessibility scores were significantly different across study types (ANOVA, d.f. 7, F=36.37, p=5.500E−5).

We also predicted a significant difference in the number of accessibility criteria satisfied across journals due to differences in reporting policies. The largest proportion of articles screened were obtained from Circulation with a total number of 226 articles, and 74% of those were empirical research papers (168 out of 226). 63% of JACC articles (138 out of 220) and 45% of Circulation articles (87 out of 193) were empirical research for which we could calculate an accessibility score. After computing the average accessibility score for all three journals, Circulation had the highest average accessibility score (0.34), followed by the Journals of the American College of Cardiology (0.24), then the European Heart Journal (0.19) (Figure 4B). Accessibility scores vary significantly across journals (ANOVA, d.f. 2, F = 80.07, p = 1.230E−2). There was not a significant interaction between study type and journal (p = 0.284).

Categories of accessible resources varies across study types

We also predicted some categories of resources will be accessible significantly more frequently than other categories (e.g. materials availability vs analysis script availability). We found that there was a significant difference in the proportion of papers that shared specific categories of resources (X² = 55.2, p = 6.17e−12). We then further identified whether different study types exhibited differences in the proportions of shared resources, for example, whether there was a difference in the proportion of papers that have accessible data depending on study type. Note that we did not include Clinical Studies with Interventions, Meta-analysis or Systematic Reviews, Non-clinical Secondary Data Analysis, or Laboratory Cell or Molecular Studies in this analysis due to their small sample size. For all four shareable resources (Materials, Methods, Data, and Analysis), we found a significant difference across study types (Table 2).

Table 2

	Yes	No	Study type chi-square p-value with Bonferroni correction
Materials	46	308	3.05E−13
Methods	107	247	4.91E−35 (case studies omitted)
Data	107	247	2.31E−12
Analysis	36	318	1.58E−12

To determine specifically how study types differ in which resources they share, we performed multiple chi-squared tests with Bonferroni correction, comparing each study type with every other study type for a given resource (Figure 5; Table 3). We found that a significantly higher proportion of laboratory animal studies stated that they would share materials than clinical observational studies or clinical trials. We also, as expected, found a significantly higher proportion of clinical trials shared their methods compared to any other study type. With regard to data sharing, we found that a significantly higher proportion of laboratory animal studies shared their data than any other study type. Lastly, a significantly higher proportion of laboratory animal studies shared their analysis code compared to clinical observational studies and clinical trials.

Figure 5

Download asset Open asset

Table 3

	Materials	Methods	Data	Analysis Code
Clinical Case Study or Series vs Clinical Observational Study	1.0	1.0	1.0	1.0
Clinical Case Study or Series vs Clinical Trial	1.0	3.14E−13	0.34	1.0
Clinical Case Study or Series vs Laboratory Animal Study	0.04	1.0	1.86E−5	0.18
Clinical Observational Study vs Clinical Trial	1.0	1.35E−26	0.25	1.0
Clinical Observational Study vs Laboratory Animal Study	1.58E−12	1.0	1.74E−11	2.66E−12
Clinical Trial vs Laboratory Animal Study	1.42E−3	3.96E−19	2.07E−3	1.92E−3

We reviewed 639 papers that were published in 2019 in three of the top cardiovascular research journals to determine how accessible, and therefore reproducible, their research was. Our intention with this study was not to diminish existing studies, but to share current practices with the cardiovascular field and identify ways to improve those practices to enhance reproducibility and transparency in the future. We hope to identify opportunities for journals and scientists to adapt their practices to further reproducible science and encourage the use of higher standards and consistent formats, among cardiovascular scientific literature.

In general, we found that the simple majority, but not the vast majority, of publications are lacking one or more of the resources (materials, methods, data, or analysis scripts) to replicate or reproduce a study. Only 5 out of 393 provided sufficient resources to fully replicate their work, and only 7 out of 393 provided sufficient resources to reproduce their work. Although there were statistically significant differences in accessibility scores across study types and journals, overall, accessibility scores were consistently low.

In an effort to increase replicability and reproducibility of future published cardiovascular literature, there are some initiatives that could be taken. Journals have implemented practices to incentivize researchers to share their materials, methods, data, and analysis scripts, such as open science badges (Kidwell et al., 2016), reporting checklists (Han et al., 2017), and checking articles for reproducibility (Organic Syntheses, 2023) or replicability (Nosek et al., 2015). Our findings suggest increasing these efforts and expanding them to funding agencies would help promote open science practices across the field.

All materials, data, and analysis scripts associated with this study are available on Open Science Framework.

1. 1. Tzng E
   2. Heckerman G
   3. Campos-Melendez A
   4. Mueller A
   5. Ekwueme C

   (2025) Open Science Framework

   Transparency of research practices in cardiovascular literature.

   https://doi.org/10.17605/OSF.IO/FUDKA

1. 1. Borghi JA
   2. Van Gulick AE

   (2018) Data management and sharing in neuroimaging: practices and perceptions of MRI researchers

   PLOS ONE 13:e0200562.

   https://doi.org/10.1371/journal.pone.0200562

   • PubMed
   • Google Scholar
2. 1. Campos-Melendez A
   2. Ekwueme CS
   3. Heckerman GO

   (2021) Assessment of accessibility and reproducible research practices in cardiovascular literature

   OSF 1:QFSTH.

   https://doi.org/10.17605/OSF.IO/QFSTH

   • Google Scholar
3. Website

   1. DataSeer

   (2023) DataSeer

   Accessed January 6, 2023.

   https://dataseer.ai/
4. Website

   1. Davis RJ
   2. Espinosa J
   3. Green MR
   4. Massagué J
   5. Duojia P

   (2014) Reproducibility Project: Cancer Biology

   Accessed January 6, 2023.

   https://elifesciences.org/collections/9b1e83d1/reproducibility-project-cancer-biology
5. 1. Errington TM
   2. Mathur M
   3. Soderberg CK
   4. Denis A
   5. Perfito N
   6. Iorns E
   7. Nosek BA

   (2021a) Investigating the replicability of preclinical cancer biology

   eLife 10:e71601.

   https://doi.org/10.7554/eLife.71601

   • Google Scholar
6. 1. Errington TM
   2. Denis A
   3. Perfito N

   (2021b) Reproducibility in cancer biology: challenges for assessing replicability in preclinical cancer biology

   eLife 10:e67995.

   https://doi.org/10.7554/eLife.67995

   • Google Scholar
7. 1. Filazzola A
   2. Cahill JF

   (2021) Replication in field ecology: Identifying challenges and proposing solutions

   Methods in Ecology and Evolution 12:1780–1792.

   https://doi.org/10.1111/2041-210X.13657

   • Google Scholar
8. 1. Goldacre B
   2. Drysdale H
   3. Dale A
   4. Milosevic I
   5. Slade E
   6. Hartley P
   7. Marston C
   8. Powell-Smith A
   9. Heneghan C
   10. Mahtani KR

   (2019) Compare: a prospective cohort study correcting and monitoring 58 misreported trials in real time

   Trials 20:118.

   https://doi.org/10.1186/s13063-019-3173-2

   • PubMed
   • Google Scholar
9. 1. Hamra GB
   2. Goldstein ND
   3. Harper S

   (2019) Resource sharing to improve research quality

   Journal of the American Heart Association 8:e012292.

   https://doi.org/10.1161/JAHA.119.012292

   • PubMed
   • Google Scholar
10. 1. Han S
    2. Olonisakin TF
    3. Pribis JP
    4. Zupetic J
    5. Yoon JH
    6. Holleran KM
    7. Jeong K
    8. Shaikh N
    9. Rubio DM
    10. Lee JS

    (2017) A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

    PLOS ONE 12:e0183591.

    https://doi.org/10.1371/journal.pone.0183591

    • PubMed
    • Google Scholar
11. 1. Hardwicke TE
    2. Mathur MB
    3. MacDonald K
    4. Nilsonne G
    5. Banks GC
    6. Kidwell MC
    7. Hofelich Mohr A
    8. Clayton E
    9. Yoon EJ
    10. Henry Tessler M
    11. Lenne RL
    12. Altman S
    13. Long B
    14. Frank MC

    (2018) Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition

    Royal Society Open Science 5:180448.

    https://doi.org/10.1098/rsos.180448

    • PubMed
    • Google Scholar
12. 1. Iqbal SA
    2. Wallach JD
    3. Khoury MJ
    4. Schully SD
    5. Ioannidis JPA

    (2016) Reproducible research practices and transparency across the biomedical literature

    PLOS Biology 14:e1002333.

    https://doi.org/10.1371/journal.pbio.1002333

    • PubMed
    • Google Scholar
13. 1. Kemper JM
    2. Rolnik DL
    3. Mol BWJ
    4. Ioannidis JPA

    (2020) Reproducible research practices and transparency in reproductive endocrinology and infertility articles

    Fertility and Sterility 114:1322–1329.

    https://doi.org/10.1016/j.fertnstert.2020.05.020

    • PubMed
    • Google Scholar
14. 1. Kidwell MC
    2. Lazarević LB
    3. Baranski E
    4. Hardwicke TE
    5. Piechowski S
    6. Falkenberg LS
    7. Kennett C
    8. Slowik A
    9. Sonnleitner C
    10. Hess-Holden C
    11. Errington TM
    12. Fiedler S
    13. Nosek BA

    (2016) Badges to acknowledge open practices: a simple, low-cost, effective method for increasing transparency

    PLOS Biology 14:e1002456.

    https://doi.org/10.1371/journal.pbio.1002456

    • PubMed
    • Google Scholar
15. 1. Kim J
    2. Kim H
    3. Bell E
    4. Bath T
    5. Paul P
    6. Pham A
    7. Jiang X
    8. Zheng K
    9. Ohno-Machado L

    (2019) Patient perspectives about decisions to share medical data and biospecimens for research

    JAMA Network Open 2:e199550.

    https://doi.org/10.1001/jamanetworkopen.2019.9550

    • PubMed
    • Google Scholar
16. 1. Menke J
    2. Eckmann P
    3. Ozyurt IB
    4. Roelandse M
    5. Anderson N
    6. Grethe J
    7. Gamst A
    8. Bandrowski A

    (2022) Establishing institutional scores with the rigor and transparency index: large-scale analysis of scientific reporting quality

    Journal of Medical Internet Research 24:e37324.

    https://doi.org/10.2196/37324

    • PubMed
    • Google Scholar
17. 1. Munafò MR
    2. Nosek BA
    3. Bishop DVM
    4. Button KS
    5. Chambers CD
    6. du Sert NP
    7. Simonsohn U
    8. Wagenmakers EJ
    9. Ware JJ
    10. Ioannidis JPA

    (2017) A manifesto for reproducible science

    Nature Human Behaviour 1:0021.

    https://doi.org/10.1038/s41562-016-0021

    • PubMed
    • Google Scholar
18. Book

    1. National Academies of Sciences, Engineering, and Medicine

    (2019) Reproducibility and Replicability in Science

    Washington, DC: The National Academies Press.

    https://doi.org/10.17226/25303

    • Google Scholar
19. 1. Nosek BA
    2. Alter G
    3. Banks GC
    4. Borsboom D
    5. Bowman SD
    6. Breckler SJ
    7. Buck S
    8. Chambers CD
    9. Chin G
    10. Christensen G
    11. Contestabile M
    12. Dafoe A
    13. Eich E
    14. Freese J
    15. Glennerster R
    16. Goroff D
    17. Green DP
    18. Hesse B
    19. Humphreys M
    20. Ishiyama J
    21. Karlan D
    22. Kraut A
    23. Lupia A
    24. Mabry P
    25. Madon T
    26. Malhotra N
    27. Mayo-Wilson E
    28. McNutt M
    29. Miguel E
    30. Paluck EL
    31. Simonsohn U
    32. Soderberg C
    33. Spellman BA
    34. Turitto J
    35. VandenBos G
    36. Vazire S
    37. Wagenmakers EJ
    38. Wilson R
    39. Yarkoni T

    (2015) Promoting an open research culture

    Science 348:1422–1425.

    https://doi.org/10.1126/science.aab2374

    • Google Scholar
20. 1. Opthof T

    (2019) Comparison of the impact factors of the most-cited cardiovascular journals

    Circulation Research 124:1718–1724.

    https://doi.org/10.1161/CIRCRESAHA.119.315249

    • PubMed
    • Google Scholar
21. Website

    1. Organic Syntheses

    (2023) Organic Syntheses: A publication of Reliable Methods for the Preparation of Organic Compounds

    Accessed February 10, 2023.

    https://www.orgsyn.org/
22. 1. Page MJ
    2. McKenzie JE
    3. Bossuyt PM
    4. Boutron I
    5. Hoffmann TC
    6. Mulrow CD
    7. Shamseer L
    8. Tetzlaff JM
    9. Akl EA
    10. Brennan SE
    11. Chou R
    12. Glanville J
    13. Grimshaw JM
    14. Hróbjartsson A
    15. Lalu MM
    16. Li T
    17. Loder EW
    18. Mayo-Wilson E
    19. McDonald S
    20. McGuinness LA
    21. Stewart LA
    22. Thomas J
    23. Tricco AC
    24. Welch VA
    25. Whiting P
    26. Moher D

    (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

    BMJ 372:n71.

    https://doi.org/10.1136/bmj.n71

    • PubMed
    • Google Scholar
23. Website

    1. Repeat

    (2023) Reproducible Evidence: Practices to Enhance and Achieve Transparency

    Accessed January 6, 2023.

    https://www.repeatinitiative.org/
24. 1. Russo JJ
    2. Aleksova N
    3. Pitcher I
    4. Couture E
    5. Parlow S
    6. Faraz M
    7. Visintini S
    8. Simard T
    9. Di Santo P
    10. Mathew R
    11. So DY
    12. Takeda K
    13. Garan AR
    14. Karmpaliotis D
    15. Takayama H
    16. Kirtane AJ
    17. Hibbert B

    (2019) Left ventricular unloading during extracorporeal membrane oxygenation in patients with cardiogenic shock

    Journal of the American College of Cardiology 73:654–662.

    https://doi.org/10.1016/j.jacc.2018.10.085

    • PubMed
    • Google Scholar
25. 1. Seltzer E
    2. Goldshear J
    3. Guntuku SC
    4. Grande D
    5. Asch DA
    6. Klinger EV
    7. Merchant RM

    (2019) Patients’ willingness to share digital health and non-health data for research: a cross-sectional study

    BMC Medical Informatics and Decision Making 19:157.

    https://doi.org/10.1186/s12911-019-0886-9

    • PubMed
    • Google Scholar
26. 1. Sherry CE
    2. Pollard JZ
    3. Tritz D
    4. Carr BK
    5. Pierce A
    6. Vassar M

    (2020) Assessment of transparent and reproducible research practices in the psychiatry literature

    General Psychiatry 33:e100149.

    https://doi.org/10.1136/gpsych-2019-100149

    • PubMed
    • Google Scholar
27. 1. Smith CA
    2. Nolan J
    3. Tritz DJ
    4. Heavener TE
    5. Pelton J
    6. Cook K
    7. Vassar M

    (2021) Evaluation of reproducible and transparent research practices in pulmonology

    Pulmonology 27:134–143.

    https://doi.org/10.1016/j.pulmoe.2020.07.001

    • PubMed
    • Google Scholar
28. 1. Sumner JQ
    2. Vitale CH
    3. McIntosh LD

    (2021) RipetaScore: measuring the quality, transparency, and trustworthiness of a scientific work

    Frontiers in Research Metrics and Analytics 6:751734.

    https://doi.org/10.3389/frma.2021.751734

    • PubMed
    • Google Scholar
29. 1. Walters C
    2. Harter ZJ
    3. Wayant C
    4. Vo N
    5. Warren M
    6. Chronister J
    7. Tritz D
    8. Vassar M

    (2019) Do oncology researchers adhere to reproducible and transparent principles? A cross-sectional survey of published oncology literature

    BMJ Open 9:e033962.

    https://doi.org/10.1136/bmjopen-2019-033962

    • PubMed
    • Google Scholar
30. 1. Wang SV
    2. Sreedhara SK
    3. Schneeweiss S
    4. REPEAT Initiative

    (2022) Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions

    Nature Communications 13:5126.

    https://doi.org/10.1038/s41467-022-32310-3

    • PubMed
    • Google Scholar

Author details

1. Gabriel O Heckerman

   1. Western Kentucky University, Bowling Green, United States
   2. Stanford Cardiovascular Institute, Stanford, United States

   Contribution

   Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Eileen Tzng and Arely Campos-Melendez

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-6420-8909

2. Eileen Tzng

   1. Stanford Cardiovascular Institute, Stanford, United States
   2. Cornell University, Ithaca, United States

   Contribution

   Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Gabriel O Heckerman and Arely Campos-Melendez

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7513-9192

3. Arely Campos-Melendez

   1. Stanford Cardiovascular Institute, Stanford, United States
   2. University of California, Los Angeles, Los Angeles, United States

   Contribution

   Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   Gabriel O Heckerman and Eileen Tzng

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0322-1170

4. Chisomaga Ekwueme

   1. Stanford Cardiovascular Institute, Stanford, United States
   2. University of California, Davis, Davis, United States

   Contribution

   Software, Validation, Investigation, Visualization, Methodology

   Competing interests

   No competing interests declared

5. Adrienne Mueller

   Stanford Cardiovascular Institute, Stanford, United States

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Investigation, Methodology, Writing – original draft, Project administration, Writing – review and editing

   For correspondence

   alm04@stanford.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-9161-5323

• 277

  views

• 26

  downloads

• 0

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.