1. Epidemiology and Global Health
  2. Human Biology and Medicine
Download icon

Standardized mean differences cause funnel plot distortion in publication bias assessments

  1. Peter-Paul Zwetsloot  Is a corresponding author
  2. Mira Van Der Naald
  3. Emily S Sena
  4. David W Howells
  5. Joanna IntHout
  6. Joris AH De Groot
  7. Steven AJ Chamuleau
  8. Malcolm R MacLeod
  9. Kimberley E Wever  Is a corresponding author
  1. University Medical Center Utrecht, Netherlands
  2. University of Edinburgh, United Kingdom
  3. University of Tasmania, Australia
  4. Radboud Institute for Health Sciences, Radboud University Medical Center, Netherlands
  5. The University of Edinburgh, United Kingdom
  6. Radboud University Medical Center, Netherlands
Research Article
  • Cited 34
  • Views 2,089
  • Annotations
Cite this article as: eLife 2017;6:e24260 doi: 10.7554/eLife.24260

Abstract

Meta-analyses are increasingly used for synthesis of evidence, and often include an assessment of publication bias based on detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.

Article and author information

Author details

  1. Peter-Paul Zwetsloot

    Department of Cardiology, Experimental Cardiology Laboratory, University Medical Center Utrecht, Utrecht, Netherlands
    For correspondence
    P.P.M.Zwetsloot@umcutrecht.nl
    Competing interests
    The authors declare that no competing interests exist.
  2. Mira Van Der Naald

    Department of Cardiology, Experimental Cardiology Laboratory, University Medical Center Utrecht, Utrecht, Netherlands
    Competing interests
    The authors declare that no competing interests exist.
  3. Emily S Sena

    Center for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
  4. David W Howells

    School of Medicine, University of Tasmania, Hobart, Australia
    Competing interests
    The authors declare that no competing interests exist.
  5. Joanna IntHout

    Department for Health Evidence, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, Netherlands
    Competing interests
    The authors declare that no competing interests exist.
  6. Joris AH De Groot

    Department for Health Evidence, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, Netherlands
    Competing interests
    The authors declare that no competing interests exist.
  7. Steven AJ Chamuleau

    Department of Cardiology, Experimental Cardiology Laboratory, University Medical Center Utrecht, Utrecht, Netherlands
    Competing interests
    The authors declare that no competing interests exist.
  8. Malcolm R MacLeod

    Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
  9. Kimberley E Wever

    Systematic Review Centre for Laboratory Animal Experimentation, Radboud University Medical Center, Nijmegen, Netherlands
    For correspondence
    kim.wever@radboudumc.nl
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3635-3660

Funding

National Institute of Environmental Health Sciences (National Toxicology Program research funding)

  • Kimberley E Wever

Netherlands Cardiovascular Research Initiative (CVON-HUSTCARE)

  • Steven AJ Chamuleau

National Centre for the Replacement, Refinement and Reduction of Animals in Research (Infrastructure Award)

  • Emily S Sena
  • Malcolm R MacLeod

Alexander Suerman program (PhD student Scholarship)

  • Peter-Paul Zwetsloot

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. M Dawn Teare, University of Sheffield, United Kingdom

Publication history

  1. Received: December 14, 2016
  2. Accepted: August 21, 2017
  3. Accepted Manuscript published: September 8, 2017 (version 1)
  4. Version of Record published: September 29, 2017 (version 2)

Copyright

© 2017, Zwetsloot et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,089
    Page views
  • 248
    Downloads
  • 34
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Epidemiology and Global Health
    2. Microbiology and Infectious Disease
    June H Tan et al.
    Research Advance

    Parasitic helminths use two benzoquinones as electron carriers in the electron transport chain. In normoxia they use ubiquinone (UQ), but in the anaerobic conditions inside the host, they require rhodoquinone (RQ) and greatly increase RQ levels. We previously showed the switch from UQ to RQ synthesis is driven by a change in substrates by the polyprenyltransferase COQ-2 (Del Borrello et al., 2019; Roberts Buceta et al., 2019) - how this substrate choice is made is unknown. Here, we show helminths make two coq-2 splice forms, coq-2a and coq-2e, and the coq-2e-specific exon is only found in species that make RQ. We show that in C. elegans COQ-2e is required for efficient RQ synthesis and for survival in cyanide. Crucially, parasites switch from COQ-2a to COQ-2e as they transition into anaerobic environments. We conclude helminths switch from UQ to RQ synthesis principally via changes in the alternative splicing of coq-2.

    1. Epidemiology and Global Health
    Flavia Camponovo et al.
    Research Article Updated

    Tanzanian adult male volunteers were immunized by direct venous inoculation with radiation-attenuated, aseptic, purified, cryopreserved Plasmodium falciparum (Pf) sporozoites (PfSPZ Vaccine) and protective efficacy assessed by homologous controlled human malaria infection (CHMI). Serum immunoglobulin G (IgG) responses were analyzed longitudinally using a Pf protein microarray covering 91% of the proteome, providing first insights into naturally acquired and PfSPZ Vaccine-induced whole parasite antibody profiles in malaria pre-exposed Africans. Immunoreactivity was identified against 2239 functionally diverse Pf proteins, showing a wide breadth of humoral response. Antibody-based immune ‘fingerprints’ in these individuals indicated a strong person-specific immune response at baseline, with little changes in the overall humoral immunoreactivity pattern measured after immunization. The moderate increase in immunogenicity following immunization and the extensive and variable breadth of humoral immune response observed in the volunteers at baseline suggest that pre-exposure reduces vaccine-induced antigen reactivity in unanticipated ways.