Standardized mean differences cause funnel plot distortion in publication bias assessments
Abstract
Meta-analyses are increasingly used for synthesis of evidence, and often include an assessment of publication bias based on detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.
Article and author information
Author details
Funding
National Institute of Environmental Health Sciences (National Toxicology Program research funding)
- Kimberley E Wever
Netherlands Cardiovascular Research Initiative (CVON-HUSTCARE)
- Steven AJ Chamuleau
National Centre for the Replacement, Refinement and Reduction of Animals in Research (Infrastructure Award)
- Emily S Sena
- Malcolm R MacLeod
Alexander Suerman program (PhD student Scholarship)
- Peter-Paul Zwetsloot
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Copyright
© 2017, Zwetsloot et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 6,186
- views
-
- 457
- downloads
-
- 160
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Citations by DOI
-
- 160
- citations for umbrella DOI https://doi.org/10.7554/eLife.24260