Standardized mean differences cause funnel plot distortion in publication bias assessments
Abstract
Meta-analyses are increasingly used for synthesis of evidence, and often include an assessment of publication bias based on detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.
Article and author information
Author details
Funding
National Institute of Environmental Health Sciences (National Toxicology Program research funding)
- Kimberley E Wever
Netherlands Cardiovascular Research Initiative (CVON-HUSTCARE)
- Steven AJ Chamuleau
National Centre for the Replacement, Refinement and Reduction of Animals in Research (Infrastructure Award)
- Emily S Sena
- Malcolm R MacLeod
Alexander Suerman program (PhD student Scholarship)
- Peter-Paul Zwetsloot
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Copyright
© 2017, Zwetsloot et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 5,991
- views
-
- 437
- downloads
-
- 134
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Epidemiology and Global Health
- Genetics and Genomics
Background:
Maternal smoking has been linked to adverse health outcomes in newborns but the extent to which it impacts newborn health has not been quantified through an aggregated cord blood DNA methylation (DNAm) score. Here, we examine the feasibility of using cord blood DNAm scores leveraging large external studies as discovery samples to capture the epigenetic signature of maternal smoking and its influence on newborns in White European and South Asian populations.
Methods:
We first examined the association between individual CpGs and cigarette smoking during pregnancy, and smoking exposure in two White European birth cohorts (n=744). Leveraging established CpGs for maternal smoking, we constructed a cord blood epigenetic score of maternal smoking that was validated in one of the European-origin cohorts (n=347). This score was then tested for association with smoking status, secondary smoking exposure during pregnancy, and health outcomes in offspring measured after birth in an independent White European (n=397) and a South Asian birth cohort (n=504).
Results:
Several previously reported genes for maternal smoking were supported, with the strongest and most consistent association signal from the GFI1 gene (6 CpGs with p<5 × 10-5). The epigenetic maternal smoking score was strongly associated with smoking status during pregnancy (OR = 1.09 [1.07, 1.10], p=5.5 × 10-33) and more hours of self-reported smoking exposure per week (1.93 [1.27, 2.58], p=7.8 × 10-9) in White Europeans. However, it was not associated with self-reported exposure (p>0.05) among South Asians, likely due to a lack of smoking in this group. The same score was consistently associated with a smaller birth size (–0.37±0.12 cm, p=0.0023) in the South Asian cohort and a lower birth weight (–0.043±0.013 kg, p=0.0011) in the combined cohorts.
Conclusions:
This cord blood epigenetic score can help identify babies exposed to maternal smoking and assess its long-term impact on growth. Notably, these results indicate a consistent association between the DNAm signature of maternal smoking and a small body size and low birth weight in newborns, in both White European mothers who exhibited some amount of smoking and in South Asian mothers who themselves were not active smokers.
Funding:
This study was funded by the Canadian Institutes of Health Research Metabolomics Team Grant: MWG-146332.
-
- Epidemiology and Global Health
- Microbiology and Infectious Disease
Background:
Few national-level studies have evaluated the impact of ‘hybrid’ immunity (vaccination coupled with recovery from infection) from the Omicron variants of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).
Methods:
From May 2020 to December 2022, we conducted serial assessments (each of ~4000–9000 adults) examining SARS-CoV-2 antibodies within a mostly representative Canadian cohort drawn from a national online polling platform. Adults, most of whom were vaccinated, reported viral test-confirmed infections and mailed self-collected dried blood spots (DBSs) to a central lab. Samples underwent highly sensitive and specific antibody assays to spike and nucleocapsid protein antigens, the latter triggered only by infection. We estimated cumulative SARS-CoV-2 incidence prior to the Omicron period and during the BA.1/1.1 and BA.2/5 waves. We assessed changes in antibody levels and in age-specific active immunity levels.
Results:
Spike levels were higher in infected than in uninfected adults, regardless of vaccination doses. Among adults vaccinated at least thrice and infected more than 6 months earlier, spike levels fell notably and continuously for the 9-month post-vaccination. In contrast, among adults infected within 6 months, spike levels declined gradually. Declines were similar by sex, age group, and ethnicity. Recent vaccination attenuated declines in spike levels from older infections. In a convenience sample, spike antibody and cellular responses were correlated. Near the end of 2022, about 35% of adults above age 60 had their last vaccine dose more than 6 months ago, and about 25% remained uninfected. The cumulative incidence of SARS-CoV-2 infection rose from 13% (95% confidence interval 11–14%) before omicron to 78% (76–80%) by December 2022, equating to 25 million infected adults cumulatively. However, the coronavirus disease 2019 (COVID-19) weekly death rate during the BA.2/5 waves was less than half of that during the BA.1/1.1 wave, implying a protective role for hybrid immunity.
Conclusions:
Strategies to maintain population-level hybrid immunity require up-to-date vaccination coverage, including among those recovering from infection. Population-based, self-collected DBSs are a practicable biological surveillance platform.
Funding:
Funding was provided by the COVID-19 Immunity Task Force, Canadian Institutes of Health Research, Pfizer Global Medical Grants, and St. Michael’s Hospital Foundation. PJ and ACG are funded by the Canada Research Chairs Program.