Standardized mean differences cause funnel plot distortion in publication bias assessments
Abstract
Meta-analyses are increasingly used for synthesis of evidence, and often include an assessment of publication bias based on detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.
Article and author information
Author details
Funding
National Institute of Environmental Health Sciences (National Toxicology Program research funding)
- Kimberley E Wever
Netherlands Cardiovascular Research Initiative (CVON-HUSTCARE)
- Steven AJ Chamuleau
National Centre for the Replacement, Refinement and Reduction of Animals in Research (Infrastructure Award)
- Emily S Sena
- Malcolm R MacLeod
Alexander Suerman program (PhD student Scholarship)
- Peter-Paul Zwetsloot
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Copyright
© 2017, Zwetsloot et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 6,106
- views
-
- 450
- downloads
-
- 143
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Epidemiology and Global Health
Background: The role of circulating metabolites on child development is understudied. We investigated associations between children's serum metabolome and early childhood development (ECD).
Methods: Untargeted metabolomics was performed on serum samples of 5,004 children aged 6-59 months, a subset of participants from the Brazilian National Survey on Child Nutrition (ENANI-2019). ECD was assessed using the Survey of Well-being of Young Children's milestones questionnaire. The graded response model was used to estimate developmental age. Developmental quotient (DQ) was calculated as the developmental age divided by chronological age. Partial least square regression selected metabolites with a variable importance projection ≥ 1. The interaction between significant metabolites and the child's age was tested.
Results: Twenty-eight top-ranked metabolites were included in linear regression models adjusted for the child's nutritional status, diet quality, and infant age. Cresol sulfate (β = -0.07; adjusted-p < 0.001), hippuric acid (β = -0.06; adjusted-p < 0.001), phenylacetylglutamine (β = -0.06; adjusted-p < 0.001), and trimethylamine-N-oxide (β = -0.05; adjusted-p = 0.002) showed inverse associations with DQ. We observed opposite directions in the association of DQ for creatinine (for children aged -1 SD: β = -0.05; p =0.01; +1 SD: β = 0.05; p =0.02) and methylhistidine (-1 SD: β = - 0.04; p =0.04; +1 SD: β = 0.04; p =0.03).
Conclusion: Serum biomarkers, including dietary and microbial-derived metabolites involved in the gut-brain axis, may potentially be used to track children at risk for developmental delays.
Funding: Supported by the Brazilian Ministry of Health and the Brazilian National Research Council.
-
- Epidemiology and Global Health
Given the rapid cross-country spread of SARS-CoV-2 and the resulting difficulty in tracking lineage spread, we investigated the potential of combining mobile service data and fine-granular metadata (such as postal codes and genomic data) to advance integrated genomic surveillance of the pandemic in the federal state of Thuringia, Germany. We sequenced over 6500 SARS-CoV-2 Alpha genomes (B.1.1.7) across 7 months within Thuringia while collecting patients’ isolation dates and postal codes. Our dataset is complemented by over 66,000 publicly available German Alpha genomes and mobile service data for Thuringia. We identified the existence and spread of nine persistent mutation variants within the Alpha lineage, seven of which formed separate phylogenetic clusters with different spreading patterns in Thuringia. The remaining two are subclusters. Mobile service data can indicate these clusters’ spread and highlight a potential sampling bias, especially of low-prevalence variants. Thereby, mobile service data can be used either retrospectively to assess surveillance coverage and efficiency from already collected data or to actively guide part of a surveillance sampling process to districts where these variants are expected to emerge. The latter concept was successfully implemented as a proof-of-concept for a mobility-guided sampling strategy in response to the surveillance of Omicron sublineage BQ.1.1. The combination of mobile service data and SARS-CoV-2 surveillance by genome sequencing is a valuable tool for more targeted and responsive surveillance.