Response to comment on 'The distribution of antibiotic use and its association with antibiotic resistance'
We are writing to reply to the comment by Pouwels et al., 2019 about our recent study (Olesen et al., 2018) on antibiotic use and antibiotic resistance.https://doi.org/10.7554/eLife.47124.001
We thank Pouwels and colleagues for their thoughtful critique and appreciate their concern for the role that analyses like ours could play in public health policy (Pouwels et al., 2019). Moreover, we appreciate their willingness to discuss and refine these ideas through both formal and informal conversations, as we work together towards the shared goal of addressing the challenges posed by antimicrobial resistance. Although we disagree on certain points, discussed below, we note that our disagreements depend on remarkably subtle arguments about specific aspects of prescriber behavior and its role in reverse causality. We take this as an overall sign that research into the epidemiology of antibiotic resistance will benefit from further study of individual prescriber behavior in response to patient-level risks and population-level prevalence of antibiotic resistance.
In our original analysis, we evaluated several questions about the relationships between antibiotic use and resistance. First, we described the distribution of antibiotic use across the population and showed that it is uneven, as some individuals had no antibiotic pharmacy claims and others had many. Second, we characterized the landscape of correlations between antibiotic use and resistance for antibiotics and many pathogens. Third, we investigated whether first or repeat antibiotic use is more strongly associated with resistance at the population level. To do so, we partitioned a person's number of antibiotic pharmacy claims in a year into the sum of ‘first use’ – zero if the person had no claims for that antibiotic, one if they had at least one claim – and ‘repeat use’, the number of claims beyond the first. Previous reports speculated that repeat use would have a greater effect on population-level antibiotic resistance than first use, in other words, that ‘intensive’ use of antibiotics in a few people is more important for resistance than ‘extensive’ use across many people. In our analysis, we found no evidence that repeat use was more strongly associated with resistance. If true, these results imply that antibiotic stewardship should not necessarily focus their attention on a few, high-consuming antibiotic users.
Pouwels et al., 2019 raised several concerns with the results about the relationship between population-level antibiotic resistance and first vs repeat antibiotic use. First, they argue that reverse causality (rational prescriber response to local resistance rates) dominates over forward causality (the biological selection for resistance) in associations between antibiotic use and resistance, so that our analytical results about the relationships between first use, repeat use, and antibiotic resistance do not support our conclusions about how antibiotic stewardship should be performed. Second, they argue that the relative roles of forward and reverse causality differ for first and repeat use in a way that further undermines our original conclusions. Third, they argue that analyzing use of all antibiotics, rather than use of particular classes of antibiotics, changes the relative role of forward and reverse causality and find evidence that, if this assumption were correct, would even further undermine our conclusions. Finally, they argue that there are many confounders, data problems, and open methodological questions that limit the applicability of our results for determining public health policy.
First, does reverse causality (rational prescriber response to local resistance rates) dominate over forward causality (the biological selection for resistance) in associations between antibiotic use and resistance, as Pouwels et al., 2019 maintain? We disagree. As outpatient prescribers do not in general rationally account for resistance rates (Simpson et al., 2007), we are not convinced that reverse causality dominates. However, the degree to which clinicians alter their antibiotic prescribing practice in response to local rates of antibiotic resistance is an empirical question, and our debate could be resolved by better direct quantification of reverse causality.
Second, does reverse causality influence repeat use more than it influences first use? We again disagree with Pouwels et al., 2019. We expect that high population-level resistance against an antibiotic would discourage rational prescribers from both first and repeat use of that antibiotic. In contrast, in cases of low population-level resistance, rational prescribers would be more discouraged against repeat use than first use, because resistance may arise because of antibiotic exposure in that patient. In other words, prior antibiotic use is a more important risk factor for patient-level resistance when population-level resistance is low compared to when population-level resistance is high. We therefore expect that reverse causality, when population-level resistance affects individual-level prescribing, would have a greater influence on first use than repeat use. If this is true, then the associations between first use and resistance we observed in our original analyses, which tended to be positive, are actually underestimates of the forward-causal effect of antibiotic use on resistance, in fact strengthening the interpretation of our original result that first use has a more important forward-causal effect on resistance than repeat use.
Third, how does analyzing overall antibiotic, rather than the use of particular antibiotic classes, affect the interpretation of the resulting models? We disagree with the interpretation of this kind of analysis in Pouwels et al., 2019. In the original analyses (Olesen et al., 2018), we found that the associations between repeat use of an antibiotic and resistance to that antibiotic tend to be negative. Re-analyzing our data, Pouwels et al., 2019 found that associations between overall repeat antibiotic use and resistance to particular classes of antibiotics tend to be positive. They asserted that focusing on overall antibiotic use limits the role of reverse causality and concluded that repeat use has a more important forward-causality effect on resistance than repeat use, the opposite of our original finding. Although we generally agree with their logic about how focusing on overall use limits the role of reverse causality, we expect that focusing on overall use limits the role of forward causality as well. It is not clear to us that reverse causality is limited to a greater degree than is forward causality. If this is true, the finding of Pouwels et al., 2019 – that overall repeat use is better associated with resistance than overall first use – does not contradict our original finding – that first use of individual antibiotic classes is better associated with resistance to those antibiotics than repeat use of those antibiotics. We explain our logic in the context of first and repeat use separately.
With respect to first use, Pouwels et al., 2019 asserted that overall first use – that is, the proportion of people who claimed any antibiotic – is an indicator of how many patients require an antibiotic prescription. This measure should be less affected by antibiotic resistance levels than first use rates for any particular class of antibiotics. Thus, models of overall first use rather than first use of particular antibiotic classes should be less susceptible to reverse causality. Although we agree with this logic about reverse causality, it does not address how the change in model predictors affects forward causality. In the absence of reverse causality, use of antibiotic A should be a better predictor for resistance to A than use of any antibiotic, even given co-selection and other mechanisms by which use of other antibiotics can select for resistance to antibiotic A. In other words, using overall first use as a predictor dilutes forward causality. The relative importance of the two effects is not clear to us: is the dilution of forward causality sufficiently counterbalanced by the diminution of reverse causality?
We similarly question the logic that models of overall repeat use rather than repeat use of particular antibiotic classes should be less susceptible to reverse causality. Pouwels et al., 2019 mentioned that a high resistance rate to a first-line therapy will mean that, on average, more unique antibiotics will be required to cure the infection. Overall repeat use captures this drug-switching, emphasizing reverse causality: when resistance to an antibiotic is high for a given pathogen, more antibiotic prescriptions, especially for antibiotics of different classes, will be used to treat infections due to that pathogen. As with first use, it is unclear to us why overall repeat antibiotic use should have a stronger forward-causal, biologically selective effect on resistance to a particular class of antibiotics, compared to repeat use of antibiotics in that particular class. Again, however, the lack of strong evidence about the role of antibiotic resistance in prescribers' decisions to switch antibiotic treatments for a patient limits a productive debate.
In contrast to the above, we heartily agree with the final point of Pouwels et al., 2019 that there are many confounders, data problems, and open methodological questions that may limit the applicability of our results for determining public health policy. We welcome further research in the areas of reverse causality and the determination of precisely which antibiotic use, by whom and of what drugs, are the principal drivers of resistance. We also applaud efforts to generate more reliable data sources so that estimates of antibiotic resistance in the US need not rely on, as we did, a convenience samples of hospital antibiotic resistance reports. However, in the meantime, we hold that today's public health policy must proceed based on the best available evidence.
General practitioners' perceptions of antimicrobial resistance: a qualitative studyJournal of Antimicrobial Chemotherapy 59:292–296.https://doi.org/10.1093/jac/dkl467
Article and author information
National Institute of General Medical Sciences (U54GM088558)
- Yonatan H Grad
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of General Medical Sciences or the National Institutes of Health.
Senior and Reviewing Editor
- Neil M Ferguson, Imperial College London, United Kingdom
- Received: March 26, 2019
- Accepted: April 24, 2019
- Version of Record published: May 3, 2019 (version 1)
© 2019, Olesen et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
- Page views
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
- Epidemiology and Global Health
- Microbiology and Infectious Disease
Antibiotic use is a primary driver of antibiotic resistance. However, antibiotic use can be distributed in different ways in a population, and the association between the distribution of use and antibiotic resistance has not been explored. Here, we tested the hypothesis that repeated use of antibiotics has a stronger association with population-wide antibiotic resistance than broadly-distributed, low-intensity use. First, we characterized the distribution of outpatient antibiotic use across US states, finding that antibiotic use is uneven and that repeated use of antibiotics makes up a minority of antibiotic use. Second, we compared antibiotic use with resistance for 72 pathogen-antibiotic combinations across states. Finally, having partitioned total use into extensive and intensive margins, we found that intense use had a weaker association with resistance than extensive use. If the use-resistance relationship is causal, these results suggest that reducing total use and selection intensity will require reducing broadly distributed, low-intensity use.
- Epidemiology and Global Health
Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022.
We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance.
Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models.
Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks.
AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 (https://www.nfdi4health.de/task-force-covid-19-2) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).