The effect of combining antibiotics on resistance: A systematic review and meta-analysis

Reviewer #1 (Public Review):

Summary:
The investigators have performed a state-of-the art systematic review and meta-analysis of studies that may help to answer the research question: if administration of multiple antibiotics simultaneously prevents antibiotic resistance development in individuals. The amount of studies eligible for analysis is very low, and within that low number, there is huge variability in bug-drug combinations studied and most studies had a high risk of bias, further limiting the capability of meta-analysis to answer the research question. In addition, based on I2 values there is also huge statistical heterogeneity between outcomes of studies compared, further limiting the predictive value of meta-analysis. In fact, the only 2 studies meeting all eligibility criteria addressed the treatment of mycobacterium tuberculosis, for which the research question is hardly applicable. The authors, therefore, conclude that "our analysis could not identify any benefit or harm of using a higher or a lower number of antibiotics regarding within-patient resistance development." Apart from articulating this knowledge gap, the findings will not have consequences for patient care, but may stimulate the scientific community to better address this research question in future studies.

Strengths:
The systematic and rigorous approach for the review and meta-analysis.

Weaknesses:
None identified.

Reviewer #2 (Public Review):

Summary:
The authors performed a systematic review and meta-analysis to investigate whether the frequency of emergence of resistance is different if combination antibiotic therapy is used compared to fewer antibiotics. The review shows that there is currently insufficient evidence to reach a conclusion due to the limited sample size. High-quality studies evaluating appropriate antimicrobial resistance endpoints are needed.

Strengths:
The strengths of the manuscript are that the article addresses a relevant research question that is often debated. The article is well-written and the methodology used is valid. The review shows that there is currently insufficient evidence to reach a conclusion due to the limited sample size. High-quality studies evaluating appropriate antimicrobial resistance endpoints are needed. I have several comments and suggestions for the manuscript.

Weaknesses:
Weaknesses of the manuscript are the large clinical and statistical heterogeneity and the lack of clear definitions of acquisition of resistance. Both these weaknesses complicate the interpretation of the study results.

Major comments:
My main concern about the manuscript is the extent of both clinical and statistical heterogeneity, which complicates the interpretation of the results. I don't understand some of the antibiotic comparisons that are included in the systematic review. For instance the study by Paul et al (50), where vancomycin (as monotherapy) is compared to co-trimoxazole (as combination therapy). Emergence (or selection) of co-trimoxazole in S. aureus is in itself much more common than vancomycin resistance. It is logical and expected to have more resistance in the co-trimoxazole group compared to the vancomycin group, however, this difference is due to the drug itself and not due to co-trimoxazole being a combination therapy. It is therefore unfair to attribute the difference in resistance to combination therapy. Another example is the study by Walsh (71) where rifampin + novobiocin is compared to rifampin + co-trimoxazole. There is more emergence of resistance in the rifampin + co-trimoxazole group but this could be attributed to novobiocin being a different type of antibiotic than co-trimoxazole instead of the difference being attributed to combination therapy. To improve interpretation and reduce heterogeneity my suggestion would be to limit the primary analyses to regimens where the antibiotics compared are the same but in one group one or more antibiotic(s) are added (i.e. A versus A+B). The other analyses are problematic in their interpretation and should be clearly labeled as secondary and their interpretation discussed.

Another concern is about the definition of acquisition of resistance, which is unclear to me. If for example meropenem is administered and the follow-up cultures show Enterococcus species (which is intrinsically resistant to meropenem), does this constitute acquisition of resistance? If so, it would be misleading to determine this as an acquisition of resistance, as many people are colonized with Enterococci and selection of Enterococci under therapy is very common. If this is not considered as the acquisition of resistance please include how the acquisition of resistance is defined per included study. Table S1 is not sufficiently clear because it often only contains how susceptibility testing was done but not which antibiotics were tested and how a strain was classified as resistant or susceptible.

Line 85: "Even though within-patient antibiotic resistance development is rare, it may contribute to the emergence and spread of resistance."
Depending on the bug-drug combination, there is great variation in the propensity to develop within-patient antibiotic resistance. For example: within-patient development of ciprofloxacin resistance in Pseudomonas is fairly common while within-patient development of methicillin resistance in S. aureus is rare. Based on these differences, large clinical heterogeneity is expected and it is questionable where these studies should be pooled.

Line 114: "The overall pooled OR for acquisition of resistance comparing a lower number of antibiotics versus a higher one was 1.23 (95% CI 0.68 - 2.25), with substantial heterogeneity between studies (I2=77.4%)"
What consequential measures did the authors take after determining this high heterogeneity? Did they explore the source of this large heterogeneity? Considering this large heterogeneity, do the authors consider it appropriate to pool these studies?

Author Response

Reviewer #1 (Public Review):

Summary:

The investigators have performed a state-of-the art systematic review and meta-analysis of studies that may help to answer the research question: if administration of multiple antibiotics simultaneously prevents antibiotic resistance development in individuals. The amount of studies eligible for analysis is very low, and within that low number, there is huge variability in bug-drug combinations studied and most studies had a high risk of bias, further limiting the capability of meta-analysis to answer the research question. In addition, based on I2 values there is also huge statistical heterogeneity between outcomes of studies compared, further limiting the predictive value of meta-analysis. In fact, the only 2 studies meeting all eligibility criteria addressed the treatment of mycobacterium tuberculosis, for which the research question is hardly applicable. The authors, therefore, conclude that "our analysis could not identify any benefit or harm of using a higher or a lower number of antibiotics regarding within-patient resistance development." Apart from articulating this knowledge gap, the findings will not have consequences for patient care, but may stimulate the scientific community to better address this research question in future studies.

Strengths:

The systematic and rigorous approach for the review and meta-analysis.

Weaknesses:

None identified.

We thank the reviewer for this thoughtful and positive appraisal of our work.

Reviewer #2 (Public Review):

Summary:

The authors performed a systematic review and meta-analysis to investigate whether the frequency of emergence of resistance is different if combination antibiotic therapy is used compared to fewer antibiotics. The review shows that there is currently insufficient evidence to reach a conclusion due to the limited sample size. High-quality studies evaluating appropriate antimicrobial resistance endpoints are needed.

Strengths:

The strengths of the manuscript are that the article addresses a relevant research question that is often debated. The article is well-written and the methodology used is valid. The review shows that there is currently insufficient evidence to reach a conclusion due to the limited sample size. High-quality studies evaluating appropriate antimicrobial resistance endpoints are needed. I have several comments and suggestions for the manuscript.

Weaknesses:

Weaknesses of the manuscript are the large clinical and statistical heterogeneity and the lack of clear definitions of acquisition of resistance. Both these weaknesses complicate the interpretation of the study results.

We thank the reviewer for the positive comments and pointing out where our work can be improved.

Major comments:

My main concern about the manuscript is the extent of both clinical and statistical heterogeneity, which complicates the interpretation of the results. I don't understand some of the antibiotic comparisons that are included in the systematic review. For instance the study by Paul et al (50), where vancomycin (as monotherapy) is compared to co-trimoxazole (as combination therapy). Emergence (or selection) of co-trimoxazole in S. aureus is in itself much more common than vancomycin resistance. It is logical and expected to have more resistance in the co-trimoxazole group compared to the vancomycin group, however, this difference is due to the drug itself and not due to co-trimoxazole being a combination therapy. It is therefore unfair to attribute the difference in resistance to combination therapy. Another example is the study by Walsh (71) where rifampin + novobiocin is compared to rifampin + co-trimoxazole. There is more emergence of resistance in the rifampin + co-trimoxazole group but this could be attributed to novobiocin being a different type of antibiotic than co-trimoxazole instead of the difference being attributed to combination therapy. To improve interpretation and reduce heterogeneity my suggestion would be to limit the primary analyses to regimens where the antibiotics compared are the same but in one group one or more antibiotic(s) are added (i.e. A versus A+B). The other analyses are problematic in their interpretation and should be clearly labeled as secondary and their interpretation discussed.

We acknowledge the presence of statistical and clinical heterogeneity in our overall analysis. The decision to pursue this comprehensive examination was predefined in our previously published study protocol (PROSPERO CRD42020187257) and driven by our interest whether, despite some differences, we could either identify an overarching effect of combination therapy on resistance or identify factors that explain potential differences of the effect of combination therapy across pathogens/drugs. We indeed, find that heterogeneity is high, however identifying the driving factors of this heterogeneity is difficult as evidence is limited.

We carried out several subgroup analyses, e.g. explicitly focusing on specific pathogen groups and medical conditions or exploring heterogeneity in treatment arms (figure 3, supplementary materials section 6). However, it is important to highlight that the number of studies available for these subgroup analyses was low. Additionally, recognizing the high heterogeneity within treatment arms, we performed a subgroup analysis focusing solely on resistances of antibiotics common to both arms (supplementary material section 6.1.8; which would avoid comparisons such as the one between vancomycin and co-trimoxazole raised by the reviewer). Unfortunately, this also revealed substantial heterogeneity. While we aimed to address heterogeneity through these subgroup analyses, limitations arose due to the number of studies meeting specific criteria and the nature of data provided by these studies.

Moreover, regarding the concern on interpretation of co-trimoxazole as combination therapy, we acknowledge the confusion surrounding its classification as one or two antibiotics. Despite the common contemporary view of co-trimoxazole as a single antibiotic, we chose to consider it as two antibiotics due to historical practices, as observed in Black et al. (1982), where trimethoprim was compared to trimethoprim and sulfamethoxazole. We recognize that this decision may lead to confusion and we consider conducting a further sensitivity analysis in the future version of this manuscript, exploring the possibility of considering co-trimoxazole as a single antibiotic. We agree that the slight trend of less antibiotics performing better overserved for MRSA, should not be over interpreted as this is driven by the two studies Walsh et al 1993 and Paul et al 2015 as pointed out by the reviewer. In lines 183-186 we discuss this issue that for better evaluation of antibiotic combination therapy, more studies which use identical antibiotics (i.e. A versus A+B) are needed. We will try to clarify and highlight this in the future version of the manuscript.

Another concern is about the definition of acquisition of resistance, which is unclear to me. If for example meropenem is administered and the follow-up cultures show Enterococcus species (which is intrinsically resistant to meropenem), does this constitute acquisition of resistance? If so, it would be misleading to determine this as an acquisition of resistance, as many people are colonized with Enterococci and selection of Enterococci under therapy is very common. If this is not considered as the acquisition of resistance please include how the acquisition of resistance is defined per included study.

Thank you for pointing out this potential ambiguity. Our definition of “acquisition of resistance” is agnostic to bacterial species and hence intrinsically resistant species can be included if they were only detected during the follow-up culture by the studies. We will clarify this in the definition of “acquisition of the resistance” in the manuscript (see l. 259-260). However, it was not always clear from the studies which pathogens were acquired or whether intrinsically resistant species were not reported. Therefore, we rely on the studies' specifications of resistant and non-resistant without further classifying data into intrinsic and non-intrinsic resistance. The outcome “acquisition of resistance” can be seen more of a risk assessment for having any resistant bacterium during or after treatment. In contrast, the outcome “emergence of resistance” is more rigorous, demanding the same species to be measured as more resistant during or after treatment.

Table S1 is not sufficiently clear because it often only contains how susceptibility testing was done but not which antibiotics were tested and how a strain was classified as resistant or susceptible.

In Table S1, we omitted the listing of antibiotics for which susceptibility testing was performed, as this information is already presented in the main text (Table 1). However, we agree that linking this information better in a future version would benefit the understanding. Given the variability in methods used to assess resistance and the variability in drugs, the comparability of breakpoints is limited. Hence, we decided not to provide further details on this aspect so far.

Line 85: "Even though within-patient antibiotic resistance development is rare, it may contribute to the emergence and spread of resistance."

Depending on the bug-drug combination, there is great variation in the propensity to develop within-patient antibiotic resistance. For example: within-patient development of ciprofloxacin resistance in Pseudomonas is fairly common while within-patient development of methicillin resistance in S. aureus is rare. Based on these differences, large clinical heterogeneity is expected and it is questionable where these studies should be pooled.

We agree that our formulation neglects differences in prevalence of within-host resistance emergence depending on bug-drug combinations. We will correct this in our upcoming version. (i.e. we will correct our statement to: “Within-patient antibiotic resistance development, even if rare, can contribute to the emergence and spread of resistance.”)

Line 114: "The overall pooled OR for acquisition of resistance comparing a lower number of antibiotics versus a higher one was 1.23 (95% CI 0.68 - 2.25), with substantial heterogeneity between studies (I2=77.4%)"

What consequential measures did the authors take after determining this high heterogeneity? Did they explore the source of this large heterogeneity? Considering this large heterogeneity, do the authors consider it appropriate to pool these studies?

Thank you for highlighting this lack of clarity. In our upcoming version, we will emphasize the sub-analyses conducted to explore heterogeneity (i.e., figure 3 and supplementary materials section 6). Nevertheless, these analyses faced limitations due to the scarcity of evidence and the data provided by the studies. Given the lack of appropriate evidence, it is hard to identify the source of heterogeneity. The decision to pool all studies was pre-specified in our previously published study protocol (PROSPERO CRD42020187257) and was motivated by the question whether there is a general effect of combination therapy on resistance development or identify factors that explain potential differences of the effect of combination therapy across bug-drug combinations.