Abstract
Bacteria inhibit and kill one another with a diverse array of compounds, including bacteriocins and antibiotics. These attacks are highly regulated, but we lack a clear understanding of the evolutionary logic underlying this regulation. Here, we combine a detailed dynamic model of bacterial competition with evolutionary game theory to study the rules of bacterial warfare. We model a large range of possible combat strategies based upon the molecular biology of bacterial regulatory networks. Our model predicts that regulated strategies, which use quorum sensing or stress responses to regulate toxin production, will readily evolve as they outcompete constitutive toxin production. Amongst regulated strategies, we show that a particularly successful strategy is to upregulate toxin production in response to an incoming competitor’s toxin, which can be achieved via stress responses that detect cell damage (competition sensing). Mirroring classical game theory, our work suggests a fundamental advantage to reciprocation. However, in contrast to classical results, we argue that reciprocation in bacteria serves not to promote peaceful outcomes but to enable efficient and effective attacks.
Introduction
Bacteria commonly live in dense and diverse communities where competition for space and nutrients can be intense (Kim et al., 2014; Hibbing et al., 2010). As a response to such ecology, bacteria have evolved a wide range of competitive traits (Granato et al., 2019), including contactdependent inhibition (Ghequire and De Mot, 2014; Hayes et al., 2010; Aoki et al., 2010), the type VI secretion system (Miyata et al., 2013; Russell et al., 2014; Ho et al., 2014; Hood et al., 2010), narrowspectrum bacteriocins, and broadspectrum antibiotics (Bérdy, 2005; Riley and Wertz, 2002; Chao and Levin, 1981), which can kill or inhibit other strains. These mechanisms are extremely widespread. Bacteriocidal toxins are found in almost all major bacterial lineages (Riley and Wertz, 2002; Granato et al., 2019) and single species commonly make use of multiple toxins and diverse means of attack (Granato et al., 2019; Be'er et al., 2010; Zhang et al., 2012; Steele et al., 2017; Jamet and Nassif, 2015). This toxinbased warfare is also important for bacterial ecology and evolution, with evidence that toxin production can prevent competing strains from invading a niche (SassoneCorsi et al., 2016; Nakatsuji et al., 2017; Zipperer et al., 2016; O’Sullivan et al., 2019), kill off coexisting strains (Granato et al., 2019; Majeed et al., 2011; Speare et al., 2018; Ma et al., 2014), or help strains to invade new niches (Kommineni et al., 2015; Wiener, 1996; Sana et al., 2016).
The production and regulation of bacterial toxins have been studied for decades because of their potential as clinical antibiotics (Lewis, 2013; Slattery et al., 2001). This work has revealed that toxin production is often tightly regulated (Miyata et al., 2013; Anderson et al., 2012; Ghazaryan et al., 2014; Bernard et al., 2010). Indeed, it is thought that there are many new antibiotics that remain undetected because they are only activated under certain conditions (Maldonado et al., 2003; Abrudan et al., 2015; Traxler et al., 2013). A major form of regulation in bacteria is quorum sensing (Fuqua et al., 1994; Navarro et al., 2008; Eickhoff and Bassler, 2018) whereby cells secrete a small molecule and respond to it dependent upon its concentration. Some antibiotics and bacteriocins are regulated by quorum sensing, which is thought to ensure that toxin production occurs at the right cell density (Hibbing et al., 2010; Chandler et al., 2012). Other factors also regulate bacterial toxin production, including particular nutrient conditions and diverse stress responses (Storz and Hengge, 2011). This led to the argument that, in addition to quorum sensing, bacteria engage in ‘competition sensing’ whereby they use nutrient stress and cell damage to detect ecological competition (Cornforth and Foster, 2013; Lories et al., 2020).
Bacteria, therefore, have the potential for a wide range of responses during combat. Evolutionary theory has so far focused on the evolution of unregulated toxin production. This work has highlighted that factors such as strain frequency, nutrient level, the level of strain mixing (relatedness), and the cost of toxin production are all important for whether bacteria employ toxins at all (Bucci et al., 2011; Gardner et al., 2004; Brown et al., 2006; Gordon and Riley, 1999; Frank, 1994; Levin, 1988). Other models have highlighted how natural selection for warfare can have consequences for the evolution of diversity (Frank, 1994; Biernaskie et al., 2013; Kelsic et al., 2015; McNally et al., 2017), including via rockpaperscissor dynamics between different genotypes (Czárán et al., 2002; Kerr et al., 2002). However, to understand the strategic potential of warring bacteria, we must consider the regulation of their toxins and other weapons (Granato et al., 2019; Cornforth and Foster, 2013).
Here, we study the evolution of strategy during bacterial warfare by combining a detailed differential equation model of toxinbased competition with game theory to identify the most evolutionarily successful strategies. Informed by the large empirical literature on factors that regulate bacteriocins and antibiotics, we compare four major classes of potential strategies: constitutive (unregulated) toxin production, and regulation via nutrient level, quorum sensing, or by damage from a competitor’s toxin. We study the behaviours and competitive success of each strategy when in competition with other strains across a range of scenarios. We find that all three types of regulated strategies carry benefits relative to nonregulated production and, for shortlived resources, the three types of regulation offer largely equivalent alternatives for controlling attacks. However, for longlived environments, responding to incoming attacks is often the best performing strategy. A key benefit to such reciprocation in such environments is the ability to downregulate a toxin once a competitor is defeated, thereby saving the energy that would be lost in needless aggression.
Results and discussion
Overview
We are interested in how competition between strains and species of bacteria shapes the evolution of toxin regulation. The core of our approach is a set of detailed ordinary differential equations (ODEs) that capture ecological competition between bacteria (Figure 1a), which are built upon an earlier model of bacterial siderophore production (Niehus et al., 2017). After exploring the case of constitutive toxin production only, we extend our model to incorporate different strategies of regulated production (Figure 1b). We use these differential equations to model ecological interactions of bacterial strains and determine the outcome of competition for a given strategy against another strategy when they meet locally.
These locallevel competitions are embedded into a larger metapopulation framework that determines longterm evolutionary outcomes (Maynard Smith, 1982; Figure 1c and d). This metapopulation modelling includes invasion analysis, in the tradition of the branch of evolutionary game theory developed by Maynard Smith and Price, 1973 and the field of adaptive dynamics (Materials and methods). We also later use a more explicit genetic algorithm that employs the same logic. This algorithm pits diverse strategies against each other across a large number of combinations in order to find the most successful strategies (Materials and methods). In these metapopulation models, bacterial strains are assumed to compete locally in a large number of patches, but also globally through dispersal to seed new, empty patches based on a standard life history of bacteria used in previous models (Oliveira et al., 2014; Gardner et al., 2007; Chuang et al., 2009; Cremer et al., 2012; Nadell et al., 2010; Figure 1c). Also, as discussed previously (Nadell et al., 2010), we refer to the global population as a metapopulation to distinguish it from the local bacterial cell population in each patch. This approach accounts for the possibility that a strategy can do well in local competition, but do poorly globally, and vice versa (Figure 1d). We show in later analysis that all competitive outcomes shown in Figure 1d occur in our simulations, with the first two cases being the most common.
We explore a number of different evolutionary scenarios using different calculations. We start by using invasion analysis (depicted in Figure 1d) to study the evolution of toxin producers that lack regulation and to study the evolution of toxin regulation from constitutive producers. This allows us to understand first when, and how much, a strain should invest into attacking other, and then, whether regulated production can evolutionarily replace constitutive production. We next compare different regulated strategies to one another by studying their performance when facing a diverse range of constitutively producing species. Finally, we study the case where regulated strategies compete with each other and coevolve in massive tournaments to identify the most globally successful strategies (see Materials and methods).
Our model needs to be relatively complex in order to capture the evolution of bacterial competition and regulatory networks. As a result, the form of our mathematical model is of a class that is not amenable to analytical work (Boys et al., 2008; Gutiérrez and Rosales, 1998; Liu and Chen, 2003). To confirm this, we investigated the behaviours of the dynamic model at steady state. This showed a good basic correspondence between our numerics and analytics but confirmed that the model is not amenable to further analytical work (see Appendix 1 Supplementary analytics). Nevertheless, by combining a number of different competition scenarios with wide parameter sweeps, we are able to show that our key conclusions are robust across many conditions.
Evolution of warfare via unregulated toxin production
We first ask, what favours the evolution of constitutive toxin production. While many toxins are regulated, constitutive production does occur (Mavridou et al., 2018), and we use the simple case of constitutive toxin production to first identify general principles underlying the evolution of bacterial warfare. In addition, constitutive production forms a baseline from which to compare the evolution of regulated strategies. In order to study the behaviours that result from each strategy, we use a detailed model of competition between strains based upon a system of differential equations (Materials and methods). This approach allows us to capture the temporal dynamics of strain interactions and, later, toxin regulation.
In the model, we follow nutrient concentration and cell biomass over time as the strains engage with each other (Figure 1a). We focus on competitions between two strains that each possess a toxin that does not harm the producer strain but does harm the other strain. In reality, strains may carry multiple toxins and resistances (Cordero et al., 2012; Gordon et al., 1998) and our framework can be extended to include such complexity. However, for simplicity, we focus here on a single toxin produced by each strain. We consider interactions that are pairwise at the strain level, but we later account for a multitude of competitors by letting strains have many encounters, each with a different strategy. To enable us to study a large number of strategies, our differential equations are based upon simplified wellmixed conditions.
Our goal is to understand the evolutionary fate of different strategies of toxinmediated competition. In order to do this, we need to recognise that the outcome of competition at a local scale may not be predictive of evolutionary trajectories. Consider, for example, a competition between two strains of bacteria on a particle of detritus in a pond. If one focuses solely on local competition on the particle, then any strategy that results in a focal strain making more cells than its competitor will be favoured, even if this leads to relative ruin for the winning strain with only a few cells surviving the process. However, given these competitions can happen on many such particles, it is unlikely that such extreme strategies would be favoured, because few cells will be produced to colonise new particles. Instead, the best strategies will be those that make the most cells to disperse, which may mean a strain also wins locally, but it may not (see Figure 1d).
To capture this effect, we embed the locallevel competitions within a broader framework in order to make evolutionary predictions (Maynard Smith, 1982; Weibull, 1997; Mitri et al., 2011) (see Figure 1c and d). This framework allows us to ask whether a particular, initially rare, strategy can successfully invade a metapopulation of another strategy (Materials and methods). Specifically, a rare mutant’s fitness in the metapopulation is defined by the number of cells it produces in direct competition with the resident, while the resident’s fitness is defined by its productivity when it meets another resident in a patch, as will occur in the vast majority of patches if the mutant is rare (Figure 1d, Materials and methods). For mutants that can invade, we also confirm that they then cannot be reinvaded by the previous resident (Materials and methods), which is indeed always the case here. We refer to such invasions that lead to a full replacement of the resident by the mutant – where the resident is unable to reinvade from rare  as a stable invasion. By studying large numbers of competitions, we can categorise strains by their ability to stably invade others, and thereby identify the evolutionarily stable investment into toxin production (f*). We then seek the optimal level of toxin production, which cannot be invaded by any mutant strategy, but can invade all others.
What determines the optimal level of toxin investment? Intuitively, we find that cells evolve to invest more in attacking their competitors when toxins are efficient at killing the competitor and/or the toxins persist stably in the environment (Table 1). Toxin efficiency in our model is equivalent to the relative cost of toxin production, that is, we see a high benefittocost ratio favours toxin use. This result is in line with previous theory, which has shown that the impact of toxin production on growth rate is critical for the evolutionary outcome (Bucci et al., 2011; Levin, 1988). For highly effective toxins, we find that strains will engage in an arms race that escalates to the point where populations can go extinct (Appendix 1—figure 4). Such ‘evolutionary suicide’ is known from a wide range of conflict scenarios in biology (Rankin and LópezSepulcre, 2005).
While earlier models have studied the effects of nutrients on toxin production, these studies either did not model nutrients explicitly (Frank, 1994), or the level of nutrient competition was coupled to the presence of and mixing with other strains (Bucci et al., 2011; Gardner et al., 2004). In our model we can isolate the effect of nutrients on the evolution of toxin use. When nutrients are scarce, there is not enough energy to produce effective amounts of toxins (Appendix 1—figure 3), which agrees with previous theory (Bucci et al., 2011; Gardner et al., 2004; Frank, 1994), and has also been shown experimentally in yeast (WlochSalamon et al., 2008). However, we also find that toxin benefit peaks at intermediate nutrient availability and decreases for higher nutrient levels (Appendix 1—figure 3). This can be understood in terms of a shift in the relative benefits of investing in cell division versus attack: When bacteria enter a competition at low density and resources are abundant, there is a great potential for population expansion. Under these conditions, cells evolve to invest relatively little in toxin production; energy is instead better invested in rapid growth to win a competition by outgrowing other strains. In contrast, when growth potential is limited, cells benefit from investing in warfare, unless, as mentioned above, nutrients are too scarce to produce an effective toxin concentration.
The evolution of regulated attack strategies
We next investigate what happens when cells are able to regulate their level of toxin production in response to environmental cues. The production of antibiotics and bacteriocins is commonly tightly regulated by a variety of signals and cues. As discussed above, these can be broadly divided into three major classes based upon known bacterial regulatory networks. The first is detection of cell density by canonical quorum sensing or related means (Fuqua et al., 1994; Eickhoff and Bassler, 2018), which has been demonstrated by previous modelling work to be beneficial for the regulation of cooperative traits (Cornforth et al., 2012). In addition, bacteria are highly responsive to both nutrient stress and cell damage associated stress (Storz and Hengge, 2011), which both can detect the level of ecological competition in the environment (‘competition sensing’; Cornforth and Foster, 2013; Lories et al., 2020).
We first compare the evolution of regulation by quorum, nutrient level, and the level of the competitor’s toxin when each is in competition with constitutive strains. This allows us to ask whether regulated strategies can evolutionarily replace constitutive strategies (see Materials and methods). In brief, we model regulation of toxin production using a simple step function, which is defined by toxin production in activated state (f_{induced}), production in inactivated state (f_{initial}), and a threshold of the signal for activation. All three parameters are continuous; toxin production (f_{initial} and f_{induced}) is constrained between 0 and 1, and the threshold is constrained to a region consistent with the observed range of each signal (quorum, nutrient, toxin level).
In a vast tournament consisting of millions of individual competitions, we pit all possible strategies of each mode of regulation against all possible versions of the fixed strategy. We then use invasion analysis, as before, to look for the evolution of regulated strategies that can invade all unregulated strategies. As before, we consider both global and local competition (see Figure 1d) to determine invading strategies that cannot be reinvaded by the previous resident strategy and that therefore cause stable invasion. We find that all possible outcomes of the metapopulation competitions (Figure 1d) do occur, with the typical case being that the outcome of local and global competition are the same (see Appendix 1—figure 6). In a small minority of cases (2.3%), we find that successful invading strategies can be reinvaded by the previous resident to give a mixed evolutionary outcome, and these cases are not considered further. Our analysis identifies versions of each mode of regulation that can stably invade all possible constitutive strategies (Figure 2a–c). This result is expected and confirms the basic intuition that – unless maintaining a regulatory circuit is very costly – a wellregulated trait will outcompete an unregulated one (Cornforth and Foster, 2013). This is true whether strains compete for a short or long duration, although shorter duration does select for a higher initial investment in toxin production in order to ensure that enough toxin is made in the time that a strain has to compete (Appendix 1—figure 5).
How do the different regulatory strategies achieve their success? For the great majority of cases, successful strains evolve to upregulate their attack after a delay, either based on the detection of low nutrients, high quorum, or high levels of the competitor’s toxin (Figure 2d–f). In some cases, there is no toxin production before this upregulation, as in the canonical model of quorum sensing that turns a trait from off to on. In other cases, the strategy that evolves is to begin with a baseline of constitutive production before upregulating this further upon activation (Figure 2a–c, with example shown in Figure 2f), something also seen in real systems (Mavridou et al., 2018). A difference between nutrient and quorum sensing versus toxinbased regulation is that examples of the latter not only upregulate toxin production after a delay, they also downregulate the toxin if the competitor is killed off (Figure 2e).
We also discovered winning strategies that function by downregulating toxin production after a delay. For nutrientbased regulation, there is a narrow parameter range (the small vertical strip in the lower part of Figure 2a) where strategies begin aggressively with the expression of toxin and then downregulate it when nutrients are limited (dynamics not shown). For quorumsensing strategies, some also start with high toxin investment, but these strategies are more complex. These downregulate toxins and invest in growth once they reach a high density, but will reactivate it again if their cell numbers drop due to toxin attack (Figure 2g).
In each case, regulated strategies win by only making high levels of toxin at certain times, thereby saving energy relative to constitutive producers. A corollary is that, if toxin production is cost free, regulation will no longer be benefitical relative to constitutive production. But, assuming that there is some costs to toxin production, regulation is expected to be favoured by natural selection.
In sum, there are regulated strategies of each of the three types under study that can evolutionarily replace all nonregulated strategies. However, this analysis is based on regulated strains invading metapopulations consisting of a single constitutive strategy. In some contexts, a focal strain may face a variety of competitors. Consider, for example, a situation where migration brings in a range of competiting species, each optimised to a different environment. To consider this scenario, we next ask how the different sensing strategies fare in competition with a standing diversity of constitutive strategies. We introduce diversity by letting the different sensory strategies (i.e. nutrient sensing, toxin sensing, and quorum sensing) face an increasingly diverse mix of constitutive toxinproducing opponents. We assume that the standing diversity of constitutive producers is not itself affected by the evolution of the regulated startegies, that is, there is no coevolution (we consider coevolution in the next section, however). For each set of opponents, therefore, we can identify the best performing regulated strategies simply as those that obtain the highest average biomass across the competitions with the set of opponents (Materials and methods). Based on the simulated data, we also fitted a linear regression model with sensing type as a categorical predictor and number of competitors a numerical predictor (see Materials and methods).
When opponents have a single strategy (lowest diversity), the toxin sensing strategy is most efficient in terms of its final biomass produced (Figure 3a, left panel). Moreover, the toxin sensing strategy deals most effectively with diverse competitors (Figure 3a) with the regression analysis showing a 2.5 times higher fitness for toxin sensing relative to the other strategies (pvalue < 0.001). The success of the toxin sensing strategy is associated with the reliable activation of toxin production when sensing another toxin. Quorum sensing also activates toxin production during the competition but, in some cases, is defeated without being able to attack back. This gives rise to the observed bimodal outcome of the quorumsensing strategy (Figure 3a). The nutrientsensing strategy, by contrast, attacks first and then deactivates later when nutrients decrease.
This superiority of toxin sensing is robust across a range of parameters, including different toxin efficiencies, toxin loss rates, and nutrient concentrations (Appendix 1—figure 7). There is a clear post hoc intuition to this result. A strain that only engages in conflict when attacked will be best able to deal with a range of strategies that differ in their propensity and ability to attack. More specifically, as seen in the last section, these strains inactivate toxin production after a weak opponent is eliminated, thus employing the toxin efficiently. We can directly demonstrate the importance of this tactic of toxin inactivation by shortening the duration of the strain competitions such that toxinsensing strains do not have the opportunity to downregulate toxin production. For short competition times, while regulated strategies still outperform unregulated ones (Appendix 1—figure 5), the toxin sensing strategy fails to evolve a superior performance over the other modes of regulation (Figure 3b).
The coevolution of regulated attack strategies
We have considered how regulated attack strategies perform in the face of constitutive strategies that vary in their level of aggression, and in the face of varying levels of diversity in these opponents. This revealed that regulation is generally beneficial and indicated that the sensing of an opponent’s toxin is often the best performing strategy. However, this analysis is artificial in the sense that bacteria with regulated strategies are also likely to compete against one another. Therefore, we next ask, which sensing strategy is most successful when coevolving with other sensing strategies? We first consider strains that interact with others that have a similar attack strategy, regulated by the same environmental cue. For each of the three types of regulation, we then search for the optimal strategy using a genetic algorithm (see Materials and methods) (Figure 4a). Following the logic of the earlier models, the optimal strategy is defined as one that will, on average, obtain the highest biomass across competitions with all other possible strategies.
When competing with the same strategy, all strategies initially evolve to increase toxin production during the competition (f_{initial} < f_{induced}) (Figure 4b). More specifically, strains responding to nutrient depletion initially produce near zero toxins (f_{initial} = 0.05) until they activate toxin investment, at a level higher than the optimal fixed investment (evolved f_{induced} = 0.50, while f* = 0.35). In comparison, strains responding to quorum sensing invest in more toxin initially (f_{initial} = 0.11) and also more when activated (f_{induced} = 0.60). The quorumsensing strategy is expected to be able to afford to invest more in toxin production because, unlike nutrient sensing, strains can reduce toxin investment again if biomass drops too low, thereby saving energy. The toxinsensing strategy is different again. It invests near zero toxin at the start of the competition (f_{initial} = 0.01) and responds very strongly if a competitor attacks (f_{induced} = 0.73). Interestingly, the corollary is that, at evolutionary equilbirum (when it will meet an identical toxin strategy), both remain passive and achieve a high biomass (Figure 4b center). This outcome has similarities to the success of ‘titfortat’, a reciprocal cooperating strategy in the classic evolutionary game theory tournament of Axelrod and Hamilton, 1981. There, titfortat succeeds by benefiting from mutual cooperation whenever others cooperate, while maintaining the ability to shut off cooperation whenever it meets a noncooperative strategy. When this success leads to all individuals playing titfortat, the result can be that all interactions end up as cooperative, akin to the emergence of a peaceful productive strategy in our model.
The evolution of a peaceful outcome is specific to the ability to reciprocate; we do not observe it for nutrient or quorum sensing. Nevertheless, we have identified a route by which bacteria might evolve the peaceful resolutions seen in animal and human conflicts (Axelrod and Hamilton, 1981; Kokko, 2013; Freedman, 1989). However, the model assumes that strains will only interact with other genotypes that are adopting similar strategies for warfare. This is far from guaranteed in bacteria as there exists considerable variability in weapons and their regulation, even within a single species (Mavridou et al., 2018). Moreover, microbial communities typically contain many strains and species, suggesting again that a given strain has the potential to meet a diversity of competitors and strategies.
We therefore sought to capture this complexity with a final model in which all possible regulated strategies are able to compete against each other, again using a genetic algorithm to identify optimal strategies (see Materials and methods). Despite a great number of potential combinations (over two million different competitions), and with different sets of hyperparameters of the genetic algorithm, we again see a clear winner in toxinbased regulation, both for our normal parameters (Table 1, Figure 4c) and for sweeps that consider broad ranges of these parameters (Appendix 1—figure 8) and a wide range of initial frequencies of the two strains (Appendix 1—figure 9). Moreover, as for competition against unregulated strategies (Figure 3), the success of toxinbased regulation in contests with other strategies does not come from an ability to avoid conflict and create peaceful outcomes. Instead, the winning strategies are typically aggressive when they meet another strain and they only downregulate their toxins once an opponent is on its way to being eliminated (Figure 4d–f). And, as for competition against unregulated strategies, this ability to become passive is key to their success. For short competitions, there is no benefit in turning off an attack and the competitive benefit of reciprocity over other regulated strategies is lost (Appendix 1—figure 10).
Conclusions
Bacteria use a wide variety of weaponry to harm other strains and species, which is typically under tight regulation (Ghequire and De Mot, 2014; Granato et al., 2019; Stein, 2005; MichelBriand and Baysse, 2002; Cascales et al., 2007). How bacteria employ these mechanisms of attack is central to understanding why a particular species or pathogen can invade and persist in communities, while others cannot (Granato et al., 2019; Kommineni et al., 2015). Here, we have explored the evolutionary logic underlying strategies of bacterial attack. We find that toxin production is favoured under many conditions, particularly when toxins are effective and longlasting and when the potential for population expansion is limited (Table 1). The prevalence of aggressive strategies in our model is consistent with the widespread use of toxins by bacteria (Granato et al., 2019), and the associated intensity of competition observed in experiments (Chao and Levin, 1981; Mavridou et al., 2018; Oliveira et al., 2015). We also find that wellregulated attacks can consistently outcompete strategies that lack regulation (Figure 2). This is because the benefit of employing a toxin not only changes with different competitors but also within a single competition over time. Regulation allows a strain to better tune its behaviour and follow the optimal investment at any given situation. However, the three major classes of bacterial regulatory network are not always equivalent ways to control attacks. Across a diverse range of potential competitors, responding directly to incoming attacks is the most robustly successful strategy (Figures 3,4).
Our modelling implicitly captures spatial structure at the metapopulation level with discrete patches of bacteria that compete with each other. Within patches, our ODE model best reflects environments with limited spatial structure where cells of different genotypes are mixed together. However, bacteria do also display fine scale spatiogenetic structuring within their communities (Nadell et al., 2016; Stacy et al., 2016; Krishna Kumar et al., 2021). Here, our model has the potential to capture the outcome of competition at the interface of two strains, which is expected to be critical for success and persistence in such communities (Granato et al., 2019). However, there is clear potential for other effects of local spatial structure on sensing strategies that we do not capture. For example, in contrast to the detection of competitor’s toxins, responses to quorum sensing and nutrient depletion may occur first in the middle of a patch of cells, where toxin production has the least benefit as toxin receivers are mainly clonemates (Inglis et al., 2009; Wechsler et al., 2019).
Our work predicts that sensing incoming attacks through direct or indirect means should be a widespread way of regulating toxins and other modes of attack. This hypothesis lends itself to empirical testing via the study of bacterial behaviour during toxinmediated competition with other strains and species. Some examples of reciprocation already exist. Many bacteria upregulate attack mechanisms via stress responses that detect cell damage (Cornforth and Foster, 2013). This includes recent evidence of reciprocation between warring Escherichia coli strains where DNase protein toxins activate toxin production in competing strains via the SOS response to DNA damage (Mavridou et al., 2018; Krishna Kumar et al., 2021; Gonzalez et al., 2018; Granato et al., 2019). Because many antimicrobials target the DNA of cells (Janion, 2008; Gillor et al., 2008), sensing DNA damage is likely to be a relatively robust way to achieve reciprocity. But there are other mechanisms; Pseudomonas aeruginosa senses incoming attacks via the type six secretion system (T6SS) of competitors, which delivers toxin via the molecular equivalent of a speargun (Basler and Mekalanos, 2012; Basler et al., 2013). Upon detecting an incoming attack, a cell will activate its own T6SS in response (Basler et al., 2013). Consistent with our findings, recent work suggests that a key benefit to reciprocation via the T6SS is the ability to save energy and only attack when necessary, alongside a benefit that comes from improved aiming which is specific to this mode of attack (Smith et al., 2020). Finally, there is evidence that bacteria may also detect and respond to incoming attacks via proxies such as the detection of lysate produced when surrounding cells are killed (LeRoux et al., 2015a), or molecules that are made by an attacker alongside a toxin (Cornforth and Foster, 2013; LeRoux et al., 2015b).
There is also evidence that bacterial toxins can be regulated via nutrient depletion and quorum sensing (Ghequire and De Mot, 2014; Cascales et al., 2007). Our models of regulation by quorum or nutrients typically predict that attacks will evolve to be activated at high quorum or limited nutrients, which recapitulates the typical directionality of the regulation observed in nature (Chandler et al., 2012; Fontaine et al., 2007; Inaoka et al., 2003). However, if detecting damage is the best basis for attack, why do some bacteria use these other forms of regulation? For short competition times, our model predicts that the three regulatory strategies are largely equivalent (Figure 3 and Appendix 1—figure 10). A short duration of competition between strains removes the benefit of decreasing toxin production once an attacker has been overcome. Under these conditions, the evolutionary path to one form of regulation may largely be determined by differences in costs for regulatory networks and which preexisting regulatory systems are available for cooption (Cotter and DiRita, 2000; Hockett et al., 2015). We predict, therefore, that mechanisms to reciprocate attacks are particularly valuable in environments where warfare commonly leaves a victor unchallenged for a long time afterwards. Consistent with this, one of the clearest examples of reciprocation occurs in E. coli (Mavridou et al., 2018; Krishna Kumar et al., 2021; Granato et al., 2019), which uses colicin toxins to displace other strains and persists for long periods within the mammalian microbiome (Gillor et al., 2009).
Another possible explanation for why some bacteria do not use cell damage to regulate their toxins comes from the notion of ‘silent’ toxins. These are toxins that are not easily detected by the cell’s stress responses, which may limit the potential for a toxinmediated response. For example, some toxins depolarise membranes (Yang and Konisky, 1984) and may be favoured by natural selection specifically because they do not provoke dangerous reciprocation in competitors (Gonzalez et al., 2018). In other cases, bacteria appear to use multiple forms of regulation in order to integrate information from multiple sources (Cornforth and Foster, 2013). For example, Streptomyces coelicolor regulates antibiotic production via both nutrient limitation (Hesketh et al., 2007) and mechanisms that detect incoming antibiotics (envelope stress [Hesketh et al., 2011]). A potential future use of our modelling framework would be to study how these combined regulatory strategies evolve.
Bacteria use diverse regulatory networks to attack and overcome competitors, and there is much still to understand about their evolution. Here, we have identified general principles for the function of these networks in bacterial warfare. We find there are great benefits using regulation to time an attack; both to minimise its cost and maximise its effect on an opponent. We also find that regulation that enables reciprocation can be particularly beneficial. If cells only attack when attacked, they invest their energy where and when it is most needed: against aggressive opponents. Our findings are mirrored in the classical predictions from the game theory of animal combat, which suggested that adopting a reciprocal and retaliatory strategy can be effective (Maynard Smith and Price, 1973; Kokko, 2013; Freedman, 1989; Enquist and Leimar, 1990). However, the predicted outcome was typically one of peace and the avoidance of conflict, which is indeed what is observed in many animal contests (Briffa M, 2013). In contrast to such lessons, experimental work suggests that bacteria often engage in deadly conflict (Abrudan et al., 2015; Mavridou et al., 2018; Oliveira et al., 2015; Gonzalez et al., 2018; Be'er et al., 2009; Vetsigian et al., 2011). Our models offer an evolutionary rational for this observation. The regulation of combat in bacteria is not usually about avoiding conflict; it is about timing an attack and downregulating it once a competitor is no longer a threat.
Materials and methods
Overview
In this study we use a modelling framework that captures two scales of competition (Figure 1). At the local level, we model bacterial strain competitions using systems of ODEs. These equations are well suited to model temporal dynamics on the relatively short ecological timescales at which bacterial strains interact with nutrients and competitors. At the global level, we model the evolution of different strategies within a metapopulation. This metapopulation level allows us to follow the evolution of different strategies across much longer evolutionary timescales, and to capture the important interplay of local and global fitness (Figure 1d). We use this game theory framework to identify strategies that are evolutionarily successful against a diversity of possible competitors. All questions addressed in this work require both layers of modelling. The system of ODEs that models constitutive toxin production is described in the next section and forms the basis for all of the models. Evolution at the metapopulation level is implemented using a common logic (Figure 1c,d), using variations that capture a range of questions and evolutionary scenarios as detailed below.
A differential equation model of bacterial warfare
Request a detailed protocolOur model captures pairwise competitions between bacterial strains, which have the potential to produce toxins (Figure 1a). This first model allows a strain to have a fixed investment into its toxin – below we describe the extension of this model that allows toxin regulation in response to external cues. We employ ODEs, which are well suited to capture the temporal dynamics of strain interactions happening at ecological timescales. A number of different models have been used to study the evolution of bacterial public good regulation (Niehus et al., 2017; Heilmann et al., 2015; Kümmerli and Brown, 2010). Here, we follow Bucci et al., 2011, because they model both nutrients and toxins explicitly, which are both important cues for the regulation of toxin production. We study a competition between two strains that each possess a toxin that does not harm the cells of the producer strain, but does harm the other strain. In reality, strains may carry multiple toxins and resistances (Cordero et al., 2012; Gordon et al., 1998) and the evolution of multiple mechanisms of attack and defence is an interesting question in its own right. However, we focus here on a single toxin produced by each strain. We also describe the dynamics of the nutrients and cell densities in a wellmixed environment. The interactions of cells, nutrients, and toxins can be described by the system of ODEs:
where C_{A}(t) and C_{B}(t) denote the biomasses of cell strains A and B, respectively, T_{A}(t) and T_{B}(t) denote the biomass of each strain’s toxin, and N(t) denotes the concentration of a growthlimiting nutrient for which both strains compete. We consider a pool of nutrient that is depleted by the cells. Similarly to Nadell et al., 2008, we describe the energy that is available to the cells by the Monod equation, in which K_{N} is the nutrient saturation constant. The maximum growth rate is given by µ_{max}. Toxins kill with efficiency k and are lost with rate l_{T}. We assume that all toxins have identical loss and killing rates in order to remove biochemical differences between strains and focus our analysis on the effects of different production strategies.
For constitutive toxin production, the strategy of a strain is given simply by a fixed f (f ∈ [0,1]), which captures the investment into toxin production relative to cell biomass. The production of antibiotics and bacteriocins can have significant metabolic costs and can even require a cell to lyse, as occurs with colicins and pyocins (Cascales et al., 2007; Nakayama et al., 2000). We model the cost of toxin production on cellular growth as a linear allocative tradeoff function in the growth term (Bucci et al., 2011). For example, a strain that invests f = 0.1 into its toxin will only reach 90% of its maximal growth rate.
The dynamics of cells, nutrients, and toxins are modelled as continuous for their typical range. But when a cell strain reaches a very low concentration (C(t)=10^{−6}), we assume that stochastic extinction occurs such that cell concentration drops to 0. Further, our model assumes a limited lifetime of the local patches by stopping the dynamics when 24 hr (or less for the analysis of shortened competition times, Appendix 1—figure 5) have passed.
We solve the system of ODEs numerically using an implicit Euler method. This numerical scheme is implemented in MATLAB (version 9.5.0.944444) (MATLAB, 2018). Our implementation solves the equations (Equation (1)) until the defined end time. We avoid numerical issues due to negative state variables by setting any state variables reaching a value below 10^{−8} to 0.
A model of regulated toxin attack
Request a detailed protocolTo extend the above model to include sensing, toxin production of bacterial strain A is either a function of nutrient depletion, toxin of strain B, or of quorum sensing (given as cell biomass of strain A). Each signal triggers toxin production via a simple onandoff switch (Cornforth and Foster, 2013) so that the toxin production of strain A is given through one of the equations:
where H is the Heaviside step function given as
and where ${f}_{initial}\in \left[\mathrm{0,1}\right]$ and ${f}_{induced}\in \left[\mathrm{0,1}\right]$.
These equations of regulated toxin production each comprise the initial investment into toxins (f_{initial}) when the trigger term is deactivated and the trigger term itself. The trigger term contains a Heaviside step function and becomes active when the signal increases over the sensing threshold (U_{N}/U_{TB}/U_{QS}). When activated, the trigger term changes the initial toxin investment (f_{initial}) to become the induced toxin investment (f_{induced}). We allow the induced toxin investment to be smaller (when the signal is a repressor) or larger than the initial toxin investment (when the signal is an activator).
Invasion analysis
Request a detailed protocolWe use our first models to predict the optimal constitutive toxin production strategy across different ecological conditions. Here, the assumed scenario is a monomorphic metapopulation (all strains have identical warfare strategy), where a rare mutant strategy appears that may or may not invade this metapopulation. As time progresses toward infinity, the metapopulation will finally be dominated by a strategy that can invade the metapopulation of any other strategy and that can itself not be invaded. We implement this scenario using classic pairwise invasion analysis. More specifically, we employ game theory and, in particular, invasion analysis to find the best strategies (Nowak and Sigmund, 2004; McElreath and Boyd, 2013), where the best strategy is one that, if adopted by the whole population, cannot be invaded by any other strategy. These strategies are also called evolutionarily stable strategies (Maynard Smith, 1982).
We follow previous work (Oliveira et al., 2014; Cremer et al., 2012) by assuming a microbial life cycle that consists of a seeding step where local patches are seeded with two competing strains, a competition step where strains grow and interact according to the differential equations explained above, and a mixing step where cells from all patches disperse and mix, leading to a new seeding episode (Figure 1c). The proportion of the different strains (or strategies) that are seeded is determined by the strain frequencies after the competition step. Without explicitly modelling this life cycle, invasion analysis (McElreath and Boyd, 2013) asks whether a particular strain with strategy f_{inv} when rare can invade a population dominated by another strategy f_{res} (the resident Weibull, 1997; Figure 1d). To answer this, we calculate the fitness of the resident strategy (w_{res}) and the fitness of the invading strategy (w_{inv}). The fitness of the resident is its final biomass when in competition with an identical strategy so that w_{res} = w(f_{res}f_{res}) and the fitness of the rare invader is determined by its final biomass in the competition between invader and resident strategy, w_{inv} = w(f_{inv}f_{res}). We then calculate the invasion index for an invading strategy according to Mitri et al., 2011 as
When the invasion index I_{inv} is larger than 1, the rare strategy can invade the resident strategy; when the index is smaller than 1, the rare strategy cannot invade, and it disappears. Finally, we also test for backinvasion and compute I_{inv} for when the resident is rare and the mutant is the resident. We implement strain competitions by solving the system of ODEs described above. We define evoluationarily stable strategies as those strategies that have an I_{inv} larger than 1 against all studied competitors (and both as rare and resident strategy). By calculating the invasion index for a large number of invading strategyresident strategy pairs, we obtain a pairwise invasibility plot (Brännström et al., 2013) (insets in Appendix 1—figure 4). Using this plot, we find a single evolutionarily stable strategy f* that can invade all strategies and that cannot be invaded by any other strategy. We determine this globally optimal strategy using the algorithm outlined in the Appendix 1—code 1. We can then ask how the parameters of the model affect the evolution of toxin investment (Table 1).
Invasion analysis of sensing strategies
Request a detailed protocolWe next ask whether regulated strategies will evolutionarily replace constitutive production. Here, the ecological scenario is the same as above: monomorphic populations of constitutive toxin production strategies are threatened to be invaded by rare strategies that can sense (Figure 1d). We perform a parameter grid search that tests a large number of sensing strategies (stepping: ∆f_{initial}/∆f_{induced} = 0.02 and ∆U = 0.002, constraints: ${f}_{initial}\in \left[0,1\right]$, ${f}_{induced}\u03f5\left[0,1\right]$, ${U}_{N}\in \left[0,1\right]$, ${U}_{TB}\in \left[0,20\right]$, ${U}_{QS}\in \left[0,20\right]$) against the range of constitutive strategies. For the constitutive strategies, we select from a fine grid spacing that also includes the optimal constitutive strategy (f_{fixed} = [0.00, 0.01, 0.02, …, 1.00]). For each pair of sensing and nonsensing strategies, we compute the invasion index once for the sensing strategy as the resident and again for the nonsensing strategy being the resident. We then search for those sensing strategies that can invade all nonsensing strategies and that themselves cannot be invaded by any other nonsensing strategy. We show where those strategies lie in the parameter space of ${f}_{initial}$ and ${f}_{induced}$ (Figure 2ac).
Sensing strategies against standing diversity
Request a detailed protocolWe also study the evolutionary success of the three different types of sensing when being in constant competition with a diverse set of competitors. Here, the ecological scenario is a polymorphic metapopulation – a mix of different constitutive production strategies – with a given diversity. We assume that this diverse set of strategies is not influenced by evolution in the focal sensing strategy due to, for example, immigration that continually resupplies the diversity of competitors. We then ask what happens when a rare sensing strain enters this metapopulation, where its success depends on its success across pairwise competitions with the different resident strategies.
We implement this by competing focal sensing strategies against a set of different constitutive strategies and computing their fitness from the average biomass produced across those competitions. Specifically, for each of the three different sensing types, we perform a parameter grid search, creating a large number of predefined strategies across the parameter range of f_{initial} (∈[0,1], at increments of 0.05), f_{induced} (∈[0,1], 0.05) and respective thresholds U_{N} (∈[0,1], 0.02), U_{TB} (∈[0.001,4], 0.0005), and U_{QS} (∈[0.01,1.2], 0.01). Each of those sensing strategies is competed against a fixed set of constitutive strategies, one at a time, by solving the above system of equations. We then compute for each sensing strategy the average fitness across its competitions. Within each of the three sensing types we find the single strategy with the highest average fitness. For those winners we show the average fitness as bars in Figure 3 together with the fitnesses obtained against each individual constitutive strategy. We repeat this entire procedure for five different levels of diversity among the constitutive strategies. Starting with the lowest diversity set, which contains only a single constitutive strategy (f = 0.5), we then add increasingly extreme strategies, yielding three competitors (f = 0.4,0.5,0.6), five competitors (f = 0.3,0.4,0.5,0.6,0.7), seven competitors (f = 0.2,0.3,0.4,0.5,0.6,0.7,0.8), and finally nine competitors (f = 0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9).
Using the simulated results, we fit linear regression model with the sensing type as a categorical predictor variable and the number of competitors as a numerical predictor variable. The regression takes the form
where F_{i} is the fitness of the ith competition, which we assume to be normally distributed around a mean given by a linear equation and with standard deviation σ. The fixed intercept is given through α_{S[i]}, where S[i] is the ith element of integer vector S that contains only two possible values indicating whether the toxin sensor or a different sensor is in the competition i. D[i] is the ith element of integer vector D, which gives the number of competitors in the ith competition. Finally, β gives the change of fitness when adding one competitor. We fit the regression model using R (version 3.6.1) (R Development Core Team, 2011).
Genetic algorithm
Request a detailed protocolFinally, we study which sensing strategies are most successful in competition with other sensing strategies. We study this both within each type of sensing and across all three different types. Here, the scenario is a polymorphic metapopulation of coevolving sensing strategies. Mutation and migration create new strategies inside this population. A strategy’s achieved biomass in pairwise competitions with other strains determines its ability to stay and amplify in the metapopulation. The model initially studies a wide variety of strategies competing with one another. However, as time passes, the metapopulation converges and consists of increasingly optimal strategies. As this happens, the analysis then approximates the invasion analyses described above, where most strains are largely identical and rare mutants are pitted against this majority in the metapopulation (Figure 1d).
Specifically, we use a genetic algorithm to search for the evolutionarily stable strategy in the large space of possible strategies of a single type of regulation (and also in the space of all possible regulating strategies). This algorithm adapts the typical structure of a genetic algorithm (Melanie, 1996) where in each round a population of individuals is first tested to evaluate fitness and it is then replaced by a new daughter generation. Individuals of this new generation are created by a mix of cloning and mutating individuals from the previous parent generation selected based on their fitness and by addition of novel random strategies. As is typical in nonadaptive algorithms , the control parameters of the algorithm (e.g. number of generations, number of strategies in the population, rate of mutation, etc.) are chosen to achieve short simulation times and good convergence behaviour as determined by visually inspecting the distribution of population parameters over time (Melanie, 1996). Our population of competing strategies has a constant size of n=60. Initially a set of random strategies is created, whereby the three parameters that define an individual sensing strategy are drawn from a uniform distribution with given parameter constraints (${f}_{initial}\in \left[0,1\right]$, ${f}_{induced}\in \left[0,1\right]$, ${U}_{N}\in \left[0,1\right]$, ${U}_{TB}\in \left[0,4\right]$, ${U}_{QS}\in \left[0,1.2\right]$). The constraints for the sensing thresholds take the range of the respective signals as they are observed across the large number of competitions performed in the invasion analysis of sensing strains described above. (For the initial population in the case where all three sensing types compete, the sensing type is chosen at random with equal probability for all three types and, to avoid long run times and artefactual superiority due to parameter constraints, initial parameter values start at the optimum from the within strategy competition.)
In every round then, each strategy competes against all n strategies, including its own type. The final biomass of every strategy is summed across its competitions to give its competitive fitness. Then, a new daughter generation is generated. The four most competitive parent strategies are chosen to move into the next generation without parameter mutation, 36 strategies are drawn from the parent generation with probability proportional to their fitness and one of their parameters is mutated by adding a value drawn from a normal distribution with mean of 0 and standard deviation of 0.001. If, after mutation, a daughter strategy violates the parameter constraints, the random draw gets repeated until the constraints are met. Finally, 10 immigrant strategies are generated by choosing their sensing parameters as random draws from a uniform distribution within the constraints. (In the case of all three strategies competing, the sensing type is first drawn at random with equal probability, and then the sensing parameters are drawn at random.) For the competition between types of a single sensing strategies, the algorithm is run for 100 generation. (For the tournament with all three strategies, we ran the first 20 generations without selection, where we replace the population each generation with migrants, to allow comparison with the case where selection occurs, Figure 4c). The evolving parameter values for the top four strategies are averaged for each generation and saved (Figure 4a). The averaged values in the last timestep give the evolutionary stable strategy for each tournament (Figure 4b). In our sensitivity analysis, we also examine the results of the genetic algorithm with alternative sets of control parameters, including a smaller and a larger size of the mutation standard deviation (0.01 and 0.0005), a smaller and larger proportion of ‘migrating’ strategies in each generation (5 of 60, and 20 of 60), and five different sizes of the population of strategies (50, 70, 80, 90, 100). This yields a total of 20 alternative parameter combinations.
Code availability
Request a detailed protocolThe MATLAB code for the regulated toxin model, the invasion analysis, and the evolutionary tournament is available on GitHub (https://github.com/reneniehus/bact_warfare, copy archived at swh:1:rev:923e104aa634230547ba464c6bc8fee07f662ffa, Niehus, 2021).
Appendix 1
Appendix 1—code 1: Pseudo code to find the evolutionarily stable strategy of fixed toxin production.
Initialization: Find the initial resident fresini (for example, start with the highest average resident biomass)
forward = 1 indicates if the next migrant will have a higher or lower toxin investment than the resident. Can be 1 or 1.
previousforward = 1 record the direction of strategy change from the previous iteration
previousfres = 0 record the resident from the previous iteration
step = 0.1 initial step for the strategy change (initially coarse to localize the ESS)
minstep = 0.0001 the precision we want for the strategy value
singular = 0 boolean saying if a singular strategy is localised
nbflip = 0 record the number of consecutive flips in direction
fres = fresini
newres = 0 boolean saying if there is a new resident
while (!singular) {do this while no singular strategy is localised
if (newres) {
res vs N res competition. Compute the resident against the resident. Record the resident average strategy: wresav.
}
fmut = fres + forward*step pick an invader that differs from resident according to direction
mut vs N res competition. Compute mutant vs resident competition. Record the mutant biomass (wmut) at the end of the competition.
previousforward = forward update the previous direction
previousfres = fres update the previous resident before the competition
if (wmut < wresav) {resident stays the resident.
forward =  forward change the direction, to test a migrant with lower f value at the next step
}
if (wmut > wresav) {the mutant invades and replaces the resident
fres = fmut the new resident will take the migrant value
newres = 1 we have a new resident, we'll have to compute its resident fitness at the next iteration
}
if (forward != previousforward) {compare the direction with previous direction
nbflip = nbflip + 1
}
if (forward == previousforward) {compare the direction with previous direction
nbflip = 0
}
this will allow to localise an ESS. We want fres to stay identical for 2 consecutive time steps, but we want to make sure that the higher and lower strategies have been checked. Indeed if the mutant that we just tested goes extinct, the resident will stay resident but that is not enough to say it is an ESS. We also want the number of flips in direction to be > 1.
if (previousfres == fres & nbflip > 1) {singular strategy localised
if (step <= minstep) {precision is high enough
singular = 1 we have found the optimal strategy
}
if (step > minstep) {will redo the whole procedure with tinier step
step = step/10 increase precision 10 times
nbflip = 0 reset the number of consecutive flips
}
}
}
Record the final fres (ESS).
Finally, we test whether this fres (ESS) is a GLOBAL optimum by competing it against the range of strategies given through f = [0, 0.01, 0.02, ….1].
Supplementary analytics
For the system of ODEs presented in Equation (1), the system state could be rewritten in the vectorised form $\mathbf{X}={\left({C}_{A}\left(t\right),{C}_{B}\left(t\right),{T}_{A}\left(t\right),{T}_{B}\left(t\right),N\left(t\right)\right)}^{\mathrm{T}}$ with the system dynamics denoted as
We denote a specific equilibrium state by ${\mathbf{X}}^{*}$. In order to analyse the associated linearly asymptotical stability of the above system at ${\mathbf{X}}^{*}$, we should first find solutions safisfying ${\mathbf{F}(\mathbf{X}}^{*},{f}_{A}^{*},{f}_{B}^{*},k,{l}_{T})=0.$ We note, that in Equation (1), the dynamics of N is defined by a negative derivative, and from this derivative we can see that a stable state regarding N can only be reached when there are no cells or no nutrients. However, no cells is a trivial and extreme state (i.e. no cells are left for further seeding), and no nutrients cannot be reached within finite durations. We will therefore abandon the dyanmics of N from the system, basically assuming chemostat environment where different levels of N can be acchieved through balancing consumption and influx of N. This changes of course how cell strains interact, but it will help to show in a simpler way how analytical methods fail even for this simplified system of equations.
Now, setting ${\mathbf{F}(\mathbf{X}}^{*},{f}_{A}^{*},{f}_{B}^{*},k,{l}_{T})=0$ further gives
From Equation (S2), we have
From Equation (S3), we have
From Equation (S5), we have
From Equation (S4), we have
Thus, we know from Equations (S8) and (S9) that
Similarly from Equations (S6) and (S7), we get
Therefore, from Equations (S10) and (S11), we have
From Equations (S6) and (S8), we know that
From Equations (S12) and (S13), we know
and note that here when ${f}_{A}^{*}=1$, we obtain ${f}_{B}^{*}=0$, which is not applicable.
Substituting Equation (S14) into Equation (S9), we then know
Similarly, we further have
and
And the Jacobian matrix of the system state ${\mathbf{X}}^{*}$ is
By computing the maximum real part of all the eigenvalues of $\mathbf{A}$ (denoted by $\mathrm{Re}\left(\lambda \right)$), we know that ${X}^{*}$ is linearly asymptotical stable when $\mathrm{Re}\left(\lambda \right)<0$. This canonical measure of stability indicates whether and how fast the system would return to ${X}^{*}$ after a small perturbation (Hofbauer and Sigmund, 1998; Mougi and Kondoh, 2012; Coyte et al., 2015; Allesina and Tang, 2012).
Next, we calculate the derivative of the system for different values of ${f}_{A}$ and ${f}_{B}$, choosing values for $k$ and ${l}_{T}$ to be as given in Table 1, and setting ${C}_{A}^{*}\left(t\right)$ and ${C}_{B}^{*}\left(t\right)$ to be 0.1, and ${T}_{A}^{*}\left(t\right)$ and ${T}_{B}^{*}\left(t\right)$ to be 1. We compare the analytical derivatives of the biomasses to the results of our numerical calculations (Figure 1a–d). We find that all computed values are identical.
Finally, we calculate the stability (which is quantified by $Re\left(\lambda \right)$) of ${\mathbf{X}}^{\mathbf{*}}={\left({C}_{A}^{*}\left(t\right),{C}_{B}^{*}\left(t\right),{T}_{A}^{*}\left(t\right),{T}_{B}^{*}\left(t\right),1\right)}^{\mathrm{T}}$ for different values of ${C}_{A}^{*}\left(t\right)$ and ${C}_{B}^{*}\left(t\right)$ ranging from 0.1 to 1. We choose values for $k$ and ${l}_{T}$ to be as given in Table 1, and ${f}_{A}^{*}$, $f}_{B}^{\ast$, $T}_{A}^{\ast},\phantom{\rule{thinmathspace}{0ex}}\mathrm{a}\mathrm{n}\mathrm{d}\phantom{\rule{thinmathspace}{0ex}}{T}_{B}^{\ast$ take values defined in Equations S13–S16.
We could see that all of possible values of ${C}_{A}$ and ${C}_{B}$ indicate that the system is unstable, suggesting that an analytical analysis to exactly capture the temporal unstable state of the system is not applicable.
Data availability
The MATLAB code for the regulated toxin model, the invasion analysis, and the evolutionary tournament is available on github (https://github.com/reneniehus/bact_warfare, copy archived at https://archive.softwareheritage.org/swh:1:rev:923e104aa634230547ba464c6bc8fee07f662ffa).
References

Bioactive microbial metabolitesThe Journal of Antibiotics 58:1–26.https://doi.org/10.1038/ja.2005.1

Nooks and crannies in type VI secretion regulationJournal of Bacteriology 192:3850–3860.https://doi.org/10.1128/JB.0037010

Multicoloured greenbeards, bacteriocin diversity and the rockpaperscissors gameJournal of Evolutionary Biology 26:2081–2094.https://doi.org/10.1111/jeb.12222

Bayesian inference for a discretely observed stochastic kinetic modelStatistics and Computing 18:125–135.https://doi.org/10.1007/s112220079043x

The evolution of bacteriocin production in bacterial biofilmsThe American Naturalist 178:E162–E173.https://doi.org/10.1086/662668

Colicin biologyMicrobiology and Molecular Biology Reviews 71:158–229.https://doi.org/10.1128/MMBR.0003606

Synergy and group size in microbial cooperationThe American Naturalist 180:296–305.https://doi.org/10.1086/667193

Competition sensing: the social side of bacterial stress responsesNature Reviews Microbiology 11:285–293.https://doi.org/10.1038/nrmicro2977

Bacterial virulence gene regulation: an evolutionary perspectiveAnnual Review of Microbiology 54:519–565.https://doi.org/10.1146/annurev.micro.54.1.519

The evolution of fatal fightingAnimal Behaviour 39:1–9.https://doi.org/10.1016/S00033472(05)807213

Quorumsensing regulation of the production of blp bacteriocins in Streptococcus thermophilusJournal of Bacteriology 189:7195–7205.https://doi.org/10.1128/JB.0096607

Spatial polymorphism of bacteriocins and other allelopathic traitsEvolutionary Ecology 8:369–386.https://doi.org/10.1007/BF01238189

Bacteriocins, spite and virulenceProceedings of the Royal Society of London. Series B: Biological Sciences 271:1529–1535.https://doi.org/10.1098/rspb.2004.2756

The role of stress in colicin regulationArchives of Microbiology 196:753–764.https://doi.org/10.1007/s0020301410178

Ribosomally encoded antibacterial proteins and peptides from PseudomonasFEMS Microbiology Reviews 38:523–568.https://doi.org/10.1111/15746976.12079

The role of SOS boxes in enteric bacteriocin regulationMicrobiology 154:1783–1792.https://doi.org/10.1099/mic.0.2007/0161390

Temporal changes in the frequency of colicinogeny in Escherichia coli from house miceMicrobiology 144 (Pt 8):2233–2240.https://doi.org/10.1099/0022128714482233

A theoretical and empirical investigation of the invasion dynamics of colicinogenyMicrobiology 145 (Pt 3):655–661.https://doi.org/10.1099/135008721453655

The evolution and ecology of bacterial warfareCurrent Biology 29:R521–R537.https://doi.org/10.1016/j.cub.2019.04.024

Diffusion approximations for lotka – volterra type modelsCommunications in Statistics. Stochastic Models 14:809–832.https://doi.org/10.1080/15326349808807502

Bacterial contactdependent delivery systemsAnnual Review of Genetics 44:71–90.https://doi.org/10.1146/annurev.genet.42.110807.091449

Bacterial competition: surviving and thriving in the microbial jungleNature Reviews Microbiology 8:15–25.https://doi.org/10.1038/nrmicro2259

A view to a kill: the bacterial type VI secretion systemCell Host & Microbe 15:9–21.https://doi.org/10.1016/j.chom.2013.11.008

BookEvolutionary Games and Population DynamicsCambridge University Press.https://doi.org/10.1017/CBO9781139173179

Inducible SOS response system of DNA repair and mutagenesis in Escherichia coliInternational Journal of Biological Sciences 4:338–344.https://doi.org/10.7150/ijbs.4.338

BookDyadic contests: modelling fights between two individualsIn: Hardy I. C. W, Briffa M, editors. Animal Contests. Cambridge University Press. pp. 5–32.https://doi.org/10.1017/CBO9781139051248.004

Bacterial danger sensingJournal of Molecular Biology 427:3744–3753.https://doi.org/10.1016/j.jmb.2015.09.018

FrequencyDependent selection in bacterial populationsPhilosophical Transactions of the Royal Society of London. Series B, Biological Sciences 319:459–472.https://doi.org/10.1098/rstb.1988.0059

Platforms for antibiotic discoveryNature Reviews Drug Discovery 12:371–387.https://doi.org/10.1038/nrd3975

Purification and genetic characterization of plantaricin NC8, a novel cocultureinducible twopeptide bacteriocin from Lactobacillus plantarum NC8Applied and Environmental Microbiology 69:383–389.https://doi.org/10.1128/AEM.69.1.383389.2003

BookEvolution and the Theory of GamesCambridge University Press.https://doi.org/10.1017/CBO9780511806292

BookMathematical Models of Social EvolutionChicago: University of Chicago Press.https://doi.org/10.7208/chicago/9780226558288.001.0001

Social evolution in multispecies biofilmsPNAS 108 Suppl 2:10839–10846.https://doi.org/10.1073/pnas.1100292108

Type VI secretion system regulation as a consequence of evolutionary pressureJournal of Medical Microbiology 62:663–676.https://doi.org/10.1099/jmm.0.0539830

Emergence of spatial structure in cell groups and the evolution of cooperationPLOS Computational Biology 6:e1000716.https://doi.org/10.1371/journal.pcbi.1000716

Spatial structure, cooperation and competition in biofilmsNature Reviews Microbiology 14:589–600.https://doi.org/10.1038/nrmicro.2016.84

Comparative study of the pln locus of the quorumsensing regulated bacteriocinproducing L. plantarum J51 strainInternational Journal of Food Microbiology 128:390–394.https://doi.org/10.1016/j.ijfoodmicro.2008.08.004

Human skin Microbiota is a rich source of bacteriocinproducing staphylococci that kill human pathogensFEMS Microbiology Ecology 95:fiy241.https://doi.org/10.1093/femsec/fiy241

SoftwareR: A Language and Environment for Statistical ComputingR Foundation for Statistical Computing, Vienna, Austria.

Bacteriocins: evolution, ecology, and applicationAnnual Review of Microbiology 56:117–137.https://doi.org/10.1146/annurev.micro.56.012302.161024

Type VI secretion system effectors: poisons with a purposeNature Reviews Microbiology 12:137–148.https://doi.org/10.1038/nrmicro3185

The evolution of titfortat in Bacteria via the type VI secretion systemNature Communications 11:5395.https://doi.org/10.1038/s4146702019017z

The biogeography of polymicrobial infectionNature Reviews Microbiology 14:93–105.https://doi.org/10.1038/nrmicro.2015.8

Bacillus subtilis antibiotics: structures, syntheses and specific functionsMolecular Microbiology 56:845–857.https://doi.org/10.1111/j.13652958.2005.04587.x

Understanding policing as a mechanism of cheater control in cooperating BacteriaJournal of Evolutionary Biology 32:412–424.https://doi.org/10.1111/jeb.13423

Experimental studies on the ecological role of antibiotic production in BacteriaEvolutionary Ecology 10:405–421.https://doi.org/10.1007/BF01237726

Effect of dispersal and nutrient availability on the competitive ability of toxinproducing yeastProceedings of the Royal Society B: Biological Sciences 275:535–541.https://doi.org/10.1098/rspb.2007.1461

Colicin Vtreated Escherichia coli does not generate membrane potentialJournal of Bacteriology 158:757–759.https://doi.org/10.1128/jb.158.2.757759.1984
Decision letter

AnneFlorence BitbolReviewing Editor; Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland

Aleksandra M WalczakSenior Editor; École Normale Supérieure, France

Rolf KümmerliReviewer; University of Zurich, Switzerland
In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.
Acceptance summary:
Bacteria often produce toxins to fight competitors. There has been strong interest in understanding the molecular mechanisms of toxin production, release and mode of actions. Less well understood are the costs and benefits of toxin production and how natural selection acts on regulatory circuits controlling toxin production. The paper tackles this problem. Using computer simulations, the authors show that regulated toxin production is generally a better strategy than constitutive toxin production, and reciprocication is fundamental in competitions. Interestingly, reciprocication doesn't evolve into peaceful coexistence, but a strenuous battle.
Decision letter after peer review:
[Editors’ note: the authors submitted for reconsideration following the decision after peer review. What follows is the decision letter after the first round of review.]
Thank you for submitting your article "The Evolution of Strategy in Bacterial Warfare" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Rolf Kümmerli (Reviewer #3).
Our decision has been reached after consultation between the reviewers. Based on these discussions, we regret to inform you that we cannot accept your work for publication in eLife at present. However, if you address all the major concerns below through indepth changes to the manuscript and new computations, we will be happy to consider a suitably revised manuscript as a new submission, that will be sent back to the same referees. Suggestions below were synthesized from the 3 reviews by the Reviewing Editor.
Summary:
Bacteria typically produce toxins to fight competitors. There has been strong interest in understanding the molecular mechanisms of toxin production, release and mode of actions. Less well understood are the costs and benefits of toxin production and how natural selection should act on regulatory circuits controlling toxin production. The paper tackles this problem. Using computer simulations, the authors show that regulated toxin production is generally a better strategy than constitutive toxin production, and reciprocication is fundamental in competitions. Interestingly, reciprocication doesn't evolve into peaceful coexistence, but a strenuous battle.
All Reviewers and the Reviewing Editor found that the paper makes important and interesting conclusions, and agreed that the model is relevant. However, there were significant reservations about methodology, and about whether the approach fully warrants the conclusions of the paper.
Essential revisions:
1) Because of the computational approach of searching parameter space of a quite complex ODE model, a clear quantitative understanding is not provided, but rather an intuition for the observed results. While the model captures the different toxin regulatory systems, which may be complicated to tackle analytically, some analytical insight would really help to demonstrate the generality of the conclusions.
For instance, an analytical model contrasting constitutive vs. regulated traits (first part of the paper), would build a stronger foundation.
2) The authors should clarify why the local versus global analysis is required. This is all the more true that the effect of spatial structure was not explored in the current paper.
If all competitions are pairwise and the main focus are strategies that invade all other strategies and cannot be reinvaded, why is a metapopulation analysis necessary? Also, the concept is presented is presented in Figure 1, but it is not farther discussed.
The strategies that win/lose locally, but lose/win globally should be discussed to shed light on the importance of this metapopulation analysis. This might matter with regards to possible extensions of the model to spatial settings.
3) The connection to (evolutionary) game theory appears superficial. The authors should clarify this point and be particularly careful about wording when they explain the bases of their model.
In particular, the sentence "we combine evolutionary game theory with differential equation modelling" (line 20) is unfortunate. Evolutionary game theory is primarily differential equation modelling, as for instance, the replicator and replicator mutator equations are ordinary differential equations. (One exception regards finite population analysis, where tools from statistical physics enter.)
In addition, invasion analysis by itself is static game theory. It implies dynamics from stability of fixed points, but not the actual dynamics itself which is taken into account when studying evolutionary games.
4) Are the initial frequencies of the two competing populations important for the final outcome? This is related to the previous question regarding different outcomes between local and global. Also, while the model is currently deterministic, in real case scenarios noise plays a big role, so it would be good to briefly discuss this.
5) In the regulated models, the parameter constraints allow f_induced and f_initial to range between 0 and 1. But because of how the model is setup, this means that it can happen (and it definitely does looking at Figure 2) that f_initial+f_induced=f is outside the [0,1] range, which is though the constrained range for the f of the constitutive competing strain. This might cause strange model behavior and definitely an unfair competition, so these parameters should be removed/checked.
1) The model developed in the manuscript captures the toxin regulatory system in bacteria, which is very interesting. However, the title of the manuscript and the abstract should be revised to better reflect the specific system under study.
2) The population dynamic equations in eq set 1 could be analysed further to get some analytical handle to promote our understanding. The authors could start from the following paper Vasconcelos, P., Rueffler C., 2020, How Does Joint Evolution of Consumer Traits Affect Resource Specialization? The American Naturalist 195: 331348.
In addition, the final biomass densities used in the calculation of the invasion fitness (Eq. 2) could be further simplified by focusing on the equilibrium solutions of Eqs. 1 (perhaps under which assumptions the solutions are possible could already be interesting). Providing an expression for the invasion fitness would be a real plus.
3) Please clarify the rationale for choosing parameters in the simulations, and discuss robustness, beyond the statement "parameters of the algorithm […] are chosen to achieve short simulation times and good convergence behaviour as determined by visually inspecting the distribution of population parameters over time." (727730)
4) Some of the findings are intuitive, as the authors acknowledge. E.g. regulated traits outperform nonregulated traits, but others are more surprising and very interesting. For example, many toxins are quorumsensing regulated (e.g. phenazines in P. aeruginosa). But the authors show that this is not an ideal mechanism because a strain might be killed before it's reaching a high enough density to switch on QSregulated toxins. I'd like to see a more detailed discussion on this putative mismatch. Related to this, my intuition is that the three mechanisms might work in concert, i.e. a toxin is QS controlled, but the threshold for QS activation is lowered by competition sensing. This would be interesting to discuss.
5) The conclusion on lines 3235 is strong. It might apply to mixed conditions as simulated here. However, in spatially structured habitats reciprocal fighting might lead to coexistence as competitors manage to defend their 'territories' and fighting only occurs at the borders. The swift elimination of competitors is maybe less common than assumed in realworld set ups.
6) The argument is that toxin sensing works best as the winner switches off toxin production once the competitor is eliminated. However, toxins might outlast their producers, such that the switching off might take longer than assumed. Can the authors comments on this?
[Editors’ note: further revisions were suggested prior to acceptance, as described below.]
Thank you for submitting your revised article "The evolution of strategy in bacterial warfare: quorum sensing, stress responses, and the regulation of bacteriocins and antibiotics" for consideration by eLife. Your article has been reviewed by 2 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Aleksandra Walczak as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Rolf Kümmerli (Reviewer #2).
Both reviewers are very positive about your revised manuscript and recommend publication. However, both reviewers and the reviewing editor agree that one revision would improve your manuscript. Therefore, we would like to ask you to do this revision before we accept your manuscript for publication.
Essential revision:
While the motivation for the local vs. global analysis is well explained, the results from this analysis should be clarified. In figure 1, interesting examples are proposed of strategies that might win/lose locally and then lose/win globally. These are very interesting cases that point at nontrivial competition dynamics. It would be important to add a figure or table (perhaps in the supplement) and an associated discussion paragraph where it is shown that doing this additional metapopulation analysis is necessary and adds something to the local competition part. The aim of this addition would be to address the following question: Would the optimal strategy change if one didn't do the global competition step? Some statistics of counterintuituive scenarios explored, based on their classification in figure 1d, would address this point.
Reviewer #1 (Recommendations for the authors):
The manuscript puts forward exciting hypotheses about the strategies of toxin production in bacteria, which would be very interesting to test experimentally. Also, future work that tries to shed deeper analytical insight in the results would undoubtedly be very interesting.
I think the work is definitely worth publication. I find the presentation of the results sometimes difficult to follow, partially because the details of the models, which is very helpful to understand the results, is detailed at the end of the paper in the methods. But probably this is unavoidable given the format.
I believe the current version of the manuscript reads better and clarifies some of the misunderstandings in previous versions. There are, however, some criticisms that have not, in my opinion, been well addressed.
1) I appreciate the complexity of the model and the strengths that come with including this complexity. The new analytical work carried out to investigate stability of the fixed points helps towards the analytical interpretation of the results. I think, however, the criticism previously raised by the reviewers was trying to determine whether a simpler model, analytically tractable, would be able to reproduce some of the results showed here, while giving more analytical insight. I don't think the manuscript currently addresses this issue. On the other hand, I also think that it can be left to future work as it is a very interesting and challenging research direction.
2) I am satisfied with the motivation behind the local versus global analysis, which I think is very important, especially in potential future applications to spatial settings of this work. The motivation behind the analysis was never in question. What I don't understand are the results from this analysis. In figure 1, interesting examples are proposed of strategies that might win/lose locally and then lose/win globally. These are very interesting cases that point at nontrivial competition dynamics that would be interesting to investigate further in future work, but I don't see these cases discussed anywhere in the results. I understand that the results of the algorithm come from a sequence of local competition, dispersal, seeding, etc…, that include this metapopulation competition, but I would like to see a figure/paragraph/discussion where it is shown that doing this additional metapopulation analysis adds something to the local competition part. Would the optimal strategy change if one didn't do the global competition step? If the answer is yes, which I imagine it is, where do I see this?
Reviewer #2 (Recommendations for the authors):
The authors have done a very good job in revising their paper. The main issue that arose during the first round of reviewing was the lack of an analytical model to establish a stronger foundation of the principles of competition sensing. Although it was not me who brought up this issue, I have consulted the authors' responses, edits in the main paper and extra analysis in the supplements with great care. My opinion is that the authors have convincingly solved the debate. There is simply no possibility to device an analytical model that captures even the simplest regulatory circuits involved in competition sensing and toxin production. The most important thing is that the authors have not only used verbal arguments to make their point, but have immensely invested in analytical modelling to show why the approach does not work. I believe that the strength of the paper is its biological realism and the fact that it generates predictions that can be empirically tested.
Moreover, the authors have adequately addressed my own comments. The discussion on interactions between regulatory mechanisms and the role of ecology have significantly improved the paper. The addition on local vs. global interactions is also important, especially for nonspecialist readers.
https://doi.org/10.7554/eLife.69756.sa1Author response
[Editors’ note: the authors resubmitted a revised version of the paper for consideration. What follows is the authors’ response to the first round of review.]
Essential revisions:
1) Because of the computational approach of searching parameter space of a quite complex ODE model, a clear quantitative understanding is not provided, but rather an intuition for the observed results. While the model captures the different toxin regulatory systems, which may be complicated to tackle analytically, some analytical insight would really help to demonstrate the generality of the conclusions.
For instance, an analytical model contrasting constitutive vs. regulated traits (first part of the paper), would build a stronger foundation.
We agree that analytical solutions are desirable whenever possible. We did not go this route in this study because our goal is to capture the details of bacterial regulation and competition that microbiologists care about, and to study when and why such complex regulation evolves. The issue then is that analytical solutions would require much simpler starting equations, or simplifying assumptions, with which we would not have achieved our goal. This is already evident in our equation 1, which is based upon a previously published study on bacterial competition via toxins (Bucci, Nadell, and Xavier 2011). This equation, although it does not include the details of toxin regulation, is already of a class that is problematic for analytics.
Nevertheless, over these last months, we have engaged in a serious effort to see what explorations might be possible analytically with our model or a simplified version of it that still captured the relevant biology. To do this, we brought on board a mathematician (Aming Li) who is an expert in the analytics of ecological and evolutionary processes. More specifically, we used Equation 1 from our manuscript and wrote the analytical form that gives the system's derivative for a given state with respect to time as well as the Jacobian matrix to explore the stability of the system (new Supplementary analytics section of the manuscript). It is important to note that we are making the simplifying assumption of setting nutrients to a constant, akin to a chemostat version of our model. We cannot make a similar assumption regarding toxins, as this would remove the key way in which the bacterial strains are interacting.
We then analytically explored the system's stability for a set initial conditions and a range of toxin investments. We compared the analytical results for the derivatives with the results from our numerical simulation, and we found them to be identical (Supplementary analytics section, Supplementary Analytics Figure 1). This is then a confirmation that analytics and our numerics give the same results, although this result is expected because we have implemented our numerical simulations with step sizes chosen to extremely accurately approximate an analytical solution. Our stability analysis of the system shows that all equilibria are unstable (Supplementary analytics section, Supplementary Analytics Figure 1), which is because our model contains feedback loops that render it unstable for nontrivial states.
Supplementary Analytics Figure 1 also demonstrates clearly how analytical explorations fall short of tackling the essence of our system: the results show that the toxin investment of strain A at time t does not impact biomass B, which seems counterintuitive. This is because the analytical solution explores an instant in time, but fails entirely to capture the key interactions between strains. Those occur through toxin production (and in the full model through consumption of limited nutrients), and those create feedback loops within the system (for example toxin investment of A is responsible for reduction in biomass of B and then in turn for a reduction in toxin B), analogous to more classical predatorprey models, which we note have previously been shown to be intractable by analytics (Boys, Wilkinson, and Kirkwood 2008; Liu and Chen 2003; Gutiérrez and Rosales 1998).
In summary, we found that an analytical exploration of the model did not capture the key processes at hand, nor did it help us with our understanding. We have now added a new paragraph where we discuss this ("Overview" section of "Results and Discussion") and better explain why we rely on numerics. We also now include the analytical stability analysis as a Supplementary Method (Supplementary Analytics section).
In place of analytical results, we have numerically explored the evolution of different regulatory strategies under a wide range of conditions, including i) against one optimised unregulated strategy (Figure 2), ii) against a diversity of unregulated strategies (Figure 3), iii) against the same class of unregulated strategy (Figure 4a,b) and, finally, iv) against an open set of all of the regulated and unregulated strategies (Figure 4c,d). Across all of these scenarios, the strategy class that performs the best is toxin sensing, our key conclusion. Moreover, we have tested the robustness of this prediction across numerous parameter sweeps (Figures 2, S4, S5, S6), which again predict the supremacy of toxin sensing. Finally, we are able to provide an intuitive post hoc explanation for why toxin sensing does so well – it is able to downregulate toxin production after winning a competition – and, importantly, we demonstrate this explanation is indeed causal by shortening competition time which removes this advantage (Figures 3, S7). All of these points combine to give us a lot of confidence in our predictions.
To close this discussion, we would also like to emphasise that – while our work does not have proofs of the sort that are seen in some papers in theoretical biology – our attention to biological details does mean that we generate readily testable prediction e.g. bacteria should often use damage or cues of damage to coordinate their attacks. Our approach then allows us to fulfil our goals better than generating simpler models that lack these details in order to look for more abstract principles. That is, while our work may not be amenable to robustness testing via analytics, it is amenable to experimental testing by microbiologists, and this is our research priority.
2) The authors should clarify why the local versus global analysis is required. This is all the more true that the effect of spatial structure was not explored in the current paper.
If all competitions are pairwise and the main focus are strategies that invade all other strategies and cannot be reinvaded, why is a metapopulation analysis necessary? Also, the concept is presented is presented in Figure 1, but it is not farther discussed.
The strategies that win/lose locally, but lose/win globally should be discussed to shed light on the importance of this metapopulation analysis. This might matter with regards to possible extensions of the model to spatial settings.
Our modelling captures different strategies for bacterial interactions and we are interested in the expected longterm evolutionary fate for these strategies. To achieve this in a way that is relevant to the real world, we consider the potential competition both within and between patches of bacteria, which recognises that the outcome of local competition in any given patch will often not be sufficient to predict evolutionary trajectories. Consider, for example, a competition between two strains of bacteria in one of the pores of your tongue. If we focus solely on local competition within the pore, then any strategy that results in a focal strain making more cells than its competitor will be favoured, even if this leads to relative ruin for the winning strain, with only a few cells surviving the process. However, given these competitions can go on in many pores across many tongues, it is unlikely that such extreme strategies would be favoured as they mean few cells are produced to colonise new patches. Instead, the best strategies are those that make the most cells to disperse, which can also mean they win locally, but it might not. This is what our global analysis captures and it is why it is critical for our modelling. To make this clearer, we now include the above thought experiment in the main text (under section "Evolution of warfare via unregulated toxin production") in reference to Figure 1. We also now discuss spatial structure both with regards to the local (ODE) model, and with regards to our metapopulation level (see "Conclusions", second paragraph).
3) The connection to (evolutionary) game theory appears superficial. The authors should clarify this point and be particularly careful about wording when they explain the bases of their model.
In particular, the sentence "we combine evolutionary game theory with differential equation modelling" (line 20) is unfortunate. Evolutionary game theory is primarily differential equation modelling, as for instance, the replicator and replicator mutator equations are ordinary differential equations. (One exception regards finite population analysis, where tools from statistical physics enter.)
In addition, invasion analysis by itself is static game theory. It implies dynamics from stability of fixed points, but not the actual dynamics itself which is taken into account when studying evolutionary games.
We agree that our wording in the example above and in other places was not very clear. However, our work is more than superficially linked to game theory. To appreciate this, one has to realise that there are, by now, two rather different approaches used in evolutionary game theory. Evolutionary game theory, as originally conceived by Maynard Smith and Price, is based on calculations that ask whether a given strategy will invade from initially being rare (static game theory e.g. MaynardSmith 1982. Evolution and the Theory of Games, Cambridge University Press). These are the kinds of tests that we are performing. More recently, an alternative set of approaches that use differential equations to follow the evolutionary process have also been called evolutionary game theory (dynamic game theory e.g. Hofbauer and Sigmund 1998. Evolutionary games and population dynamics. Cambridge University Press).
The confusion here then lies in the fact that we are not using this second form of game theory, but we are using ODEs. However, in our case the ODEs follow competitions on ecological timescales, which we use to calculate the success of a strategy against another strategy who meet locally. It is because of these biological details – the competition for nutrients and with toxins and the resulting feedback loops – that we need to capture in our local competitions that our system is not amenable to using dynamic game theory (see our response to point (1)). And of course we cannot just stop at modelling local competition without studying how this will affect the entire metapopulations on an evolutionary timescale (as discussed in the last point). To achieve this, we combine our numerical solutions of our local competition with static game theory to ask whether this local invasion will lead to a global invasion (Maynard Smith and Price). To avoid further confusion for any reader, we now make it clearer that we are using static game theory in the text as follows:
–line 33: Abstract “Here we combine a detailed dynamic model of bacterial competition with static game theory to study the rules of bacterial warfare.” This avoids too many detailed modelling terms before we define them clearly in the main text
(previously: “Here we combine evolutionary game theory with differential equation modelling to study the rules of bacterial warfare.”)
– line 95: Introduction “Here we study the evolution of strategy during bacterial warfare by combining a detailed differential equation model of toxinbased competition with static game theory to identify the most evolutionarily successful strategies.” (previously: “Here we study the evolution of strategy during bacterial warfare by combining an explicit differential equation model of toxinbased competition with evolutionary game theory.”)
– line 119: Results and Discussion, Overview “We use these differential equations to model ecological interactions of bacterial strains and determine the outcome of competition for a given strategy against another strategy when they meet locally. These locallevel competitions are embedded into a larger metapopulation framework that determines longterm evolutionary outcomes [56] (Figure 1c and d). This metapopulation modelling includes invasion analysis, in the tradition of static game theory developed by Maynard Smith and Price [57], and a more explicit genetic algorithm that employs the same logic (Methods). This algorithm pits diverse strategies against each other across a large number of combinations in order to find the most successful strategies. In these metapopulation models, bacterial strains are assumed to compete locally in a large number of patches, but also globally through dispersal to seed new, empty patches based on a standard life history of bacteria used in previous models [58–62] (Figure 1c). Also as previously [62], we refer to the global population as a metapopulation to distinguish it from the local bacterial cell population in each patch. This approach accounts for the possibility that a strategy can do well in local competition, but do poorly globally, and vice versa (Figure 1d). ” (previously: “We use these equations to pit different strategies of attack against one another, and these local competitions are imbedded into a larger framework that uses evolutionary game theory [55] to understand which strategies will evolve (Figure 1 c and d).”)
– line 225: “To capture this effect, we embed the locallevel competitions within a broader metapopulation framework in order to make evolutionary predictions [56,69,70] (see Figure 1 c and d). This framework allows us to ask whether a particular, initially rare, strategy can successfully invade a metapopulation of another strategy (Methods).” (previously: “We embed these competitions within a broader framework of evolutionary game theory[55,64,65]“)
4) Are the initial frequencies of the two competing populations important for the final outcome? This is related to the previous question regarding different outcomes between local and global. Also, while the model is currently deterministic, in real case scenarios noise plays a big role, so it would be good to briefly discuss this.
This question is covered by one of several parameter sweeps that were in the original submission (Figure S6). There we study the effects of initial frequency in a model of stochastic variation in the initial strain frequencies, using the most general model where all strategies compete with one another. Our results show our predictions are robust for different initial frequencies in the two strains. We have now extended our description of this result in the main text (section "The coevolution of regulated attack strategies') to make this clearer.
5) In the regulated models, the parameter constraints allow f_induced and f_initial to range between 0 and 1. But because of how the model is setup, this means that it can happen (and it definitely does looking at Figure 2) that f_initial+f_induced=f is outside the [0,1] range, which is though the constrained range for the f of the constitutive competing strain. This might cause strange model behavior and definitely an unfair competition, so these parameters should be removed/checked.
We are sorry about this confusion in the text. To be clear about the definitions: f_initial indicates the toxin investment at the initial condition, f_induced indicates the level of toxin investment when a sensor is triggered. Thus, the change in investment occuring due to the trigger is f_inducedf_initial. We clarify this now in the text and in the legend of Figure 2.
1) The model developed in the manuscript captures the toxin regulatory system in bacteria, which is very interesting. However, the title of the manuscript and the abstract should be revised to better reflect the specific system under study.
This is a good suggestion, and we have now revised the abstract and title accordingly to be more specific about the types of regulation (quorum sensing and stress responses) and the phenotype under regulation (toxin production).
2) The population dynamic equations in eq set 1 could be analysed further to get some analytical handle to promote our understanding. The authors could start from the following paper Vasconcelos, P., Rueffler C., 2020, How Does Joint Evolution of Consumer Traits Affect Resource Specialization? The American Naturalist 195: 331348.
In addition, the final biomass densities used in the calculation of the invasion fitness (Eq. 2) could be further simplified by focusing on the equilibrium solutions of Eqs. 1 (perhaps under which assumptions the solutions are possible could already be interesting). Providing an expression for the invasion fitness would be a real plus.
Thank you for this suggestion, we looked through this paper and considered it as a starting point but it does not capture our problem well and, as discussed above, analytics more generally are incapable of capturing key features of our system.
3) Please clarify the rationale for choosing parameters in the simulations, and discuss robustness, beyond the statement "parameters of the algorithm […] are chosen to achieve short simulation times and good convergence behaviour as determined by visually inspecting the distribution of population parameters over time." (727730)
As is typical in nonadaptive genetic algorithms, we base our choice of the control parameters of the algorithm on positive convergence behaviour and short convergence times (Melanie 1996). Further, we reran the simulations with 20 different sets of hyperparameters to confirm that our findings are not specific to the set of parameters we used initially. We now add this explanation into the "Genetic Algorithm" section: "In our sensitivity analysis, we also examine the results of the genetic algorithm with alternative sets of control parameters, including a smaller and a larger size of the mutation standard deviation (sd=0.01 and sd=0.0005), a smaller and larger proportion of “migrating” strategies in each generation (5 of 60, and 20 of 60), and five different sizes of the population of strategies (50, 70, 80, 90, 100). This yields 20 alternative parameters combinations." (see "Genetic Algorithm" section). The results were consistent with our main simulation. We have also extended the appropriate sentence in the Results section: "Despite a great number of potential combinations (over two million different competitions), and with different sets of hyperparameters of the genetic algorithm, we again see a clear winner in toxinbased regulation, both for our normal parameters (Table 1, Figure 4c) and for sweeps that consider broad ranges of these parameters (Figures S5) and a wide range of initial frequencies of the two strains (Figure S6). "
4) Some of the findings are intuitive, as the authors acknowledge. E.g. regulated traits outperform nonregulated traits, but others are more surprising and very interesting. For example, many toxins are quorumsensing regulated (e.g. phenazines in P. aeruginosa). But the authors show that this is not an ideal mechanism because a strain might be killed before it's reaching a high enough density to switch on QSregulated toxins. I'd like to see a more detailed discussion on this putative mismatch. Related to this, my intuition is that the three mechanisms might work in concert, i.e. a toxin is QS controlled, but the threshold for QS activation is lowered by competition sensing. This would be interesting to discuss.
We agree, these are two very interesting points. We have now included a more detailed discussion of why QS might be more common than we are expecting from our results. We now say (in "Conclusion" section) "However, if detecting damage is the best basis for attack, why do some bacteria use these other forms of regulation? For short competition times, our model predicts that the three regulatory strategies are largely equivalent (Figures 3 and S7). A short duration of competition between strains removes the benefit of decreasing toxin production once an attacker has been overcome. Under these conditions, the evolutionary path to one form of regulation may largely be determined by differences in costs for regulatory networks and which preexisting regulatory systems are available for cooption [98,99]. We predict, therefore, that mechanisms to reciprocate attacks are particularly valuable in environments where warfare commonly leaves a victor unchallenged for a long time afterwards. Consistent with this, one of the clearest examples of reciprocation occurs in E. coli [66,84,88], which uses colicin toxins to displace other strains and persists for long periods within the mammalian microbiome [100].
Another possible explanation for why some bacteria do not use cell damage to regulate their toxins comes from the notion of ‘silent’ toxins. These are toxins that are not easily detected by the cell’s stress responses, which may limit the potential for a toxinmediated response. For example, some toxins depolarise membranes[101] and may be favoured by natural selection specifically because they do not provoke dangerous reciprocation in competitors [87].".
We have also extended our discussion on how multiple sensory mechanisms might work in concert ("Conclusion" section): "[…] bacteria appear to use multiple forms of regulation in order to integrate information from multiple sources [41]. For example, Streptomyces coelicolor regulates antibiotic production via both nutrient limitation [102] and mechanisms that detect incoming antibiotics (envelope stress [103]). A potential future use of our modelling framework would be to study how these combined regulatory strategies evolve."
5) The conclusion on lines 3235 is strong. It might apply to mixed conditions as simulated here. However, in spatially structured habitats reciprocal fighting might lead to coexistence as competitors manage to defend their 'territories' and fighting only occurs at the borders. The swift elimination of competitors is maybe less common than assumed in realworld set ups.
We agree with this, and have now removed this sentence from our abstract.
6) The argument is that toxin sensing works best as the winner switches off toxin production once the competitor is eliminated. However, toxins might outlast their producers, such that the switching off might take longer than assumed. Can the authors comments on this?
Yes, in our local competitions toxins can outlast their producers. Consistent with this, winning strains commonly downregulate their toxin once the competitor toxin falls to a value that is small, but not exactly zero. That is, these strategies use low rather than zero toxin as a cue for elimination of the opponent. In response, we have altered our discussion of this as follows: "Instead, the winning strategies are initially aggressive and only become passive once an opponent is eliminated (Figure 4 df) – or nearly eliminated, because eliminated strains can be outlasted by their toxins, which in turn will take a while to be lost entirely. Indeed the winning strains in these competitions downregulate their toxin once the competitor toxin falls below a value that is small but still nonzero."
[Editors’ note: what follows is the authors’ response to the second round of review.]
Essential revision:
While the motivation for the local vs. global analysis is well explained, the results from this analysis should be clarified. In figure 1, interesting examples are proposed of strategies that might win/lose locally and then lose/win globally. These are very interesting cases that point at nontrivial competition dynamics. It would be important to add a figure or table (perhaps in the supplement) and an associated discussion paragraph where it is shown that doing this additional metapopulation analysis is necessary and adds something to the local competition part. The aim of this addition would be to address the following question: Would the optimal strategy change if one didn't do the global competition step? Some statistics of counterintuituive scenarios explored, based on their classification in figure 1d, would address this point.
Reviewer #1 (Recommendations for the authors):
The manuscript puts forward exciting hypotheses about the strategies of toxin production in bacteria, which would be very interesting to test experimentally. Also, future work that tries to shed deeper analytical insight in the results would undoubtedly be very interesting.
I think the work is definitely worth publication. I find the presentation of the results sometimes difficult to follow, partially because the details of the models, which is very helpful to understand the results, is detailed at the end of the paper in the methods. But probably this is unavoidable given the format.
I believe the current version of the manuscript reads better and clarifies some of the misunderstandings in previous versions. There are, however, some criticisms that have not, in my opinion, been well addressed.
1) I appreciate the complexity of the model and the strengths that come with including this complexity. The new analytical work carried out to investigate stability of the fixed points helps towards the analytical interpretation of the results. I think, however, the criticism previously raised by the reviewers was trying to determine whether a simpler model, analytically tractable, would be able to reproduce some of the results showed here, while giving more analytical insight. I don't think the manuscript currently addresses this issue. On the other hand, I also think that it can be left to future work as it is a very interesting and challenging research direction.
2) I am satisfied with the motivation behind the local versus global analysis, which I think is very important, especially in potential future applications to spatial settings of this work. The motivation behind the analysis was never in question. What I don't understand are the results from this analysis. In figure 1, interesting examples are proposed of strategies that might win/lose locally and then lose/win globally. These are very interesting cases that point at nontrivial competition dynamics that would be interesting to investigate further in future work, but I don't see these cases discussed anywhere in the results. I understand that the results of the algorithm come from a sequence of local competition, dispersal, seeding, etc…, that include this metapopulation competition, but I would like to see a figure/paragraph/discussion where it is shown that doing this additional metapopulation analysis adds something to the local competition part. Would the optimal strategy change if one didn't do the global competition step? If the answer is yes, which I imagine it is, where do I see this?
We were very pleased that the referees were enthusiastic about publication. As requested, we have now addressed the remaining concerns about the nontrivial competition dynamics shown in Figure 1d. As suggested, we have added a Supplementary Table that reports on the occurrence of all four possible competition dynamics (as per Figure 1 d) occurring in the simulations underlying Figure 2. This shows that all of the four possible outcomes occur. The less intuitive outcomes are rarer than the more intuitive ones, as one would expect, but the fact that all cases occur underlines that the correct metric for fitness is one that accounts for global competition as well as local competition. We now highlight this analysis in the description of the Figure 1, and we also discuss the new Table where we describe our findings of Figure 2.
https://doi.org/10.7554/eLife.69756.sa2Article and author information
Author details
Funding
EPSRC (EP/G50029/1)
 Rene Niehus
European Research Council (787932)
 Kevin R Foster
Wellcome Trust (209397/Z/17/Z)
 Kevin R Foster
Wellcome Trust (Interdisciplinary Fellowship)
 Nuno M Oliveira
Biotechnology and Biological Sciences Research Council (BB/T009098/1)
 Nuno M Oliveira
PKUBaidu (2020BD017)
 Aming Li
College of Engineering, Peking University (Startup funding)
 Aming Li
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
Thank you to Michael Bentley, Jonas Schluter, Kat Coyte, Nicholas Davies, Wook Kim, and Rolf Kümmerli for feedback. RN is funded by the EPSRCfunded Systems Biology Doctoral Training Centre studentship EP/G50029/1. NMO is funded by the Herchel Smith Fellowship. AGF is funded by the Vice Chancellor’s Fellowship from the University of Sheffield. KRF is funded by European Research Council Grant 787932 and Wellcome Trust Investigator award 209397/Z/17/Z. This project is supported by a Templeton World Charity Foundation grant.
Senior Editor
 Aleksandra M Walczak, École Normale Supérieure, France
Reviewing Editor
 AnneFlorence Bitbol, Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland
Reviewer
 Rolf Kümmerli, University of Zurich, Switzerland
Publication history
 Received: April 25, 2021
 Accepted: August 1, 2021
 Version of Record published: September 7, 2021 (version 1)
Copyright
© 2021, Niehus et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 990
 Page views

 110
 Downloads

 0
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.