The evolution of strategy in bacterial warfare via the regulation of bacteriocins and antibiotics
Abstract
Bacteria inhibit and kill one another with a diverse array of compounds, including bacteriocins and antibiotics. These attacks are highly regulated, but we lack a clear understanding of the evolutionary logic underlying this regulation. Here, we combine a detailed dynamic model of bacterial competition with evolutionary game theory to study the rules of bacterial warfare. We model a large range of possible combat strategies based upon the molecular biology of bacterial regulatory networks. Our model predicts that regulated strategies, which use quorum sensing or stress responses to regulate toxin production, will readily evolve as they outcompete constitutive toxin production. Amongst regulated strategies, we show that a particularly successful strategy is to upregulate toxin production in response to an incoming competitor’s toxin, which can be achieved via stress responses that detect cell damage (competition sensing). Mirroring classical game theory, our work suggests a fundamental advantage to reciprocation. However, in contrast to classical results, we argue that reciprocation in bacteria serves not to promote peaceful outcomes but to enable efficient and effective attacks.
Introduction
Bacteria commonly live in dense and diverse communities where competition for space and nutrients can be intense (Kim et al., 2014; Hibbing et al., 2010). As a response to such ecology, bacteria have evolved a wide range of competitive traits (Granato et al., 2019), including contactdependent inhibition (Ghequire and De Mot, 2014; Hayes et al., 2010; Aoki et al., 2010), the type VI secretion system (Miyata et al., 2013; Russell et al., 2014; Ho et al., 2014; Hood et al., 2010), narrowspectrum bacteriocins, and broadspectrum antibiotics (Bérdy, 2005; Riley and Wertz, 2002; Chao and Levin, 1981), which can kill or inhibit other strains. These mechanisms are extremely widespread. Bacteriocidal toxins are found in almost all major bacterial lineages (Riley and Wertz, 2002; Granato et al., 2019) and single species commonly make use of multiple toxins and diverse means of attack (Granato et al., 2019; Be'er et al., 2010; Zhang et al., 2012; Steele et al., 2017; Jamet and Nassif, 2015). This toxinbased warfare is also important for bacterial ecology and evolution, with evidence that toxin production can prevent competing strains from invading a niche (SassoneCorsi et al., 2016; Nakatsuji et al., 2017; Zipperer et al., 2016; O’Sullivan et al., 2019), kill off coexisting strains (Granato et al., 2019; Majeed et al., 2011; Speare et al., 2018; Ma et al., 2014), or help strains to invade new niches (Kommineni et al., 2015; Wiener, 1996; Sana et al., 2016).
The production and regulation of bacterial toxins have been studied for decades because of their potential as clinical antibiotics (Lewis, 2013; Slattery et al., 2001). This work has revealed that toxin production is often tightly regulated (Miyata et al., 2013; Anderson et al., 2012; Ghazaryan et al., 2014; Bernard et al., 2010). Indeed, it is thought that there are many new antibiotics that remain undetected because they are only activated under certain conditions (Maldonado et al., 2003; Abrudan et al., 2015; Traxler et al., 2013). A major form of regulation in bacteria is quorum sensing (Fuqua et al., 1994; Navarro et al., 2008; Eickhoff and Bassler, 2018) whereby cells secrete a small molecule and respond to it dependent upon its concentration. Some antibiotics and bacteriocins are regulated by quorum sensing, which is thought to ensure that toxin production occurs at the right cell density (Hibbing et al., 2010; Chandler et al., 2012). Other factors also regulate bacterial toxin production, including particular nutrient conditions and diverse stress responses (Storz and Hengge, 2011). This led to the argument that, in addition to quorum sensing, bacteria engage in ‘competition sensing’ whereby they use nutrient stress and cell damage to detect ecological competition (Cornforth and Foster, 2013; Lories et al., 2020).
Bacteria, therefore, have the potential for a wide range of responses during combat. Evolutionary theory has so far focused on the evolution of unregulated toxin production. This work has highlighted that factors such as strain frequency, nutrient level, the level of strain mixing (relatedness), and the cost of toxin production are all important for whether bacteria employ toxins at all (Bucci et al., 2011; Gardner et al., 2004; Brown et al., 2006; Gordon and Riley, 1999; Frank, 1994; Levin, 1988). Other models have highlighted how natural selection for warfare can have consequences for the evolution of diversity (Frank, 1994; Biernaskie et al., 2013; Kelsic et al., 2015; McNally et al., 2017), including via rockpaperscissor dynamics between different genotypes (Czárán et al., 2002; Kerr et al., 2002). However, to understand the strategic potential of warring bacteria, we must consider the regulation of their toxins and other weapons (Granato et al., 2019; Cornforth and Foster, 2013).
Here, we study the evolution of strategy during bacterial warfare by combining a detailed differential equation model of toxinbased competition with game theory to identify the most evolutionarily successful strategies. Informed by the large empirical literature on factors that regulate bacteriocins and antibiotics, we compare four major classes of potential strategies: constitutive (unregulated) toxin production, and regulation via nutrient level, quorum sensing, or by damage from a competitor’s toxin. We study the behaviours and competitive success of each strategy when in competition with other strains across a range of scenarios. We find that all three types of regulated strategies carry benefits relative to nonregulated production and, for shortlived resources, the three types of regulation offer largely equivalent alternatives for controlling attacks. However, for longlived environments, responding to incoming attacks is often the best performing strategy. A key benefit to such reciprocation in such environments is the ability to downregulate a toxin once a competitor is defeated, thereby saving the energy that would be lost in needless aggression.
Results and discussion
Overview
We are interested in how competition between strains and species of bacteria shapes the evolution of toxin regulation. The core of our approach is a set of detailed ordinary differential equations (ODEs) that capture ecological competition between bacteria (Figure 1a), which are built upon an earlier model of bacterial siderophore production (Niehus et al., 2017). After exploring the case of constitutive toxin production only, we extend our model to incorporate different strategies of regulated production (Figure 1b). We use these differential equations to model ecological interactions of bacterial strains and determine the outcome of competition for a given strategy against another strategy when they meet locally.
These locallevel competitions are embedded into a larger metapopulation framework that determines longterm evolutionary outcomes (Maynard Smith, 1982; Figure 1c and d). This metapopulation modelling includes invasion analysis, in the tradition of the branch of evolutionary game theory developed by Maynard Smith and Price, 1973 and the field of adaptive dynamics (Materials and methods). We also later use a more explicit genetic algorithm that employs the same logic. This algorithm pits diverse strategies against each other across a large number of combinations in order to find the most successful strategies (Materials and methods). In these metapopulation models, bacterial strains are assumed to compete locally in a large number of patches, but also globally through dispersal to seed new, empty patches based on a standard life history of bacteria used in previous models (Oliveira et al., 2014; Gardner et al., 2007; Chuang et al., 2009; Cremer et al., 2012; Nadell et al., 2010; Figure 1c). Also, as discussed previously (Nadell et al., 2010), we refer to the global population as a metapopulation to distinguish it from the local bacterial cell population in each patch. This approach accounts for the possibility that a strategy can do well in local competition, but do poorly globally, and vice versa (Figure 1d). We show in later analysis that all competitive outcomes shown in Figure 1d occur in our simulations, with the first two cases being the most common.
We explore a number of different evolutionary scenarios using different calculations. We start by using invasion analysis (depicted in Figure 1d) to study the evolution of toxin producers that lack regulation and to study the evolution of toxin regulation from constitutive producers. This allows us to understand first when, and how much, a strain should invest into attacking other, and then, whether regulated production can evolutionarily replace constitutive production. We next compare different regulated strategies to one another by studying their performance when facing a diverse range of constitutively producing species. Finally, we study the case where regulated strategies compete with each other and coevolve in massive tournaments to identify the most globally successful strategies (see Materials and methods).
Our model needs to be relatively complex in order to capture the evolution of bacterial competition and regulatory networks. As a result, the form of our mathematical model is of a class that is not amenable to analytical work (Boys et al., 2008; Gutiérrez and Rosales, 1998; Liu and Chen, 2003). To confirm this, we investigated the behaviours of the dynamic model at steady state. This showed a good basic correspondence between our numerics and analytics but confirmed that the model is not amenable to further analytical work (see Appendix 1 Supplementary analytics). Nevertheless, by combining a number of different competition scenarios with wide parameter sweeps, we are able to show that our key conclusions are robust across many conditions.
Evolution of warfare via unregulated toxin production
We first ask, what favours the evolution of constitutive toxin production. While many toxins are regulated, constitutive production does occur (Mavridou et al., 2018), and we use the simple case of constitutive toxin production to first identify general principles underlying the evolution of bacterial warfare. In addition, constitutive production forms a baseline from which to compare the evolution of regulated strategies. In order to study the behaviours that result from each strategy, we use a detailed model of competition between strains based upon a system of differential equations (Materials and methods). This approach allows us to capture the temporal dynamics of strain interactions and, later, toxin regulation.
In the model, we follow nutrient concentration and cell biomass over time as the strains engage with each other (Figure 1a). We focus on competitions between two strains that each possess a toxin that does not harm the producer strain but does harm the other strain. In reality, strains may carry multiple toxins and resistances (Cordero et al., 2012; Gordon et al., 1998) and our framework can be extended to include such complexity. However, for simplicity, we focus here on a single toxin produced by each strain. We consider interactions that are pairwise at the strain level, but we later account for a multitude of competitors by letting strains have many encounters, each with a different strategy. To enable us to study a large number of strategies, our differential equations are based upon simplified wellmixed conditions.
Our goal is to understand the evolutionary fate of different strategies of toxinmediated competition. In order to do this, we need to recognise that the outcome of competition at a local scale may not be predictive of evolutionary trajectories. Consider, for example, a competition between two strains of bacteria on a particle of detritus in a pond. If one focuses solely on local competition on the particle, then any strategy that results in a focal strain making more cells than its competitor will be favoured, even if this leads to relative ruin for the winning strain with only a few cells surviving the process. However, given these competitions can happen on many such particles, it is unlikely that such extreme strategies would be favoured, because few cells will be produced to colonise new particles. Instead, the best strategies will be those that make the most cells to disperse, which may mean a strain also wins locally, but it may not (see Figure 1d).
To capture this effect, we embed the locallevel competitions within a broader framework in order to make evolutionary predictions (Maynard Smith, 1982; Weibull, 1997; Mitri et al., 2011) (see Figure 1c and d). This framework allows us to ask whether a particular, initially rare, strategy can successfully invade a metapopulation of another strategy (Materials and methods). Specifically, a rare mutant’s fitness in the metapopulation is defined by the number of cells it produces in direct competition with the resident, while the resident’s fitness is defined by its productivity when it meets another resident in a patch, as will occur in the vast majority of patches if the mutant is rare (Figure 1d, Materials and methods). For mutants that can invade, we also confirm that they then cannot be reinvaded by the previous resident (Materials and methods), which is indeed always the case here. We refer to such invasions that lead to a full replacement of the resident by the mutant – where the resident is unable to reinvade from rare  as a stable invasion. By studying large numbers of competitions, we can categorise strains by their ability to stably invade others, and thereby identify the evolutionarily stable investment into toxin production (f*). We then seek the optimal level of toxin production, which cannot be invaded by any mutant strategy, but can invade all others.
What determines the optimal level of toxin investment? Intuitively, we find that cells evolve to invest more in attacking their competitors when toxins are efficient at killing the competitor and/or the toxins persist stably in the environment (Table 1). Toxin efficiency in our model is equivalent to the relative cost of toxin production, that is, we see a high benefittocost ratio favours toxin use. This result is in line with previous theory, which has shown that the impact of toxin production on growth rate is critical for the evolutionary outcome (Bucci et al., 2011; Levin, 1988). For highly effective toxins, we find that strains will engage in an arms race that escalates to the point where populations can go extinct (Appendix 1—figure 4). Such ‘evolutionary suicide’ is known from a wide range of conflict scenarios in biology (Rankin and LópezSepulcre, 2005).
While earlier models have studied the effects of nutrients on toxin production, these studies either did not model nutrients explicitly (Frank, 1994), or the level of nutrient competition was coupled to the presence of and mixing with other strains (Bucci et al., 2011; Gardner et al., 2004). In our model we can isolate the effect of nutrients on the evolution of toxin use. When nutrients are scarce, there is not enough energy to produce effective amounts of toxins (Appendix 1—figure 3), which agrees with previous theory (Bucci et al., 2011; Gardner et al., 2004; Frank, 1994), and has also been shown experimentally in yeast (WlochSalamon et al., 2008). However, we also find that toxin benefit peaks at intermediate nutrient availability and decreases for higher nutrient levels (Appendix 1—figure 3). This can be understood in terms of a shift in the relative benefits of investing in cell division versus attack: When bacteria enter a competition at low density and resources are abundant, there is a great potential for population expansion. Under these conditions, cells evolve to invest relatively little in toxin production; energy is instead better invested in rapid growth to win a competition by outgrowing other strains. In contrast, when growth potential is limited, cells benefit from investing in warfare, unless, as mentioned above, nutrients are too scarce to produce an effective toxin concentration.
The evolution of regulated attack strategies
We next investigate what happens when cells are able to regulate their level of toxin production in response to environmental cues. The production of antibiotics and bacteriocins is commonly tightly regulated by a variety of signals and cues. As discussed above, these can be broadly divided into three major classes based upon known bacterial regulatory networks. The first is detection of cell density by canonical quorum sensing or related means (Fuqua et al., 1994; Eickhoff and Bassler, 2018), which has been demonstrated by previous modelling work to be beneficial for the regulation of cooperative traits (Cornforth et al., 2012). In addition, bacteria are highly responsive to both nutrient stress and cell damage associated stress (Storz and Hengge, 2011), which both can detect the level of ecological competition in the environment (‘competition sensing’; Cornforth and Foster, 2013; Lories et al., 2020).
We first compare the evolution of regulation by quorum, nutrient level, and the level of the competitor’s toxin when each is in competition with constitutive strains. This allows us to ask whether regulated strategies can evolutionarily replace constitutive strategies (see Materials and methods). In brief, we model regulation of toxin production using a simple step function, which is defined by toxin production in activated state (f_{induced}), production in inactivated state (f_{initial}), and a threshold of the signal for activation. All three parameters are continuous; toxin production (f_{initial} and f_{induced}) is constrained between 0 and 1, and the threshold is constrained to a region consistent with the observed range of each signal (quorum, nutrient, toxin level).
In a vast tournament consisting of millions of individual competitions, we pit all possible strategies of each mode of regulation against all possible versions of the fixed strategy. We then use invasion analysis, as before, to look for the evolution of regulated strategies that can invade all unregulated strategies. As before, we consider both global and local competition (see Figure 1d) to determine invading strategies that cannot be reinvaded by the previous resident strategy and that therefore cause stable invasion. We find that all possible outcomes of the metapopulation competitions (Figure 1d) do occur, with the typical case being that the outcome of local and global competition are the same (see Appendix 1—figure 6). In a small minority of cases (2.3%), we find that successful invading strategies can be reinvaded by the previous resident to give a mixed evolutionary outcome, and these cases are not considered further. Our analysis identifies versions of each mode of regulation that can stably invade all possible constitutive strategies (Figure 2a–c). This result is expected and confirms the basic intuition that – unless maintaining a regulatory circuit is very costly – a wellregulated trait will outcompete an unregulated one (Cornforth and Foster, 2013). This is true whether strains compete for a short or long duration, although shorter duration does select for a higher initial investment in toxin production in order to ensure that enough toxin is made in the time that a strain has to compete (Appendix 1—figure 5).
How do the different regulatory strategies achieve their success? For the great majority of cases, successful strains evolve to upregulate their attack after a delay, either based on the detection of low nutrients, high quorum, or high levels of the competitor’s toxin (Figure 2d–f). In some cases, there is no toxin production before this upregulation, as in the canonical model of quorum sensing that turns a trait from off to on. In other cases, the strategy that evolves is to begin with a baseline of constitutive production before upregulating this further upon activation (Figure 2a–c, with example shown in Figure 2f), something also seen in real systems (Mavridou et al., 2018). A difference between nutrient and quorum sensing versus toxinbased regulation is that examples of the latter not only upregulate toxin production after a delay, they also downregulate the toxin if the competitor is killed off (Figure 2e).
We also discovered winning strategies that function by downregulating toxin production after a delay. For nutrientbased regulation, there is a narrow parameter range (the small vertical strip in the lower part of Figure 2a) where strategies begin aggressively with the expression of toxin and then downregulate it when nutrients are limited (dynamics not shown). For quorumsensing strategies, some also start with high toxin investment, but these strategies are more complex. These downregulate toxins and invest in growth once they reach a high density, but will reactivate it again if their cell numbers drop due to toxin attack (Figure 2g).
In each case, regulated strategies win by only making high levels of toxin at certain times, thereby saving energy relative to constitutive producers. A corollary is that, if toxin production is cost free, regulation will no longer be benefitical relative to constitutive production. But, assuming that there is some costs to toxin production, regulation is expected to be favoured by natural selection.
In sum, there are regulated strategies of each of the three types under study that can evolutionarily replace all nonregulated strategies. However, this analysis is based on regulated strains invading metapopulations consisting of a single constitutive strategy. In some contexts, a focal strain may face a variety of competitors. Consider, for example, a situation where migration brings in a range of competiting species, each optimised to a different environment. To consider this scenario, we next ask how the different sensing strategies fare in competition with a standing diversity of constitutive strategies. We introduce diversity by letting the different sensory strategies (i.e. nutrient sensing, toxin sensing, and quorum sensing) face an increasingly diverse mix of constitutive toxinproducing opponents. We assume that the standing diversity of constitutive producers is not itself affected by the evolution of the regulated startegies, that is, there is no coevolution (we consider coevolution in the next section, however). For each set of opponents, therefore, we can identify the best performing regulated strategies simply as those that obtain the highest average biomass across the competitions with the set of opponents (Materials and methods). Based on the simulated data, we also fitted a linear regression model with sensing type as a categorical predictor and number of competitors a numerical predictor (see Materials and methods).
When opponents have a single strategy (lowest diversity), the toxin sensing strategy is most efficient in terms of its final biomass produced (Figure 3a, left panel). Moreover, the toxin sensing strategy deals most effectively with diverse competitors (Figure 3a) with the regression analysis showing a 2.5 times higher fitness for toxin sensing relative to the other strategies (pvalue < 0.001). The success of the toxin sensing strategy is associated with the reliable activation of toxin production when sensing another toxin. Quorum sensing also activates toxin production during the competition but, in some cases, is defeated without being able to attack back. This gives rise to the observed bimodal outcome of the quorumsensing strategy (Figure 3a). The nutrientsensing strategy, by contrast, attacks first and then deactivates later when nutrients decrease.
This superiority of toxin sensing is robust across a range of parameters, including different toxin efficiencies, toxin loss rates, and nutrient concentrations (Appendix 1—figure 7). There is a clear post hoc intuition to this result. A strain that only engages in conflict when attacked will be best able to deal with a range of strategies that differ in their propensity and ability to attack. More specifically, as seen in the last section, these strains inactivate toxin production after a weak opponent is eliminated, thus employing the toxin efficiently. We can directly demonstrate the importance of this tactic of toxin inactivation by shortening the duration of the strain competitions such that toxinsensing strains do not have the opportunity to downregulate toxin production. For short competition times, while regulated strategies still outperform unregulated ones (Appendix 1—figure 5), the toxin sensing strategy fails to evolve a superior performance over the other modes of regulation (Figure 3b).
The coevolution of regulated attack strategies
We have considered how regulated attack strategies perform in the face of constitutive strategies that vary in their level of aggression, and in the face of varying levels of diversity in these opponents. This revealed that regulation is generally beneficial and indicated that the sensing of an opponent’s toxin is often the best performing strategy. However, this analysis is artificial in the sense that bacteria with regulated strategies are also likely to compete against one another. Therefore, we next ask, which sensing strategy is most successful when coevolving with other sensing strategies? We first consider strains that interact with others that have a similar attack strategy, regulated by the same environmental cue. For each of the three types of regulation, we then search for the optimal strategy using a genetic algorithm (see Materials and methods) (Figure 4a). Following the logic of the earlier models, the optimal strategy is defined as one that will, on average, obtain the highest biomass across competitions with all other possible strategies.
When competing with the same strategy, all strategies initially evolve to increase toxin production during the competition (f_{initial} < f_{induced}) (Figure 4b). More specifically, strains responding to nutrient depletion initially produce near zero toxins (f_{initial} = 0.05) until they activate toxin investment, at a level higher than the optimal fixed investment (evolved f_{induced} = 0.50, while f* = 0.35). In comparison, strains responding to quorum sensing invest in more toxin initially (f_{initial} = 0.11) and also more when activated (f_{induced} = 0.60). The quorumsensing strategy is expected to be able to afford to invest more in toxin production because, unlike nutrient sensing, strains can reduce toxin investment again if biomass drops too low, thereby saving energy. The toxinsensing strategy is different again. It invests near zero toxin at the start of the competition (f_{initial} = 0.01) and responds very strongly if a competitor attacks (f_{induced} = 0.73). Interestingly, the corollary is that, at evolutionary equilbirum (when it will meet an identical toxin strategy), both remain passive and achieve a high biomass (Figure 4b center). This outcome has similarities to the success of ‘titfortat’, a reciprocal cooperating strategy in the classic evolutionary game theory tournament of Axelrod and Hamilton, 1981. There, titfortat succeeds by benefiting from mutual cooperation whenever others cooperate, while maintaining the ability to shut off cooperation whenever it meets a noncooperative strategy. When this success leads to all individuals playing titfortat, the result can be that all interactions end up as cooperative, akin to the emergence of a peaceful productive strategy in our model.
The evolution of a peaceful outcome is specific to the ability to reciprocate; we do not observe it for nutrient or quorum sensing. Nevertheless, we have identified a route by which bacteria might evolve the peaceful resolutions seen in animal and human conflicts (Axelrod and Hamilton, 1981; Kokko, 2013; Freedman, 1989). However, the model assumes that strains will only interact with other genotypes that are adopting similar strategies for warfare. This is far from guaranteed in bacteria as there exists considerable variability in weapons and their regulation, even within a single species (Mavridou et al., 2018). Moreover, microbial communities typically contain many strains and species, suggesting again that a given strain has the potential to meet a diversity of competitors and strategies.
We therefore sought to capture this complexity with a final model in which all possible regulated strategies are able to compete against each other, again using a genetic algorithm to identify optimal strategies (see Materials and methods). Despite a great number of potential combinations (over two million different competitions), and with different sets of hyperparameters of the genetic algorithm, we again see a clear winner in toxinbased regulation, both for our normal parameters (Table 1, Figure 4c) and for sweeps that consider broad ranges of these parameters (Appendix 1—figure 8) and a wide range of initial frequencies of the two strains (Appendix 1—figure 9). Moreover, as for competition against unregulated strategies (Figure 3), the success of toxinbased regulation in contests with other strategies does not come from an ability to avoid conflict and create peaceful outcomes. Instead, the winning strategies are typically aggressive when they meet another strain and they only downregulate their toxins once an opponent is on its way to being eliminated (Figure 4d–f). And, as for competition against unregulated strategies, this ability to become passive is key to their success. For short competitions, there is no benefit in turning off an attack and the competitive benefit of reciprocity over other regulated strategies is lost (Appendix 1—figure 10).
Conclusions
Bacteria use a wide variety of weaponry to harm other strains and species, which is typically under tight regulation (Ghequire and De Mot, 2014; Granato et al., 2019; Stein, 2005; MichelBriand and Baysse, 2002; Cascales et al., 2007). How bacteria employ these mechanisms of attack is central to understanding why a particular species or pathogen can invade and persist in communities, while others cannot (Granato et al., 2019; Kommineni et al., 2015). Here, we have explored the evolutionary logic underlying strategies of bacterial attack. We find that toxin production is favoured under many conditions, particularly when toxins are effective and longlasting and when the potential for population expansion is limited (Table 1). The prevalence of aggressive strategies in our model is consistent with the widespread use of toxins by bacteria (Granato et al., 2019), and the associated intensity of competition observed in experiments (Chao and Levin, 1981; Mavridou et al., 2018; Oliveira et al., 2015). We also find that wellregulated attacks can consistently outcompete strategies that lack regulation (Figure 2). This is because the benefit of employing a toxin not only changes with different competitors but also within a single competition over time. Regulation allows a strain to better tune its behaviour and follow the optimal investment at any given situation. However, the three major classes of bacterial regulatory network are not always equivalent ways to control attacks. Across a diverse range of potential competitors, responding directly to incoming attacks is the most robustly successful strategy (Figures 3,4).
Our modelling implicitly captures spatial structure at the metapopulation level with discrete patches of bacteria that compete with each other. Within patches, our ODE model best reflects environments with limited spatial structure where cells of different genotypes are mixed together. However, bacteria do also display fine scale spatiogenetic structuring within their communities (Nadell et al., 2016; Stacy et al., 2016; Krishna Kumar et al., 2021). Here, our model has the potential to capture the outcome of competition at the interface of two strains, which is expected to be critical for success and persistence in such communities (Granato et al., 2019). However, there is clear potential for other effects of local spatial structure on sensing strategies that we do not capture. For example, in contrast to the detection of competitor’s toxins, responses to quorum sensing and nutrient depletion may occur first in the middle of a patch of cells, where toxin production has the least benefit as toxin receivers are mainly clonemates (Inglis et al., 2009; Wechsler et al., 2019).
Our work predicts that sensing incoming attacks through direct or indirect means should be a widespread way of regulating toxins and other modes of attack. This hypothesis lends itself to empirical testing via the study of bacterial behaviour during toxinmediated competition with other strains and species. Some examples of reciprocation already exist. Many bacteria upregulate attack mechanisms via stress responses that detect cell damage (Cornforth and Foster, 2013). This includes recent evidence of reciprocation between warring Escherichia coli strains where DNase protein toxins activate toxin production in competing strains via the SOS response to DNA damage (Mavridou et al., 2018; Krishna Kumar et al., 2021; Gonzalez et al., 2018; Granato et al., 2019). Because many antimicrobials target the DNA of cells (Janion, 2008; Gillor et al., 2008), sensing DNA damage is likely to be a relatively robust way to achieve reciprocity. But there are other mechanisms; Pseudomonas aeruginosa senses incoming attacks via the type six secretion system (T6SS) of competitors, which delivers toxin via the molecular equivalent of a speargun (Basler and Mekalanos, 2012; Basler et al., 2013). Upon detecting an incoming attack, a cell will activate its own T6SS in response (Basler et al., 2013). Consistent with our findings, recent work suggests that a key benefit to reciprocation via the T6SS is the ability to save energy and only attack when necessary, alongside a benefit that comes from improved aiming which is specific to this mode of attack (Smith et al., 2020). Finally, there is evidence that bacteria may also detect and respond to incoming attacks via proxies such as the detection of lysate produced when surrounding cells are killed (LeRoux et al., 2015a), or molecules that are made by an attacker alongside a toxin (Cornforth and Foster, 2013; LeRoux et al., 2015b).
There is also evidence that bacterial toxins can be regulated via nutrient depletion and quorum sensing (Ghequire and De Mot, 2014; Cascales et al., 2007). Our models of regulation by quorum or nutrients typically predict that attacks will evolve to be activated at high quorum or limited nutrients, which recapitulates the typical directionality of the regulation observed in nature (Chandler et al., 2012; Fontaine et al., 2007; Inaoka et al., 2003). However, if detecting damage is the best basis for attack, why do some bacteria use these other forms of regulation? For short competition times, our model predicts that the three regulatory strategies are largely equivalent (Figure 3 and Appendix 1—figure 10). A short duration of competition between strains removes the benefit of decreasing toxin production once an attacker has been overcome. Under these conditions, the evolutionary path to one form of regulation may largely be determined by differences in costs for regulatory networks and which preexisting regulatory systems are available for cooption (Cotter and DiRita, 2000; Hockett et al., 2015). We predict, therefore, that mechanisms to reciprocate attacks are particularly valuable in environments where warfare commonly leaves a victor unchallenged for a long time afterwards. Consistent with this, one of the clearest examples of reciprocation occurs in E. coli (Mavridou et al., 2018; Krishna Kumar et al., 2021; Granato et al., 2019), which uses colicin toxins to displace other strains and persists for long periods within the mammalian microbiome (Gillor et al., 2009).
Another possible explanation for why some bacteria do not use cell damage to regulate their toxins comes from the notion of ‘silent’ toxins. These are toxins that are not easily detected by the cell’s stress responses, which may limit the potential for a toxinmediated response. For example, some toxins depolarise membranes (Yang and Konisky, 1984) and may be favoured by natural selection specifically because they do not provoke dangerous reciprocation in competitors (Gonzalez et al., 2018). In other cases, bacteria appear to use multiple forms of regulation in order to integrate information from multiple sources (Cornforth and Foster, 2013). For example, Streptomyces coelicolor regulates antibiotic production via both nutrient limitation (Hesketh et al., 2007) and mechanisms that detect incoming antibiotics (envelope stress [Hesketh et al., 2011]). A potential future use of our modelling framework would be to study how these combined regulatory strategies evolve.
Bacteria use diverse regulatory networks to attack and overcome competitors, and there is much still to understand about their evolution. Here, we have identified general principles for the function of these networks in bacterial warfare. We find there are great benefits using regulation to time an attack; both to minimise its cost and maximise its effect on an opponent. We also find that regulation that enables reciprocation can be particularly beneficial. If cells only attack when attacked, they invest their energy where and when it is most needed: against aggressive opponents. Our findings are mirrored in the classical predictions from the game theory of animal combat, which suggested that adopting a reciprocal and retaliatory strategy can be effective (Maynard Smith and Price, 1973; Kokko, 2013; Freedman, 1989; Enquist and Leimar, 1990). However, the predicted outcome was typically one of peace and the avoidance of conflict, which is indeed what is observed in many animal contests (Briffa M, 2013). In contrast to such lessons, experimental work suggests that bacteria often engage in deadly conflict (Abrudan et al., 2015; Mavridou et al., 2018; Oliveira et al., 2015; Gonzalez et al., 2018; Be'er et al., 2009; Vetsigian et al., 2011). Our models offer an evolutionary rational for this observation. The regulation of combat in bacteria is not usually about avoiding conflict; it is about timing an attack and downregulating it once a competitor is no longer a threat.
Materials and methods
Overview
In this study we use a modelling framework that captures two scales of competition (Figure 1). At the local level, we model bacterial strain competitions using systems of ODEs. These equations are well suited to model temporal dynamics on the relatively short ecological timescales at which bacterial strains interact with nutrients and competitors. At the global level, we model the evolution of different strategies within a metapopulation. This metapopulation level allows us to follow the evolution of different strategies across much longer evolutionary timescales, and to capture the important interplay of local and global fitness (Figure 1d). We use this game theory framework to identify strategies that are evolutionarily successful against a diversity of possible competitors. All questions addressed in this work require both layers of modelling. The system of ODEs that models constitutive toxin production is described in the next section and forms the basis for all of the models. Evolution at the metapopulation level is implemented using a common logic (Figure 1c,d), using variations that capture a range of questions and evolutionary scenarios as detailed below.
A differential equation model of bacterial warfare
Request a detailed protocolOur model captures pairwise competitions between bacterial strains, which have the potential to produce toxins (Figure 1a). This first model allows a strain to have a fixed investment into its toxin – below we describe the extension of this model that allows toxin regulation in response to external cues. We employ ODEs, which are well suited to capture the temporal dynamics of strain interactions happening at ecological timescales. A number of different models have been used to study the evolution of bacterial public good regulation (Niehus et al., 2017; Heilmann et al., 2015; Kümmerli and Brown, 2010). Here, we follow Bucci et al., 2011, because they model both nutrients and toxins explicitly, which are both important cues for the regulation of toxin production. We study a competition between two strains that each possess a toxin that does not harm the cells of the producer strain, but does harm the other strain. In reality, strains may carry multiple toxins and resistances (Cordero et al., 2012; Gordon et al., 1998) and the evolution of multiple mechanisms of attack and defence is an interesting question in its own right. However, we focus here on a single toxin produced by each strain. We also describe the dynamics of the nutrients and cell densities in a wellmixed environment. The interactions of cells, nutrients, and toxins can be described by the system of ODEs:
where C_{A}(t) and C_{B}(t) denote the biomasses of cell strains A and B, respectively, T_{A}(t) and T_{B}(t) denote the biomass of each strain’s toxin, and N(t) denotes the concentration of a growthlimiting nutrient for which both strains compete. We consider a pool of nutrient that is depleted by the cells. Similarly to Nadell et al., 2008, we describe the energy that is available to the cells by the Monod equation, in which K_{N} is the nutrient saturation constant. The maximum growth rate is given by µ_{max}. Toxins kill with efficiency k and are lost with rate l_{T}. We assume that all toxins have identical loss and killing rates in order to remove biochemical differences between strains and focus our analysis on the effects of different production strategies.
For constitutive toxin production, the strategy of a strain is given simply by a fixed f (f ∈ [0,1]), which captures the investment into toxin production relative to cell biomass. The production of antibiotics and bacteriocins can have significant metabolic costs and can even require a cell to lyse, as occurs with colicins and pyocins (Cascales et al., 2007; Nakayama et al., 2000). We model the cost of toxin production on cellular growth as a linear allocative tradeoff function in the growth term (Bucci et al., 2011). For example, a strain that invests f = 0.1 into its toxin will only reach 90% of its maximal growth rate.
The dynamics of cells, nutrients, and toxins are modelled as continuous for their typical range. But when a cell strain reaches a very low concentration (C(t)=10^{−6}), we assume that stochastic extinction occurs such that cell concentration drops to 0. Further, our model assumes a limited lifetime of the local patches by stopping the dynamics when 24 hr (or less for the analysis of shortened competition times, Appendix 1—figure 5) have passed.
We solve the system of ODEs numerically using an implicit Euler method. This numerical scheme is implemented in MATLAB (version 9.5.0.944444) (MATLAB, 2018). Our implementation solves the equations (Equation (1)) until the defined end time. We avoid numerical issues due to negative state variables by setting any state variables reaching a value below 10^{−8} to 0.
A model of regulated toxin attack
Request a detailed protocolTo extend the above model to include sensing, toxin production of bacterial strain A is either a function of nutrient depletion, toxin of strain B, or of quorum sensing (given as cell biomass of strain A). Each signal triggers toxin production via a simple onandoff switch (Cornforth and Foster, 2013) so that the toxin production of strain A is given through one of the equations:
where H is the Heaviside step function given as
and where ${f}_{initial}\in \left[\mathrm{0,1}\right]$ and ${f}_{induced}\in \left[\mathrm{0,1}\right]$.
These equations of regulated toxin production each comprise the initial investment into toxins (f_{initial}) when the trigger term is deactivated and the trigger term itself. The trigger term contains a Heaviside step function and becomes active when the signal increases over the sensing threshold (U_{N}/U_{TB}/U_{QS}). When activated, the trigger term changes the initial toxin investment (f_{initial}) to become the induced toxin investment (f_{induced}). We allow the induced toxin investment to be smaller (when the signal is a repressor) or larger than the initial toxin investment (when the signal is an activator).
Invasion analysis
Request a detailed protocolWe use our first models to predict the optimal constitutive toxin production strategy across different ecological conditions. Here, the assumed scenario is a monomorphic metapopulation (all strains have identical warfare strategy), where a rare mutant strategy appears that may or may not invade this metapopulation. As time progresses toward infinity, the metapopulation will finally be dominated by a strategy that can invade the metapopulation of any other strategy and that can itself not be invaded. We implement this scenario using classic pairwise invasion analysis. More specifically, we employ game theory and, in particular, invasion analysis to find the best strategies (Nowak and Sigmund, 2004; McElreath and Boyd, 2013), where the best strategy is one that, if adopted by the whole population, cannot be invaded by any other strategy. These strategies are also called evolutionarily stable strategies (Maynard Smith, 1982).
We follow previous work (Oliveira et al., 2014; Cremer et al., 2012) by assuming a microbial life cycle that consists of a seeding step where local patches are seeded with two competing strains, a competition step where strains grow and interact according to the differential equations explained above, and a mixing step where cells from all patches disperse and mix, leading to a new seeding episode (Figure 1c). The proportion of the different strains (or strategies) that are seeded is determined by the strain frequencies after the competition step. Without explicitly modelling this life cycle, invasion analysis (McElreath and Boyd, 2013) asks whether a particular strain with strategy f_{inv} when rare can invade a population dominated by another strategy f_{res} (the resident Weibull, 1997; Figure 1d). To answer this, we calculate the fitness of the resident strategy (w_{res}) and the fitness of the invading strategy (w_{inv}). The fitness of the resident is its final biomass when in competition with an identical strategy so that w_{res} = w(f_{res}f_{res}) and the fitness of the rare invader is determined by its final biomass in the competition between invader and resident strategy, w_{inv} = w(f_{inv}f_{res}). We then calculate the invasion index for an invading strategy according to Mitri et al., 2011 as
When the invasion index I_{inv} is larger than 1, the rare strategy can invade the resident strategy; when the index is smaller than 1, the rare strategy cannot invade, and it disappears. Finally, we also test for backinvasion and compute I_{inv} for when the resident is rare and the mutant is the resident. We implement strain competitions by solving the system of ODEs described above. We define evoluationarily stable strategies as those strategies that have an I_{inv} larger than 1 against all studied competitors (and both as rare and resident strategy). By calculating the invasion index for a large number of invading strategyresident strategy pairs, we obtain a pairwise invasibility plot (Brännström et al., 2013) (insets in Appendix 1—figure 4). Using this plot, we find a single evolutionarily stable strategy f* that can invade all strategies and that cannot be invaded by any other strategy. We determine this globally optimal strategy using the algorithm outlined in the Appendix 1—code 1. We can then ask how the parameters of the model affect the evolution of toxin investment (Table 1).
Invasion analysis of sensing strategies
Request a detailed protocolWe next ask whether regulated strategies will evolutionarily replace constitutive production. Here, the ecological scenario is the same as above: monomorphic populations of constitutive toxin production strategies are threatened to be invaded by rare strategies that can sense (Figure 1d). We perform a parameter grid search that tests a large number of sensing strategies (stepping: ∆f_{initial}/∆f_{induced} = 0.02 and ∆U = 0.002, constraints: ${f}_{initial}\in \left[0,1\right]$, ${f}_{induced}\u03f5\left[0,1\right]$, ${U}_{N}\in \left[0,1\right]$, ${U}_{TB}\in \left[0,20\right]$, ${U}_{QS}\in \left[0,20\right]$) against the range of constitutive strategies. For the constitutive strategies, we select from a fine grid spacing that also includes the optimal constitutive strategy (f_{fixed} = [0.00, 0.01, 0.02, …, 1.00]). For each pair of sensing and nonsensing strategies, we compute the invasion index once for the sensing strategy as the resident and again for the nonsensing strategy being the resident. We then search for those sensing strategies that can invade all nonsensing strategies and that themselves cannot be invaded by any other nonsensing strategy. We show where those strategies lie in the parameter space of ${f}_{initial}$ and ${f}_{induced}$ (Figure 2ac).
Sensing strategies against standing diversity
Request a detailed protocolWe also study the evolutionary success of the three different types of sensing when being in constant competition with a diverse set of competitors. Here, the ecological scenario is a polymorphic metapopulation – a mix of different constitutive production strategies – with a given diversity. We assume that this diverse set of strategies is not influenced by evolution in the focal sensing strategy due to, for example, immigration that continually resupplies the diversity of competitors. We then ask what happens when a rare sensing strain enters this metapopulation, where its success depends on its success across pairwise competitions with the different resident strategies.
We implement this by competing focal sensing strategies against a set of different constitutive strategies and computing their fitness from the average biomass produced across those competitions. Specifically, for each of the three different sensing types, we perform a parameter grid search, creating a large number of predefined strategies across the parameter range of f_{initial} (∈[0,1], at increments of 0.05), f_{induced} (∈[0,1], 0.05) and respective thresholds U_{N} (∈[0,1], 0.02), U_{TB} (∈[0.001,4], 0.0005), and U_{QS} (∈[0.01,1.2], 0.01). Each of those sensing strategies is competed against a fixed set of constitutive strategies, one at a time, by solving the above system of equations. We then compute for each sensing strategy the average fitness across its competitions. Within each of the three sensing types we find the single strategy with the highest average fitness. For those winners we show the average fitness as bars in Figure 3 together with the fitnesses obtained against each individual constitutive strategy. We repeat this entire procedure for five different levels of diversity among the constitutive strategies. Starting with the lowest diversity set, which contains only a single constitutive strategy (f = 0.5), we then add increasingly extreme strategies, yielding three competitors (f = 0.4,0.5,0.6), five competitors (f = 0.3,0.4,0.5,0.6,0.7), seven competitors (f = 0.2,0.3,0.4,0.5,0.6,0.7,0.8), and finally nine competitors (f = 0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9).
Using the simulated results, we fit linear regression model with the sensing type as a categorical predictor variable and the number of competitors as a numerical predictor variable. The regression takes the form
where F_{i} is the fitness of the ith competition, which we assume to be normally distributed around a mean given by a linear equation and with standard deviation σ. The fixed intercept is given through α_{S[i]}, where S[i] is the ith element of integer vector S that contains only two possible values indicating whether the toxin sensor or a different sensor is in the competition i. D[i] is the ith element of integer vector D, which gives the number of competitors in the ith competition. Finally, β gives the change of fitness when adding one competitor. We fit the regression model using R (version 3.6.1) (R Development Core Team, 2011).
Genetic algorithm
Request a detailed protocolFinally, we study which sensing strategies are most successful in competition with other sensing strategies. We study this both within each type of sensing and across all three different types. Here, the scenario is a polymorphic metapopulation of coevolving sensing strategies. Mutation and migration create new strategies inside this population. A strategy’s achieved biomass in pairwise competitions with other strains determines its ability to stay and amplify in the metapopulation. The model initially studies a wide variety of strategies competing with one another. However, as time passes, the metapopulation converges and consists of increasingly optimal strategies. As this happens, the analysis then approximates the invasion analyses described above, where most strains are largely identical and rare mutants are pitted against this majority in the metapopulation (Figure 1d).
Specifically, we use a genetic algorithm to search for the evolutionarily stable strategy in the large space of possible strategies of a single type of regulation (and also in the space of all possible regulating strategies). This algorithm adapts the typical structure of a genetic algorithm (Melanie, 1996) where in each round a population of individuals is first tested to evaluate fitness and it is then replaced by a new daughter generation. Individuals of this new generation are created by a mix of cloning and mutating individuals from the previous parent generation selected based on their fitness and by addition of novel random strategies. As is typical in nonadaptive algorithms , the control parameters of the algorithm (e.g. number of generations, number of strategies in the population, rate of mutation, etc.) are chosen to achieve short simulation times and good convergence behaviour as determined by visually inspecting the distribution of population parameters over time (Melanie, 1996). Our population of competing strategies has a constant size of n=60. Initially a set of random strategies is created, whereby the three parameters that define an individual sensing strategy are drawn from a uniform distribution with given parameter constraints (${f}_{initial}\in \left[0,1\right]$, ${f}_{induced}\in \left[0,1\right]$, ${U}_{N}\in \left[0,1\right]$, ${U}_{TB}\in \left[0,4\right]$, ${U}_{QS}\in \left[0,1.2\right]$). The constraints for the sensing thresholds take the range of the respective signals as they are observed across the large number of competitions performed in the invasion analysis of sensing strains described above. (For the initial population in the case where all three sensing types compete, the sensing type is chosen at random with equal probability for all three types and, to avoid long run times and artefactual superiority due to parameter constraints, initial parameter values start at the optimum from the within strategy competition.)
In every round then, each strategy competes against all n strategies, including its own type. The final biomass of every strategy is summed across its competitions to give its competitive fitness. Then, a new daughter generation is generated. The four most competitive parent strategies are chosen to move into the next generation without parameter mutation, 36 strategies are drawn from the parent generation with probability proportional to their fitness and one of their parameters is mutated by adding a value drawn from a normal distribution with mean of 0 and standard deviation of 0.001. If, after mutation, a daughter strategy violates the parameter constraints, the random draw gets repeated until the constraints are met. Finally, 10 immigrant strategies are generated by choosing their sensing parameters as random draws from a uniform distribution within the constraints. (In the case of all three strategies competing, the sensing type is first drawn at random with equal probability, and then the sensing parameters are drawn at random.) For the competition between types of a single sensing strategies, the algorithm is run for 100 generation. (For the tournament with all three strategies, we ran the first 20 generations without selection, where we replace the population each generation with migrants, to allow comparison with the case where selection occurs, Figure 4c). The evolving parameter values for the top four strategies are averaged for each generation and saved (Figure 4a). The averaged values in the last timestep give the evolutionary stable strategy for each tournament (Figure 4b). In our sensitivity analysis, we also examine the results of the genetic algorithm with alternative sets of control parameters, including a smaller and a larger size of the mutation standard deviation (0.01 and 0.0005), a smaller and larger proportion of ‘migrating’ strategies in each generation (5 of 60, and 20 of 60), and five different sizes of the population of strategies (50, 70, 80, 90, 100). This yields a total of 20 alternative parameter combinations.
Code availability
Request a detailed protocolThe MATLAB code for the regulated toxin model, the invasion analysis, and the evolutionary tournament is available on GitHub (https://github.com/reneniehus/bact_warfare, copy archived at swh:1:rev:923e104aa634230547ba464c6bc8fee07f662ffa, Niehus, 2021).
Appendix 1
Appendix 1—code 1: Pseudo code to find the evolutionarily stable strategy of fixed toxin production.
Initialization: Find the initial resident fresini (for example, start with the highest average resident biomass)
forward = 1 indicates if the next migrant will have a higher or lower toxin investment than the resident. Can be 1 or 1.
previousforward = 1 record the direction of strategy change from the previous iteration
previousfres = 0 record the resident from the previous iteration
step = 0.1 initial step for the strategy change (initially coarse to localize the ESS)
minstep = 0.0001 the precision we want for the strategy value
singular = 0 boolean saying if a singular strategy is localised
nbflip = 0 record the number of consecutive flips in direction
fres = fresini
newres = 0 boolean saying if there is a new resident
while (!singular) {do this while no singular strategy is localised
if (newres) {
res vs N res competition. Compute the resident against the resident. Record the resident average strategy: wresav.
}
fmut = fres + forward*step pick an invader that differs from resident according to direction
mut vs N res competition. Compute mutant vs resident competition. Record the mutant biomass (wmut) at the end of the competition.
previousforward = forward update the previous direction
previousfres = fres update the previous resident before the competition
if (wmut < wresav) {resident stays the resident.
forward =  forward change the direction, to test a migrant with lower f value at the next step
}
if (wmut > wresav) {the mutant invades and replaces the resident
fres = fmut the new resident will take the migrant value
newres = 1 we have a new resident, we'll have to compute its resident fitness at the next iteration
}
if (forward != previousforward) {compare the direction with previous direction
nbflip = nbflip + 1
}
if (forward == previousforward) {compare the direction with previous direction
nbflip = 0
}
this will allow to localise an ESS. We want fres to stay identical for 2 consecutive time steps, but we want to make sure that the higher and lower strategies have been checked. Indeed if the mutant that we just tested goes extinct, the resident will stay resident but that is not enough to say it is an ESS. We also want the number of flips in direction to be > 1.
if (previousfres == fres & nbflip > 1) {singular strategy localised
if (step <= minstep) {precision is high enough
singular = 1 we have found the optimal strategy
}
if (step > minstep) {will redo the whole procedure with tinier step
step = step/10 increase precision 10 times
nbflip = 0 reset the number of consecutive flips
}
}
}
Record the final fres (ESS).
Finally, we test whether this fres (ESS) is a GLOBAL optimum by competing it against the range of strategies given through f = [0, 0.01, 0.02, ….1].
Supplementary analytics
For the system of ODEs presented in Equation (1), the system state could be rewritten in the vectorised form $\mathbf{X}={\left({C}_{A}\left(t\right),{C}_{B}\left(t\right),{T}_{A}\left(t\right),{T}_{B}\left(t\right),N\left(t\right)\right)}^{\mathrm{T}}$ with the system dynamics denoted as
We denote a specific equilibrium state by ${\mathbf{X}}^{*}$. In order to analyse the associated linearly asymptotical stability of the above system at ${\mathbf{X}}^{*}$, we should first find solutions safisfying ${\mathbf{F}(\mathbf{X}}^{*},{f}_{A}^{*},{f}_{B}^{*},k,{l}_{T})=0.$ We note, that in Equation (1), the dynamics of N is defined by a negative derivative, and from this derivative we can see that a stable state regarding N can only be reached when there are no cells or no nutrients. However, no cells is a trivial and extreme state (i.e. no cells are left for further seeding), and no nutrients cannot be reached within finite durations. We will therefore abandon the dyanmics of N from the system, basically assuming chemostat environment where different levels of N can be acchieved through balancing consumption and influx of N. This changes of course how cell strains interact, but it will help to show in a simpler way how analytical methods fail even for this simplified system of equations.
Now, setting ${\mathbf{F}(\mathbf{X}}^{*},{f}_{A}^{*},{f}_{B}^{*},k,{l}_{T})=0$ further gives
From Equation (S2), we have
From Equation (S3), we have
From Equation (S5), we have
From Equation (S4), we have
Thus, we know from Equations (S8) and (S9) that
Similarly from Equations (S6) and (S7), we get
Therefore, from Equations (S10) and (S11), we have
From Equations (S6) and (S8), we know that
From Equations (S12) and (S13), we know
and note that here when ${f}_{A}^{*}=1$, we obtain ${f}_{B}^{*}=0$, which is not applicable.
Substituting Equation (S14) into Equation (S9), we then know
Similarly, we further have
and
And the Jacobian matrix of the system state ${\mathbf{X}}^{*}$ is
By computing the maximum real part of all the eigenvalues of $\mathbf{A}$ (denoted by $\mathrm{Re}\left(\lambda \right)$), we know that ${X}^{*}$ is linearly asymptotical stable when $\mathrm{Re}\left(\lambda \right)<0$. This canonical measure of stability indicates whether and how fast the system would return to ${X}^{*}$ after a small perturbation (Hofbauer and Sigmund, 1998; Mougi and Kondoh, 2012; Coyte et al., 2015; Allesina and Tang, 2012).
Next, we calculate the derivative of the system for different values of ${f}_{A}$ and ${f}_{B}$, choosing values for $k$ and ${l}_{T}$ to be as given in Table 1, and setting ${C}_{A}^{*}\left(t\right)$ and ${C}_{B}^{*}\left(t\right)$ to be 0.1, and ${T}_{A}^{*}\left(t\right)$ and ${T}_{B}^{*}\left(t\right)$ to be 1. We compare the analytical derivatives of the biomasses to the results of our numerical calculations (Figure 1a–d). We find that all computed values are identical.
Finally, we calculate the stability (which is quantified by $Re\left(\lambda \right)$) of ${\mathbf{X}}^{\mathbf{*}}={\left({C}_{A}^{*}\left(t\right),{C}_{B}^{*}\left(t\right),{T}_{A}^{*}\left(t\right),{T}_{B}^{*}\left(t\right),1\right)}^{\mathrm{T}}$ for different values of ${C}_{A}^{*}\left(t\right)$ and ${C}_{B}^{*}\left(t\right)$ ranging from 0.1 to 1. We choose values for $k$ and ${l}_{T}$ to be as given in Table 1, and ${f}_{A}^{*}$, $f}_{B}^{\ast$, $T}_{A}^{\ast},\phantom{\rule{thinmathspace}{0ex}}\mathrm{a}\mathrm{n}\mathrm{d}\phantom{\rule{thinmathspace}{0ex}}{T}_{B}^{\ast$ take values defined in Equations S13–S16.
We could see that all of possible values of ${C}_{A}$ and ${C}_{B}$ indicate that the system is unstable, suggesting that an analytical analysis to exactly capture the temporal unstable state of the system is not applicable.
Data availability
The MATLAB code for the regulated toxin model, the invasion analysis, and the evolutionary tournament is available on github (https://github.com/reneniehus/bact_warfare, copy archived at https://archive.softwareheritage.org/swh:1:rev:923e104aa634230547ba464c6bc8fee07f662ffa).
References

Bioactive microbial metabolitesThe Journal of Antibiotics 58:1–26.https://doi.org/10.1038/ja.2005.1

Nooks and crannies in type VI secretion regulationJournal of Bacteriology 192:3850–3860.https://doi.org/10.1128/JB.0037010

Multicoloured greenbeards, bacteriocin diversity and the rockpaperscissors gameJournal of Evolutionary Biology 26:2081–2094.https://doi.org/10.1111/jeb.12222

Bayesian inference for a discretely observed stochastic kinetic modelStatistics and Computing 18:125–135.https://doi.org/10.1007/s112220079043x

The evolution of bacteriocin production in bacterial biofilmsThe American Naturalist 178:E162–E173.https://doi.org/10.1086/662668

Colicin biologyMicrobiology and Molecular Biology Reviews 71:158–229.https://doi.org/10.1128/MMBR.0003606

Synergy and group size in microbial cooperationThe American Naturalist 180:296–305.https://doi.org/10.1086/667193

Competition sensing: the social side of bacterial stress responsesNature Reviews Microbiology 11:285–293.https://doi.org/10.1038/nrmicro2977

Bacterial virulence gene regulation: an evolutionary perspectiveAnnual Review of Microbiology 54:519–565.https://doi.org/10.1146/annurev.micro.54.1.519

The evolution of fatal fightingAnimal Behaviour 39:1–9.https://doi.org/10.1016/S00033472(05)807213

Quorumsensing regulation of the production of blp bacteriocins in Streptococcus thermophilusJournal of Bacteriology 189:7195–7205.https://doi.org/10.1128/JB.0096607

Spatial polymorphism of bacteriocins and other allelopathic traitsEvolutionary Ecology 8:369–386.https://doi.org/10.1007/BF01238189

Bacteriocins, spite and virulenceProceedings of the Royal Society of London. Series B: Biological Sciences 271:1529–1535.https://doi.org/10.1098/rspb.2004.2756

The role of stress in colicin regulationArchives of Microbiology 196:753–764.https://doi.org/10.1007/s0020301410178

Ribosomally encoded antibacterial proteins and peptides from PseudomonasFEMS Microbiology Reviews 38:523–568.https://doi.org/10.1111/15746976.12079

The role of SOS boxes in enteric bacteriocin regulationMicrobiology 154:1783–1792.https://doi.org/10.1099/mic.0.2007/0161390

Temporal changes in the frequency of colicinogeny in Escherichia coli from house miceMicrobiology 144 (Pt 8):2233–2240.https://doi.org/10.1099/0022128714482233

A theoretical and empirical investigation of the invasion dynamics of colicinogenyMicrobiology 145 (Pt 3):655–661.https://doi.org/10.1099/135008721453655

The evolution and ecology of bacterial warfareCurrent Biology 29:R521–R537.https://doi.org/10.1016/j.cub.2019.04.024

Diffusion approximations for lotka – volterra type modelsCommunications in Statistics. Stochastic Models 14:809–832.https://doi.org/10.1080/15326349808807502

Bacterial contactdependent delivery systemsAnnual Review of Genetics 44:71–90.https://doi.org/10.1146/annurev.genet.42.110807.091449

Bacterial competition: surviving and thriving in the microbial jungleNature Reviews Microbiology 8:15–25.https://doi.org/10.1038/nrmicro2259

A view to a kill: the bacterial type VI secretion systemCell Host & Microbe 15:9–21.https://doi.org/10.1016/j.chom.2013.11.008

BookEvolutionary Games and Population DynamicsCambridge University Press.https://doi.org/10.1017/CBO9781139173179

Inducible SOS response system of DNA repair and mutagenesis in Escherichia coliInternational Journal of Biological Sciences 4:338–344.https://doi.org/10.7150/ijbs.4.338

BookDyadic contests: modelling fights between two individualsIn: Hardy I. C. W, Briffa M, editors. Animal Contests. Cambridge University Press. pp. 5–32.https://doi.org/10.1017/CBO9781139051248.004

Bacterial danger sensingJournal of Molecular Biology 427:3744–3753.https://doi.org/10.1016/j.jmb.2015.09.018

FrequencyDependent selection in bacterial populationsPhilosophical Transactions of the Royal Society of London. Series B, Biological Sciences 319:459–472.https://doi.org/10.1098/rstb.1988.0059

Platforms for antibiotic discoveryNature Reviews Drug Discovery 12:371–387.https://doi.org/10.1038/nrd3975

Purification and genetic characterization of plantaricin NC8, a novel cocultureinducible twopeptide bacteriocin from Lactobacillus plantarum NC8Applied and Environmental Microbiology 69:383–389.https://doi.org/10.1128/AEM.69.1.383389.2003

BookEvolution and the Theory of GamesCambridge University Press.https://doi.org/10.1017/CBO9780511806292

BookMathematical Models of Social EvolutionChicago: University of Chicago Press.https://doi.org/10.7208/chicago/9780226558288.001.0001

Social evolution in multispecies biofilmsPNAS 108 Suppl 2:10839–10846.https://doi.org/10.1073/pnas.1100292108

Type VI secretion system regulation as a consequence of evolutionary pressureJournal of Medical Microbiology 62:663–676.https://doi.org/10.1099/jmm.0.0539830

Emergence of spatial structure in cell groups and the evolution of cooperationPLOS Computational Biology 6:e1000716.https://doi.org/10.1371/journal.pcbi.1000716

Spatial structure, cooperation and competition in biofilmsNature Reviews Microbiology 14:589–600.https://doi.org/10.1038/nrmicro.2016.84

Comparative study of the pln locus of the quorumsensing regulated bacteriocinproducing L. plantarum J51 strainInternational Journal of Food Microbiology 128:390–394.https://doi.org/10.1016/j.ijfoodmicro.2008.08.004

Human skin Microbiota is a rich source of bacteriocinproducing staphylococci that kill human pathogensFEMS Microbiology Ecology 95:fiy241.https://doi.org/10.1093/femsec/fiy241

SoftwareR: A Language and Environment for Statistical ComputingR Foundation for Statistical Computing, Vienna, Austria.

Bacteriocins: evolution, ecology, and applicationAnnual Review of Microbiology 56:117–137.https://doi.org/10.1146/annurev.micro.56.012302.161024

Type VI secretion system effectors: poisons with a purposeNature Reviews Microbiology 12:137–148.https://doi.org/10.1038/nrmicro3185

The evolution of titfortat in Bacteria via the type VI secretion systemNature Communications 11:5395.https://doi.org/10.1038/s4146702019017z

The biogeography of polymicrobial infectionNature Reviews Microbiology 14:93–105.https://doi.org/10.1038/nrmicro.2015.8

Bacillus subtilis antibiotics: structures, syntheses and specific functionsMolecular Microbiology 56:845–857.https://doi.org/10.1111/j.13652958.2005.04587.x

Understanding policing as a mechanism of cheater control in cooperating BacteriaJournal of Evolutionary Biology 32:412–424.https://doi.org/10.1111/jeb.13423

Experimental studies on the ecological role of antibiotic production in BacteriaEvolutionary Ecology 10:405–421.https://doi.org/10.1007/BF01237726

Effect of dispersal and nutrient availability on the competitive ability of toxinproducing yeastProceedings of the Royal Society B: Biological Sciences 275:535–541.https://doi.org/10.1098/rspb.2007.1461

Colicin Vtreated Escherichia coli does not generate membrane potentialJournal of Bacteriology 158:757–759.https://doi.org/10.1128/jb.158.2.757759.1984
Article and author information
Author details
Funding
EPSRC (EP/G50029/1)
 Rene Niehus
European Research Council (787932)
 Kevin R Foster
Wellcome Trust (209397/Z/17/Z)
 Kevin R Foster
Wellcome Trust (Interdisciplinary Fellowship)
 Nuno M Oliveira
Biotechnology and Biological Sciences Research Council (BB/T009098/1)
 Nuno M Oliveira
PKUBaidu (2020BD017)
 Aming Li
College of Engineering, Peking University (Startup funding)
 Aming Li
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
Thank you to Michael Bentley, Jonas Schluter, Kat Coyte, Nicholas Davies, Wook Kim, and Rolf Kümmerli for feedback. RN is funded by the EPSRCfunded Systems Biology Doctoral Training Centre studentship EP/G50029/1. NMO is funded by the Herchel Smith Fellowship. AGF is funded by the Vice Chancellor’s Fellowship from the University of Sheffield. KRF is funded by European Research Council Grant 787932 and Wellcome Trust Investigator award 209397/Z/17/Z. This project is supported by a Templeton World Charity Foundation grant.
Version history
 Received: April 25, 2021
 Accepted: August 1, 2021
 Version of Record published: September 7, 2021 (version 1)
Copyright
© 2021, Niehus et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 4,488
 Page views

 513
 Downloads

 29
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading

 Ecology
The bacterium responsible for a disease that infects citrus plants across Asia facilitates its own proliferation by increasing the fecundity of its host insect.

 Ecology
 Evolutionary Biology
In the unpredictable Anthropocene, a particularly pressing open question is how certain species invade urban environments. Sexbiased dispersal and learning arguably influence movement ecology, but their joint influence remains unexplored empirically, and might vary by space and time. We assayed reinforcement learning in wildcaught, temporarily captive core, middle, or edgerange greattailed grackles—a bird species undergoing urbantracking rapid range expansion, led by dispersing males. We show, across populations, both sexes initially perform similarly when learning stimulusreward pairings, but, when reward contingencies reverse, male—versus female—grackles finish ‘relearning’ faster, making fewer choiceoption switches. How do male grackles do this? Bayesian cognitive modelling revealed male grackles’ choice behaviour is governed more strongly by the ‘weight’ of relative differences in recent foraging payoffs—i.e., they show more pronounced risksensitive learning. Confirming this mechanism, agentbased forward simulations of reinforcement learning—where we simulate ‘birds’ based on empirical estimates of our grackles’ reinforcement learning—replicate our sexdifference behavioural data. Finally, evolutionary modelling revealed natural selection should favour risksensitive learning in hypothesised urbanlike environments: stable but stochastic settings. Together, these results imply risksensitive learning is a winning strategy for urbaninvasion leaders, underscoring the potential for life history and cognition to shape invasion success in humanmodified environments.