Quorums enable optimal pooling of independent judgements in biological systems
Abstract
Collective decisionmaking is ubiquitous, and majorityvoting and the Condorcet Jury Theorem pervade thinking about collective decisionmaking. Thus, it is typically assumed that majorityvoting is the best possible decision mechanism, and that scenarios exist where individuallyweak decisionmakers should not pool information. Condorcet and its applications implicitly assume that only one kind of error can be made, yet signal detection theory shows two kinds of errors exist, ‘false positives’ and ‘false negatives’. We apply signal detection theory to collective decisionmaking to show that majority voting is frequently suboptimal, and can be optimally replaced by quorum decisionmaking. While quorums have been proposed to resolve withingroup conflicts, or manage speedaccuracy tradeoffs, our analysis applies to groups with aligned interests undertaking singleshot decisions. Our results help explain the ubiquity of quorum decisionmaking in nature, relate the use of sub and supermajority quorums to decision ecology, and may inform the design of artificial decisionmaking systems.
Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).
https://doi.org/10.7554/eLife.40368.001Introduction
Effective decisionmaking is essential in all areas of human society and, more generally, for all organisms. A fundamental question in this context is when a group of decisionmakers is superior to individual decisionmakers and viceversa (Galton, 1907; Surowiecki, 2005; Bahrami et al., 2010; Lorenz et al., 2011; Koriat, 2012; Kurvers et al., 2016). Both in human and animal collective decisionmaking, the Condorcet Jury Theorem is one of the key principles guiding our thinking about this question (List, 2004; Hastie and Kameda, 2005; King and Cowlishaw, 2007; Sumpter et al., 2008; AustenSmith and Feddersen, 2009; Conradt and List, 2009; Kao and Couzin, 2014a; Marshall et al., 2017). In a nutshell, for pairwise decision problems (e.g. disease, lie or predator detection), Condorcet’s Jury Theorem states that a group of decisionmakers employing the majority rule is superior to individual decisionmakers in contexts where individuals are relatively accurate (i.e. accuracy >50%); conversely, individual decisionmakers are superior to majority voting groups in contexts where individuals are relatively inaccurate (i.e. accuracy <50%). Consequently, across diverse fields ranging from organismal behaviour and human psychology to political sciences, two heuristics are commonly employed (List, 2004; Hastie and Kameda, 2005; King and Cowlishaw, 2007; Sumpter et al., 2008; Conradt and List, 2009; Kao and Couzin, 2014a): (i) groups of decisionmakers outperform individuals only in contexts where individuals are relatively accurate and (ii) the majority rule is a powerful mechanism to reap the benefits of collective decisionmaking. We here show that both statements are not true, and in doing so explain the ubiquity of quorum decision rules in the collective behaviour of humans and other social organisms (Seeley and Visscher, 2004; Sumpter and Pratt, 2009; Ward et al., 2012; Pratt et al., 2002; RossGillespie and Kümmerli, 2014; Walker et al., 2017; Bousquet et al., 2011).
Over the past few decades, substantial research effort has focussed on two key explicit assumptions underlying Condorcet’s Jury Theorem, independence (i.e. judgments/votes by different members of the group are assumed to be independent from each other) and homogeneity (i.e. all decisionmakers within a group are assumed to be identical, both in competence and in goals) (Kao and Couzin, 2014a; Sumpter and Pratt, 2009; Boland, 1989; Ladha, 1992; Berg, 1993; Marshall et al., 2017). We here focus on a third, implicit, assumption of Condorcet’s Jury Theorem, namely that decisionmakers make only one type of error. This assumption stands in contradiction to the wellknown fact that, when confronted with a pairwise decision problem like the one studied in Condorcet’s Jury Theorem, two different types of error are possible (i.e. false positive and false negative). Surprisingly, up to now, this basic and wellknown feature of pairwise decision problems has not been fully taken into account when investigating Condorcet’s Jury Theorem.
We start by providing a brief summary of the basic model considered in Condorcet’s Jury Theorem. We then introduce an extended model that takes into account the fact that decisionmakers can make two types of errors. Based on this extended model, we then show that Condorcet’s Jury Theorem makes several important predictive errors, which apply to the majority of decision scenarios. Moreover, majority voting is frequently suboptimal, whereas quorumbased voting with an appropriate quorum is always optimal, in that it enables groups to simultaneously maximise true positive rate and minimise false positive rate (Wolf et al., 2013). While an analytical solution for the optimal quorum threshold has been derived before (BenYashar and Nitzan, 1997), dependent on true and false positive rates and key ecological characteristics (i.e. classification error cost, prior probabilities), this analysis treated true and false positive rates as independent of these ecological characteristics, whereas in reality the former depend on the latter. In contrast, here we also make use of signal detection theory to optimise individual decisionmakers, thereby delineating precisely where nonmajority quorums should be used, depending on parameters of the decision ecology. Thus, the simple majority threshold is only a special case of the more general quorum decision mechanism, in which optimal submajority or supermajority quorums are the rule rather than the exception.
Methods
Condorcet’s Jury Theorem: the basic model
Request a detailed protocolCondorcet’s Jury Theorem considers a binary (pairwise) choice situation, in which a decisionmaker can choose between two actions, labelled +1 and –1. Each decisionmaker is characterised by a single parameter a, corresponding to the probability of making a correct decision, or decision accuracy. Importantly, the decision accuracy a of each decisionmaker is assumed to be conditionallyindependent of the realised decisions of all other decisionmakers.
Condorcet’s Jury Theorem now considers a group of identical decisionmakers of size N that performs a majority vote. A simple combinatorial argument shows that if the accuracy of decisionmakers is above 50% (i.e. a > 0.5), then the probability of making a correct choice increases with increasing group size and asymptotically approaches 1 (Boland, 1989). Conversely, if the accuracy of individual decisionmakers is below 50% (i.e. a < 0.5), then the probability of making a correct choice decreases with increasing group size and asymptotically approaches 0 (King and Cowlishaw, 2007). This is because, as group size increases, the probability of the more probable decision (+ or –) also being the majority decision rapidly increases towards one.
These results have led to three key interpretations: first, pooling independent judgements is beneficial (i.e. improves decision accuracy) whenever individuals are relatively good decisionmakers (a > 0.5) (List, 2004; King and Cowlishaw, 2007; Novaes Tump et al., 2018). Second, pooling judgements is detrimental (i.e. decreases decision accuracy) whenever individuals are poor decisionmakers (a < 0.5) (List, 2004; King and Cowlishaw, 2007; Novaes Tump et al., 2018). Third, the majority rule is the appropriate mechanism to reap the benefits of collective decisionmaking (List, 2004; Hastie and Kameda, 2005).
In the following, we show that each of these interpretations is incorrect. In particular, we show that (i) in cases where decisionmakers are good (a > 0.5) majority decisions may decrease decision accuracy, (ii) in cases where decisionmakers are poor (a < 0.5) majority decisions may increase decision accuracy, and (iii) pooling independent decisions is always beneficial as long as an appropriate quorumbased decision rule is used. Taken together, these results show that, for a large proportion of decision scenarios, the simple majority decision rule performs poorly and gives incorrect predictions about group decision accuracy.
Condorcet’s Jury Theorem: an extended model
Request a detailed protocolCondorcet’s Jury Theorem implicitly assumes that decisionmakers can make only one error, that is, the probability of making an incorrect decision is 1 – a. This is in contrast to the wellknown fact that – in pairwise decision problems – decisionmakers can make two types of errors, false positives and false negatives (Green and Swets, 1966; Swets, 1988). For example, an animal under predation risk may run away in the absence of a predator (false positive) or it may not run away in the presence of a predator (false negative) (Trimmer et al., 2008). Similarly, a doctor screening for a disease may diagnose in the absence of a disease (false positive) or not diagnose in the presence of a disease (false negative) (Wolf et al., 2013). As we discuss below, the implicit assumption of Condorcet’s Jury Theorem is equivalent to assuming that decisionmakers have an identical probability of committing the two errors – this is an important assumption that does not reflect the vast majority of real world decisions.
We here consider an extension of the abovedescribed model, which takes into account the fact that decisionmakers can make two types of errors. Again, decisionmakers can choose between two actions, +1 and –1. Unlike in the basic model above, however, and consistent with standard decision theory for pairwise decisions, we now assume that the world can be in two states, state + and state –, corresponding to, for example, the presence and absence of a predator or the presence and the absence of a disease; state + holds with probability p and state – thus holds with probability 1 – p. Action +1 is the better choice in state + (i.e. it achieves a higher payoff), while action –1 is the better choice in state –; for example, running away is better than staying in the presence of a predator, while staying is better than running away in the absence of a predator. Consequently, and in contrast to the basic Condorcet model above, each decisionmaker is now characterised by two parameters a_{+} and a_{–}, corresponding to the probabilities of making correct decisions in state + and state –, respectively; this inevitably implies that individuals make two types of errors. As in the simple model above, where decisionmakers have equal accuracies a and are independent, for any individual in a group of N individuals the probability of making a correct decision based on the true state of the world is conditionallyindependent of the probabilities of other group members making the correct decision, given that same state of the world.
To relate our analysis to predictions made by applying Condorcet, we must define expected individual accuracy a, as used in Condorcet, in terms of our state probability p and statewise accuracy parameters a_{+} and a_{–}. The expected individual accuracy is thus
From Equation 1 we can see that Condorcet implicitly assumes that accuracies in the two states of the world are equal since then p disappears from the equation; as shown in Supplementary Information, this occurs when both states of the world are equally likely, and the costs and benefits of classifications in the two states of the world are equal, although asymmetric decision problems can also result in equal accuracies (as can be confirmed with reference to Supplementary Information for Figure 1). Later, we explain how signal detection theory determines optimal values of a_{+} and a_{–} for a given decision scenario where these assumptions are violated.
Results
We are now ready to formalise when Condorcet gives incorrect predictions, and when simple majority voting is suboptimal. We begin by determining optimal values of a_{+} and a_{–}. Given the prior probability p, the cost and benefits associated with the two states of the world, and the statedependent characteristics of the cue(s) decisionmakers base their decisions on, the optimal realised individual accuracies a_{+} and a_{–} are derived by solving a signal detection problem (Figure 1). Signal detection theory has been applied repeatedly to hypothetical and real world decision problems, for example predator detection by foraging animals (Trimmer et al., 2008; Trimmer et al., 2017); diagnostic decisionmaking (Kurvers et al., 2016; Wolf et al., 2015); and lie detection (Klein and Epley, 2015). As described in the appendix, for simplicity and tractability our analysis is conducted for the simplest signal detection problem, determining which of two normal distributions a single scalar random variable is drawn from; this could, in the example of a predator detection problem, be the instantaneous volume of a sound heard by a forager. In this case signal detection theory enables us to find a Receiver Operating Characteristic (ROC) curve of optimal a_{+}, a_{–} value pairs, dependent on decision difficulty (Figure 1; Green and Swets, 1966; Shettleworth, 2010); the optimal point of this curve gives us a unique pair of a_{+} and a_{–} values dependent on costs and benefits of different decision outcomes, and the prior probability of the + state, p.
We next make the observation that, to simultaneously improve both true positive and false positive rates, a group must choose a quorum q that lies between these two values (Wolf et al., 2013); that is, an optimal group must choose q such that
The intuition behind this result is that group accuracy will converge on the appropriate accuracy for the true state of the world, as group size increases; thus, by setting a quorum between these two accuracies the true state of the world can be determined with high probability for sufficiently large groups. For further details see (Wolf et al., 2013). Since the simple majority quorum q = 1/2, assumed by Condorcet, only satisfies this inequality when both ${a}_{+}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$, when either of these conditions are violated then both simple majority decisions and Condorcetbased reasoning will be deficient. Furthermore, we should never see true positive rate ($a}_{+$) less than false positive rate ($1{a}_{}$) (dark grey region of ROC space in Figure 1), since such a decisionmaker could simultaneously improve both their true and false positive rates simply by inverting their decisions and moving above the diagonal. Therefore, the ROC space is divided into two meaningful regions: in the first ${a}_{+}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}<\phantom{\rule{thinmathspace}{0ex}}1/2$ (white region in Figure 1), so simple majority voting is asymptoticallyoptimal as group size increases, and Condorcetbased predictions are correct. In the second, ${a}_{+}\phantom{\rule{thinmathspace}{0ex}}<\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$, or ${a}_{+}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}<\phantom{\rule{thinmathspace}{0ex}}1/2,$ while ensuring $a}_{+}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1{a}_{$ (light grey regions in Figure 1); in these regions simple majority decisions will be suboptimal, and Condorcetbased reasoning will be erroneous.
We now describe the systematic errors Condorcet leads to when faced with these decision scenarios. These errors can also be described in terms of false positive (frequently referred to as type I) and false negative (type II) errors, in predicting the performance of the Condorcet majority rule, hence we label the errors in such terms.
Error Ia: Condorcet predicts group accuracy approaches 1, but majority groups do not
This error occurs when majority voting is suboptimal (${a}_{+}\phantom{\rule{thinmathspace}{0ex}}<\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$, or ${a}_{+}\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$ and ${a}_{}\phantom{\rule{thinmathspace}{0ex}}<\phantom{\rule{thinmathspace}{0ex}}1/2$; light grey regions in Figure 1, as described above), and when expected individual accuracy $a\phantom{\rule{thinmathspace}{0ex}}>\phantom{\rule{thinmathspace}{0ex}}1/2$, since under this last condition individuals on average make more correct decisions than incorrect decisions, and Condorcet thus predicts that group accuracy approaches one as group size increases (King and Cowlishaw, 2007; Boland, 1989). From Equation 1 this requirement is thus that
While Condorcet predicts group accuracy (denoted $\stackrel{}{a}$) approaches 1, that is
in fact the majority quorum q = 1/2 is either below 1 – ${a}_{}$, or above ${a}_{+}$, contra inequality 2; thus the group converges to making completely correct choices in one state of the world, and completely incorrect choices in the other state of the world (Wolf et al., 2013). Hence, group accuracy converges to
The wide range of decision scenarios in which Condorcet makes this predictive error are illustrated in Figure 2a. An example of this error is presented in Figure 3a; Figure 3b illustrates how the error can be avoided by choosing an appropriate quorum, in this case a supermajority quorum.
Error Ib: Condorcet predicts group accuracy approaches 1, but majority groups are worse than individuals
In error Ia, while group accuracy does not converge to one with increasing group size, nothing is said about whether or not groups are better than individuals. Error Ib refines error Ia, by showing that there are cases where Condorcet predicts group accuracy approaching 1, but groups actually have lower accuracy than individuals. These cases are found by refining the conditions given in error Ia to include the additional condition that the group accuracy converged to (Equation 5 above) is less than individual expected accuracy (Equation 1). This gives the conditions
The wide range of decision scenarios in which Condorcet makes this predictive error are illustrated in Figure 2b. An example of this error is presented in Figure 3c; Figure 3d illustrates how the error can be avoided by choosing an appropriate quorum, in this case a supermajority quorum.
Error II: Condorcet predicts group accuracy approaches 0, but majority groups do not
In error II individual expected accuracy is below 1/2, thus Condorcet predicts that group accuracy should converge to 0; while groups using the majority decision rule do decrease in accuracy, they converge to a nonzero group accuracy given by Equation 5 above. To find cases where this occurs we simply solve for when individual expected accuracy a < 1/2. From Equation 1 this gives us the conditions
Decision scenarios in which Condorcet makes this predictive error are illustrated in Figure 2c. An example of this error is presented in Figure 3e; Figure 3f illustrates how the error can be avoided by choosing an appropriate quorum, in this case a submajority quorum.
Note that it is not possible to find an ‘Error IIb’ case that parallels Error Ib; that is if Condorcet predicts that group accuracy approaches 0, majority groups will always be worse than individuals and never better, even if their group accuracy remains positive. This is because it is not possible simultaneously to satisfy the conditions just given (inequalities 8 and 9), and the opposite of the conditions (inequalities 6 and 7) given in Error Ib (i.e. the conditions that group accuracy converged to is greater than individual expected accuracy), as can be confirmed by algebra.
Majority voting is usually suboptimal
Combining the cases in which Condorcet makes one of the above described predictive errors, Figure 2d illustrates when Condorcet will make at least one error in predicting the performance of decisionmaking groups using majority decisionmaking. This set also corresponds to the set of decision scenarios in which majority decisionmaking is suboptimal, and is outperformed by an appropriately set sub or supermajority quorum. Figure 2d shows that Condorcet is optimal in far fewer decision scenarios than those in which it is outperformed by an appropriate quorum rule. Furthermore, our analysis also relates decisionecology to the requirement for sub or supermajority quorums. From inequality (2), submajority quorums are required whenever a_{+ }< 1/2, which corresponds to the upperleft area of Figure 2d; in contrast, supermajority quorums are required whenever 1 – a_{+ }> 1/2, which corresponds with the lowerright area of Figure 2d (see Supplementary Information). Thus, whenever the positive state of the world + is rarer, and/or false positives are relatively expensive compared to false negatives, then a submajority quorum should typically be employed, while the converse holds for supermajority quorums.
Discussion
We have shown that simple majoritybased collective decisions are often suboptimal, and that consequently sub or supermajority quorums should frequently be employed by groups of ‘likeminded’ individuals combining independent decisions. Our results are important for two reasons. First, the majoritybased decision rule, and Condorcetbased reasoning, is widespread in several major branches of collective decision theory. In the animal behaviour community the Condorcet prescription on individual accuracy exceeding ½ has been used to recommend when opinions should be pooled (King and Cowlishaw, 2007), and when experts should be favoured over group opinions (Katsikopoulos and King, 2010). Other authors have also invoked Condorcet and majority voting as the goldstandard for collective decisionmaking (Hastie and Kameda, 2005; Kao and Couzin, 2014a; Kao et al., 2014b; Miller et al., 2013). These results have been useful in highlighting the benefits of information pooling in collective decisions, but such studies implicitly neglect the reality that most decisions have two types of errors. Second, while quorums have been widely studied in collective decision theory, we here present a comprehensive theory that may help explain their prevalence. Sub and supermajority quorums have been considered theoretically (Sumpter and Pratt, 2009); Sumpter and Pratt implicitly assume only one type of error need be considered, and proceed from that point with their analysis, referring to quorum functions such as managing speedaccuracy tradeoffs (Marshall et al., 2009). Ward et al. (2008) consider quorum use for facilitating information transfer in shoaling fish, yet ignore the possibility of different error types despite their great asymmetry in the scenario studied, predator detection. Conradt and Roper (2005) in their review refer to true and false positives, but in explaining quorum usage refer back to earlier analysis as a mechanism to avoid extreme group decisions where individual fitness interests do not completely align (Conradt and Roper, 2003). However List (2004) as well as Conradt and List (2009) noted the effect of cost and prior asymmetry on quorum usage, referring back to earlier political science results (BenYashar and Nitzan, 1997) discussed below. We also note that convergence on an intermediate group accuracy between 0 and 1 has previously been observed, without consideration of signal detection theory; the result presented in Figure 1b of Kao and Couzin (2014a) occurs for a similar reason to our error Ia, in that inappropriate use of a majority decision rule leads increasing group size to result in group accuracy converging on a parameter of the decision problem, in their case the reliability of a low correlation cue.
In contrast to these previous analyses here we have shown the fundamental importance of quorums in one of the simplest possible collective behaviour scenarios, singleshot collective decisions in homogenous groups, where individuals’ interests are aligned, and decisionmaking abilities do not differ. Thus, we might expect the use of quorums to be widespread in the natural world, even in the simplest of decisions. The widespread use of quorum sensing in bacteria provides evidence of this (Gross, 2017), and may prove a particularly good testbed for our theory given its binary nature and asymmetric state priors and error costs, although evolutionary conflicts of interests within bacterial communities may result in confounds. Moreover, as humans have been shown to employ quorum rules and adaptively adjust the associated quorum thresholds, human decisionmaking experiments may also provide a powerful approach to test our predictions (Kurvers et al., 2014; Clément et al., 2015).
It is surprising that signal detection theory has seen relatively scant application to collective decisionmaking. Wolf et al. (2013) noted the potential for different error types in identifying how quorums can improve collective decision accuracy; while motivated by signal detection theory they did not directly apply it to optimise the individual decisionmakers and relate this back to group behaviour. Kirstein and von Wangenheim, 2010 also noted the possibility for independent error types, again with reference to signal detection theory; they noted the potential for Condorcet to make the same qualitatively incorrect predictions that we note here, however they did not apply the relevant theory to delineate the situations under which Condorcet reasoning is incorrect, nor did they consider the possibility for quorumbased decision rules to rescue these situations. Sorkin et al. (2001) applied signal detection theory to Condorcetlike models with varying supermajority quorums, but did not find the mechanism by which group decisions can be optimised (Wolf et al., 2013). Laan et al. (2017) noted that Condorcet and majority voting can be suboptimal, and considered ways in which voting can be improved; while they discussed signal detection theory and voting mechanisms they neither explicitly considered error types, nor quorum thresholds, focussing mainly on correlations between decisionmakers and the impact of the cost function used, as well as suggesting a datadriven machine learning approach to improving collective decisionmaking rules. The results that most closely anticipate ours are those of BenYashar and Nitzan (1997), who analytically solved for the general optimal decision rule by recognising both error cost and prior asymmetry, as well as simultaneously considering the case of variable individual decisionability (e.g. Marshall et al., 2017). These results describe the relationship between prior asymmetry and optimal quorum threshold, and error cost and optimal quorum threshold. However, because they did not apply signal detection theory to optimise individual agents’ decisions (i.e. they treated true and false positive rates of individuals as independent from the ecological parameters error cost and prior asymmetry), they were unable quantitatively to uncover the complex nonlinear relationship between these three quantities (Figure 2).
In contrast to earlier work, by applying signal detection theory we have simultaneously shown here both how fragile Condorcet and majorityvoting are, and how the use of sub or supermajority quorums should relate to decision ecology. Although simple collective behaviour models have been well studied and highly influential, our results, and others relaxing other assumptions of such models (Marshall et al., 2017), indicate the subtlety that may be revealed in collective decisionmaking by a richer consideration of individual decision theory. Other approaches to such problems should be pursued in the future. For example, it is possible to optimise individual quorums (BenYashar and Nitzan, 1997) rather than simply set them within a suitable interval as we do here, thereby giving greater benefits to smaller groups. Similarly, rather than apply signal detection theory one could apply Bayesian decisiontheory (e.g. PérezEscudero and de Polavieja, 2011; Arganda et al., 2012; PérezEscudero and de Polavieja, 2017), thereby attempting to deal with further complexities such as nonindependence of individual decisions. In many scenarios errors and correct decisions may be correlated although even when multiple individuals observe the same stimulus their information can be considered independent due to sensory noise (Marshall et al., 2017). We believe that our simple approach has, however, the benefit of tractability while still revealing the complexity of collective decisionmaking even in the simplified scenario considered.
Our work takes inspiration from political science and decision theory to address questions in behavioural ecology, but may additionally have the potential to inform work in the design of artificial decisionmaking systems, machine learning and robotics. For example, in the field of ensemble learning, in which predictions from multiple weak classifiers such as neural networks are combined to improve decision accuracy, variable quorums, referred to as ‘threshold shift’, are used (e.g. Dmochowski et al., 2010). However majority voting is still among the simplest and most ubiquitous vote fusion rules discussed (Sagi and Rokach, 2018; Krawczyk et al., 2017). Hence, we suggest that the simple perspective on how to combine votes presented here may also yield technological insight.
Appendix 1
Signal detection theory reveals cases where Condorcet Predictions are incorrect
To determine if a group of optimal decisionmakers could exist such that simple application of Condorcet would lead to erroneous predictions, but a quorum rule would allow optimal opinion pooling, we consider groups of identicallycapable individual decisionmakers (as assumed in Condorcet), modelled as making optimal decisions using signal detection theory in order to classify continuous signals arising from one of two possible normal signal distributions of equal variance (Green and Swets, 1966). That is, decisionmakers are faced with a signal
and must choose an optimal signal threshold, x, in order to classify signals as being drawn from either of the two possible normal distributions. Each distribution has a different mean (${\mu}_{+}$ versus ${\mu}_{}$) but the same standard deviation ($\sigma $). In a natural setting the signal could represent information as to whether a predator is present or not, for example, with the two states of the world, predator present versus predator absent, having different distributions for this signal.
The optimal decision threshold x is chosen to minimise the expected loss for the decisionmaker (or maximise the expected gain). The expected loss from a decision is
where ${C}_{TP}$, ${C}_{FN}$, ${C}_{TN}$ and $C}_{FP$ are respectively the costs of true positives (correctly classifying state +), false negatives (incorrectly classifying state –), true negatives (correctly classifying state –) and false positives (incorrectly classifying state +). Thus, an optimal decisionmaker should minimise (equation A.2) by appropriately choosing the decision threshold x. Since, given x,
where $\mathrm{\Phi}$ is the cumulative distribution function for the normal distribution, the optimal x can be found by substituting (equation A.3) and (equation A.4) into (equation A.2), differentiating the resulting equation and solving for zero (Green and Swets, 1966). Note that the optimal threshold x, and thus the optimal accuracies under state + and state – of the world, ${a}_{+}$and ${a}_{}$ respectively, are affected both by class imbalance ($p\ne 1/2$), and by asymmetric error costs (${C}_{TP}{C}_{FN}\ne {C}_{TN}{C}_{FP}$).
Data availability
All data generated or analysed during this study are included in the manuscript and supporting files. Source data files have been provided for Figures 1, 2 and 3.
References

Information aggregation and communication in committeesPhilosophical Transactions of the Royal Society B: Biological Sciences 364:763–769.https://doi.org/10.1098/rstb.2008.0256

The optimal decision rule for FixedSize committees in dichotomous choice situations: the general resultInternational Economic Review 38:175–186.https://doi.org/10.2307/2527413

Condorcet's jury theorem, dependency among jurorsSocial Choice and Welfare 10:87–95.https://doi.org/10.1007/BF00187435

Majority systems and the condorcet jury theoremThe Statistician 38:181–189.https://doi.org/10.2307/2348873

Moving calls: a vocal mechanism underlying quorum decisions in cohesive groupsProceedings of the Royal Society B: Biological Sciences 278:1482–1488.https://doi.org/10.1098/rspb.2010.1739

Group decisions in humans and animals: a surveyPhilosophical Transactions of the Royal Society B: Biological Sciences 364:719–742.https://doi.org/10.1098/rstb.2008.0276

Consensus decision making in animalsTrends in Ecology & Evolution 20:449–456.https://doi.org/10.1016/j.tree.2005.05.008

Maximum likelihood in costsensitive learning: model specification, approximations, and upper boundsJournal of Machine Learning Research 11:3313–3332.

Shining new light on quorum sensingCurrent Biology 27:R1293–R1296.https://doi.org/10.1016/j.cub.2017.11.068

The robust beauty of majority rules in group decisionsPsychological Review 112:494–508.https://doi.org/10.1037/0033295X.112.2.494

Collective learning and optimal consensus decisions in social animal groupsPLoS Computational Biology 10:e1003762.https://doi.org/10.1371/journal.pcbi.1003762

Decision accuracy in complex environments is often maximized by small group sizesProceedings of the Royal Society B: Biological Sciences 281:20133305.https://doi.org/10.1098/rspb.2013.3305

MAGKS Papers on EconomicsA Generalized Condorcet Jury Theorem with Two Independent Probabilities of Error, MAGKS Papers on Economics, PhilippsUniversität Marburg, Faculty of Business Administration and Economics, Department of Economics.

Ensemble learning for data stream analysis: a surveyInformation Fusion 37:132–156.https://doi.org/10.1016/j.inffus.2017.02.004

Humans use social information to adjust their quorum thresholds adaptively in a simulated predator detection experimentBehavioral Ecology and Sociobiology 68:449–456.https://doi.org/10.1007/s0026501316596

Rescuing collective wisdom when the average group opinion is wrongFrontiers in Robotics and AI 4:.https://doi.org/10.3389/frobt.2017.00056

The Condorcet jury theorem, free speech, and correlated votesAmerican Journal of Political Science 36:617–634.https://doi.org/10.2307/2111584

Democracy in animal groups: a political science perspectiveTrends in Ecology & Evolution 19:168–169.https://doi.org/10.1016/j.tree.2004.02.004

On optimal decisionmaking in brains and social insect coloniesJournal of the Royal Society Interface 6:1065–1074.https://doi.org/10.1098/rsif.2008.0511

Individual ConfidenceWeighting and group DecisionMakingTrends in Ecology & Evolution 32:636–645.https://doi.org/10.1016/j.tree.2017.06.004

Individuals fail to reap the collective benefits of diversity because of overreliance on personal informationJournal of the Royal Society Interface 15:20180155.https://doi.org/10.1098/rsif.2018.0155

Collective animal behavior from bayesian estimation and probability matchingPLoS Computational Biology 7:e1002282.https://doi.org/10.1371/journal.pcbi.1002282

Adversity magnifies the importance of social information in decisionmakingJournal of the Royal Society Interface 14:20170748.https://doi.org/10.1098/rsif.2017.0748

Quorum sensing, recruitment, and collective decisionmaking during colony emigration by the ant leptothorax albipennisBehavioral Ecology and Sociobiology 52:117–127.https://doi.org/10.1007/s002650020487x

Collective decisionmaking in microbesFrontiers in Microbiology 5:.https://doi.org/10.3389/fmicb.2014.00054

Ensemble learning: a surveyWiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8:e1249.https://doi.org/10.1002/widm.1249

Quorum sensing during nestsite selection by honeybee swarmsBehavioral Ecology and Sociobiology 56:594–601.https://doi.org/10.1007/s0026500408145

Signaldetection analysis of group decision makingPsychological Review 108:183–203.https://doi.org/10.1037/0033295X.108.1.183

Consensus decision making by fishCurrent Biology 18:1773–1777.https://doi.org/10.1016/j.cub.2008.09.064

Quorum responses and consensus decision makingPhilosophical Transactions of the Royal Society B: Biological Sciences 364:743–753.https://doi.org/10.1098/rstb.2008.0204

Measuring the accuracy of diagnostic systemsScience 240:1285–1293.https://doi.org/10.1126/science.3287615

Mammalian choices: combining fastbutinaccurate and slowbutaccurate decisionmaking systemsProceedings of the Royal Society B: Biological Sciences 275:2353–2361.https://doi.org/10.1098/rspb.2008.0417

The erroneous signals of detection theoryProceedings of the Royal Society B: Biological Sciences 284:20171852.https://doi.org/10.1098/rspb.2017.1852

Sneeze to leave: African wild dogs ( Lycaon pictus ) use variable quorum thresholds facilitated by sneezes in collective decisionsProceedings of the Royal Society B: Biological Sciences 284:20170347.https://doi.org/10.1098/rspb.2017.0347

Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positivesProceedings of the Royal Society B: Biological Sciences 280:20122777.https://doi.org/10.1098/rspb.2012.2777
Article and author information
Author details
Funding
H2020 European Research Council (647704)
 James AR Marshall
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
We thank Gavin Brown and Nikolaos Nikolaou for helpful discussions on ensemble learning theory, and Andrew King, Gonzalo de Polavieja and an anonymous reviewer for helpful comments during the review process. JARM was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement number 647704).
Version history
 Received: July 24, 2018
 Accepted: January 8, 2019
 Version of Record published: February 13, 2019 (version 1)
Copyright
© 2019, Marshall et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 1,268
 views

 167
 downloads

 19
 citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading

 Ecology
Over two decades ago, an intercropping strategy was developed that received critical acclaim for synergizing food security with ecosystem resilience in smallholder farming. The push–pull strategy reportedly suppresses lepidopteran pests in maize through a combination of a repellent intercrop (push), commonly Desmodium spp., and an attractive, border crop (pull). Key in the system is the intercrop’s constitutive release of volatile terpenoids that repel herbivores. However, the earlier described volatile terpenoids were not detectable in the headspace of Desmodium, and only minimally upon herbivory. This was independent of soil type, microbiome composition, and whether collections were made in the laboratory or in the field. Furthermore, in oviposition choice tests in a wind tunnel, maize with or without an odor background of Desmodium was equally attractive for the invasive pest Spodoptera frugiperda. In search of an alternative mechanism, we found that neonate larvae strongly preferred Desmodium over maize. However, their development stagnated and no larva survived. In addition, older larvae were frequently seen impaled and immobilized by the dense network of silicafortified, nonglandular trichomes. Thus, our data suggest that Desmodium may act through intercepting and decimating dispersing larval offspring rather than adult deterrence. As a hallmark of sustainable pest control, maize–Desmodium push–pull intercropping has inspired countless efforts to emulate stimulodeterrent diversion in other cropping systems. However, detailed knowledge of the actual mechanisms is required to rationally improve the strategy, and translate the concept to other cropping systems.

 Ecology
The bacterium responsible for a disease that infects citrus plants across Asia facilitates its own proliferation by increasing the fecundity of its host insect.