Quorums enable optimal pooling of independent judgements in biological systems

Effective decision-making is essential in all areas of human society and, more generally, for all organisms. A fundamental question in this context is when a group of decision-makers is superior to individual decision-makers and vice-versa (Galton, 1907; Surowiecki, 2005; Bahrami et al., 2010; Lorenz et al., 2011; Koriat, 2012; Kurvers et al., 2016). Both in human and animal collective decision-making, the Condorcet Jury Theorem is one of the key principles guiding our thinking about this question (List, 2004; Hastie and Kameda, 2005; King and Cowlishaw, 2007; Sumpter et al., 2008; Austen-Smith and Feddersen, 2009; Conradt and List, 2009; Kao and Couzin, 2014a; Marshall et al., 2017). In a nutshell, for pairwise decision problems (e.g. disease-, lie- or predator detection), Condorcet’s Jury Theorem states that a group of decision-makers employing the majority rule is superior to individual decision-makers in contexts where individuals are relatively accurate (i.e. accuracy >50%); conversely, individual decision-makers are superior to majority voting groups in contexts where individuals are relatively inaccurate (i.e. accuracy <50%). Consequently, across diverse fields ranging from organismal behaviour and human psychology to political sciences, two heuristics are commonly employed (List, 2004; Hastie and Kameda, 2005; King and Cowlishaw, 2007; Sumpter et al., 2008; Conradt and List, 2009; Kao and Couzin, 2014a): (i) groups of decision-makers outperform individuals only in contexts where individuals are relatively accurate and (ii) the majority rule is a powerful mechanism to reap the benefits of collective decision-making. We here show that both statements are not true, and in doing so explain the ubiquity of quorum decision rules in the collective behaviour of humans and other social organisms (Seeley and Visscher, 2004; Sumpter and Pratt, 2009; Ward et al., 2012; Pratt et al., 2002; Ross-Gillespie and Kümmerli, 2014; Walker et al., 2017; Bousquet et al., 2011).

Over the past few decades, substantial research effort has focussed on two key explicit assumptions underlying Condorcet’s Jury Theorem, independence (i.e. judgments/votes by different members of the group are assumed to be independent from each other) and homogeneity (i.e. all decision-makers within a group are assumed to be identical, both in competence and in goals) (Kao and Couzin, 2014a; Sumpter and Pratt, 2009; Boland, 1989; Ladha, 1992; Berg, 1993; Marshall et al., 2017). We here focus on a third, implicit, assumption of Condorcet’s Jury Theorem, namely that decision-makers make only one type of error. This assumption stands in contradiction to the well-known fact that, when confronted with a pairwise decision problem like the one studied in Condorcet’s Jury Theorem, two different types of error are possible (i.e. false positive and false negative). Surprisingly, up to now, this basic and well-known feature of pairwise decision problems has not been fully taken into account when investigating Condorcet’s Jury Theorem.

We start by providing a brief summary of the basic model considered in Condorcet’s Jury Theorem. We then introduce an extended model that takes into account the fact that decision-makers can make two types of errors. Based on this extended model, we then show that Condorcet’s Jury Theorem makes several important predictive errors, which apply to the majority of decision scenarios. Moreover, majority voting is frequently suboptimal, whereas quorum-based voting with an appropriate quorum is always optimal, in that it enables groups to simultaneously maximise true positive rate and minimise false positive rate (Wolf et al., 2013). While an analytical solution for the optimal quorum threshold has been derived before (Ben-Yashar and Nitzan, 1997), dependent on true and false positive rates and key ecological characteristics (i.e. classification error cost, prior probabilities), this analysis treated true and false positive rates as independent of these ecological characteristics, whereas in reality the former depend on the latter. In contrast, here we also make use of signal detection theory to optimise individual decision-makers, thereby delineating precisely where non-majority quorums should be used, depending on parameters of the decision ecology. Thus, the simple majority threshold is only a special case of the more general quorum decision mechanism, in which optimal sub-majority or super-majority quorums are the rule rather than the exception.

We are now ready to formalise when Condorcet gives incorrect predictions, and when simple majority voting is suboptimal. We begin by determining optimal values of a₊ and a_–. Given the prior probability p, the cost and benefits associated with the two states of the world, and the state-dependent characteristics of the cue(s) decision-makers base their decisions on, the optimal realised individual accuracies a₊ and a_– are derived by solving a signal detection problem (Figure 1). Signal detection theory has been applied repeatedly to hypothetical and real world decision problems, for example predator detection by foraging animals (Trimmer et al., 2008; Trimmer et al., 2017); diagnostic decision-making (Kurvers et al., 2016; Wolf et al., 2015); and lie detection (Klein and Epley, 2015). As described in the appendix, for simplicity and tractability our analysis is conducted for the simplest signal detection problem, determining which of two normal distributions a single scalar random variable is drawn from; this could, in the example of a predator detection problem, be the instantaneous volume of a sound heard by a forager. In this case signal detection theory enables us to find a Receiver Operating Characteristic (ROC) curve of optimal a₊, a_– value pairs, dependent on decision difficulty (Figure 1; Green and Swets, 1966; Shettleworth, 2010); the optimal point of this curve gives us a unique pair of a₊ and a_– values dependent on costs and benefits of different decision outcomes, and the prior probability of the + state, p.

We next make the observation that, to simultaneously improve both true positive and false positive rates, a group must choose a quorum q that lies between these two values (Wolf et al., 2013); that is, an optimal group must choose q such that

The intuition behind this result is that group accuracy will converge on the appropriate accuracy for the true state of the world, as group size increases; thus, by setting a quorum between these two accuracies the true state of the world can be determined with high probability for sufficiently large groups. For further details see (Wolf et al., 2013). Since the simple majority quorum q = 1/2, assumed by Condorcet, only satisfies this inequality when both ${\displaystyle a_{+}>1/2}$ and ${\displaystyle a_{-}>1/2}$, when either of these conditions are violated then both simple majority decisions and Condorcet-based reasoning will be deficient. Furthermore, we should never see true positive rate (${\displaystyle a_{+}}$) less than false positive rate (${\displaystyle 1-a_{-}}$) (dark grey region of ROC space in Figure 1), since such a decision-maker could simultaneously improve both their true and false positive rates simply by inverting their decisions and moving above the diagonal. Therefore, the ROC space is divided into two meaningful regions: in the first ${\displaystyle a_{+}>1/2}$ and ${\displaystyle a_{-}<1/2}$ (white region in Figure 1), so simple majority voting is asymptotically-optimal as group size increases, and Condorcet-based predictions are correct. In the second, ${\displaystyle a_{+}<1/2}$ and ${\displaystyle a_{-}>1/2}$, or ${\displaystyle a_{+}>1/2}$ and ${\displaystyle a_{-}<1/2,}$ while ensuring ${\displaystyle a_{+}>1-a_{-}}$ (light grey regions in Figure 1); in these regions simple majority decisions will be sub-optimal, and Condorcet-based reasoning will be erroneous.

We now describe the systematic errors Condorcet leads to when faced with these decision scenarios. These errors can also be described in terms of false positive (frequently referred to as type I) and false negative (type II) errors, in predicting the performance of the Condorcet majority rule, hence we label the errors in such terms.

Error Ia: Condorcet predicts group accuracy approaches 1, but majority groups do not

This error occurs when majority voting is sub-optimal (${\displaystyle a_{+}<1/2}$ and ${\displaystyle a_{-}>1/2}$, or ${\displaystyle a_{+}>1/2}$ and ${\displaystyle a_{-}<1/2}$; light grey regions in Figure 1, as described above), and when expected individual accuracy ${\displaystyle a>1/2}$, since under this last condition individuals on average make more correct decisions than incorrect decisions, and Condorcet thus predicts that group accuracy approaches one as group size increases (King and Cowlishaw, 2007; Boland, 1989). From Equation 1 this requirement is thus that

${\displaystyle p\left(a_{+}-a_{-}\right)+a_{-}>1/2}$

While Condorcet predicts group accuracy (denoted $\stackrel{-}{a}$) approaches 1, that is

(4) $\underset{n\to \infty }{\lim }\stackrel{-}{a}=1$

in fact the majority quorum q = 1/2 is either below 1 – $a_{-}$, or above $a_{+}$, contra inequality 2; thus the group converges to making completely correct choices in one state of the world, and completely incorrect choices in the other state of the world (Wolf et al., 2013). Hence, group accuracy converges to

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}\stackrel{-}{a}=\{\begin{array}{cc}p& \text{if }{a}_{+}>\frac{1}{2}\text{and }{a}_{-}<\frac{1}{2}\\ 1-p& \text{if }{a}_{+}<\frac{1}{2}\text{and }{a}_{-}>\frac{1}{2}\end{array}}$

The wide range of decision scenarios in which Condorcet makes this predictive error are illustrated in Figure 2a. An example of this error is presented in Figure 3a; Figure 3b illustrates how the error can be avoided by choosing an appropriate quorum, in this case a super-majority quorum.

Figure 2

Download asset Open asset

Figure 3

Download asset Open asset

We have shown that simple majority-based collective decisions are often suboptimal, and that consequently sub- or super-majority quorums should frequently be employed by groups of ‘like-minded’ individuals combining independent decisions. Our results are important for two reasons. First, the majority-based decision rule, and Condorcet-based reasoning, is widespread in several major branches of collective decision theory. In the animal behaviour community the Condorcet prescription on individual accuracy exceeding ½ has been used to recommend when opinions should be pooled (King and Cowlishaw, 2007), and when experts should be favoured over group opinions (Katsikopoulos and King, 2010). Other authors have also invoked Condorcet and majority voting as the gold-standard for collective decision-making (Hastie and Kameda, 2005; Kao and Couzin, 2014a; Kao et al., 2014b; Miller et al., 2013). These results have been useful in highlighting the benefits of information pooling in collective decisions, but such studies implicitly neglect the reality that most decisions have two types of errors. Second, while quorums have been widely studied in collective decision theory, we here present a comprehensive theory that may help explain their prevalence. Sub- and super-majority quorums have been considered theoretically (Sumpter and Pratt, 2009); Sumpter and Pratt implicitly assume only one type of error need be considered, and proceed from that point with their analysis, referring to quorum functions such as managing speed-accuracy trade-offs (Marshall et al., 2009). Ward et al. (2008) consider quorum use for facilitating information transfer in shoaling fish, yet ignore the possibility of different error types despite their great asymmetry in the scenario studied, predator detection. Conradt and Roper (2005) in their review refer to true and false positives, but in explaining quorum usage refer back to earlier analysis as a mechanism to avoid extreme group decisions where individual fitness interests do not completely align (Conradt and Roper, 2003). However List (2004) as well as Conradt and List (2009) noted the effect of cost and prior asymmetry on quorum usage, referring back to earlier political science results (Ben-Yashar and Nitzan, 1997) discussed below. We also note that convergence on an intermediate group accuracy between 0 and 1 has previously been observed, without consideration of signal detection theory; the result presented in Figure 1b of Kao and Couzin (2014a) occurs for a similar reason to our error Ia, in that inappropriate use of a majority decision rule leads increasing group size to result in group accuracy converging on a parameter of the decision problem, in their case the reliability of a low correlation cue.

In contrast to these previous analyses here we have shown the fundamental importance of quorums in one of the simplest possible collective behaviour scenarios, single-shot collective decisions in homogenous groups, where individuals’ interests are aligned, and decision-making abilities do not differ. Thus, we might expect the use of quorums to be widespread in the natural world, even in the simplest of decisions. The widespread use of quorum sensing in bacteria provides evidence of this (Gross, 2017), and may prove a particularly good testbed for our theory given its binary nature and asymmetric state priors and error costs, although evolutionary conflicts of interests within bacterial communities may result in confounds. Moreover, as humans have been shown to employ quorum rules and adaptively adjust the associated quorum thresholds, human decision-making experiments may also provide a powerful approach to test our predictions (Kurvers et al., 2014; Clément et al., 2015).

It is surprising that signal detection theory has seen relatively scant application to collective decision-making. Wolf et al. (2013) noted the potential for different error types in identifying how quorums can improve collective decision accuracy; while motivated by signal detection theory they did not directly apply it to optimise the individual decision-makers and relate this back to group behaviour. Kirstein and von Wangenheim, 2010 also noted the possibility for independent error types, again with reference to signal detection theory; they noted the potential for Condorcet to make the same qualitatively incorrect predictions that we note here, however they did not apply the relevant theory to delineate the situations under which Condorcet reasoning is incorrect, nor did they consider the possibility for quorum-based decision rules to rescue these situations. Sorkin et al. (2001) applied signal detection theory to Condorcet-like models with varying super-majority quorums, but did not find the mechanism by which group decisions can be optimised (Wolf et al., 2013). Laan et al. (2017) noted that Condorcet and majority voting can be suboptimal, and considered ways in which voting can be improved; while they discussed signal detection theory and voting mechanisms they neither explicitly considered error types, nor quorum thresholds, focussing mainly on correlations between decision-makers and the impact of the cost function used, as well as suggesting a data-driven machine learning approach to improving collective decision-making rules. The results that most closely anticipate ours are those of Ben-Yashar and Nitzan (1997), who analytically solved for the general optimal decision rule by recognising both error cost and prior asymmetry, as well as simultaneously considering the case of variable individual decision-ability (e.g. Marshall et al., 2017). These results describe the relationship between prior asymmetry and optimal quorum threshold, and error cost and optimal quorum threshold. However, because they did not apply signal detection theory to optimise individual agents’ decisions (i.e. they treated true and false positive rates of individuals as independent from the ecological parameters error cost and prior asymmetry), they were unable quantitatively to uncover the complex nonlinear relationship between these three quantities (Figure 2).

In contrast to earlier work, by applying signal detection theory we have simultaneously shown here both how fragile Condorcet and majority-voting are, and how the use of sub- or super-majority quorums should relate to decision ecology. Although simple collective behaviour models have been well studied and highly influential, our results, and others relaxing other assumptions of such models (Marshall et al., 2017), indicate the subtlety that may be revealed in collective decision-making by a richer consideration of individual decision theory. Other approaches to such problems should be pursued in the future. For example, it is possible to optimise individual quorums (Ben-Yashar and Nitzan, 1997) rather than simply set them within a suitable interval as we do here, thereby giving greater benefits to smaller groups. Similarly, rather than apply signal detection theory one could apply Bayesian decision-theory (e.g. Pérez-Escudero and de Polavieja, 2011; Arganda et al., 2012; Pérez-Escudero and de Polavieja, 2017), thereby attempting to deal with further complexities such as non-independence of individual decisions. In many scenarios errors and correct decisions may be correlated although even when multiple individuals observe the same stimulus their information can be considered independent due to sensory noise (Marshall et al., 2017). We believe that our simple approach has, however, the benefit of tractability while still revealing the complexity of collective decision-making even in the simplified scenario considered.

Our work takes inspiration from political science and decision theory to address questions in behavioural ecology, but may additionally have the potential to inform work in the design of artificial decision-making systems, machine learning and robotics. For example, in the field of ensemble learning, in which predictions from multiple weak classifiers such as neural networks are combined to improve decision accuracy, variable quorums, referred to as ‘threshold shift’, are used (e.g. Dmochowski et al., 2010). However majority voting is still among the simplest and most ubiquitous vote fusion rules discussed (Sagi and Rokach, 2018; Krawczyk et al., 2017). Hence, we suggest that the simple perspective on how to combine votes presented here may also yield technological insight.

All data generated or analysed during this study are included in the manuscript and supporting files. Source data files have been provided for Figures 1, 2 and 3.

1. 1. Arganda S
   2. Pérez-Escudero A
   3. de Polavieja GG

   (2012) A common rule for decision making in animal collectives across species

   PNAS 109:20508–20513.

   https://doi.org/10.1073/pnas.1210664109

   • PubMed
   • Google Scholar
2. 1. Austen-Smith D
   2. Feddersen TJ

   (2009) Information aggregation and communication in committees

   Philosophical Transactions of the Royal Society B: Biological Sciences 364:763–769.

   https://doi.org/10.1098/rstb.2008.0256

   • Google Scholar
3. 1. Bahrami B
   2. Olsen K
   3. Latham PE
   4. Roepstorff A
   5. Rees G
   6. Frith CD

   (2010) Optimally interacting minds

   Science 329:1081–1085.

   https://doi.org/10.1126/science.1185718

   • PubMed
   • Google Scholar
4. 1. Ben-Yashar RC
   2. Nitzan SI

   (1997) The optimal decision rule for Fixed-Size committees in dichotomous choice situations: the general result

   International Economic Review 38:175–186.

   https://doi.org/10.2307/2527413

   • Google Scholar
5. 1. Berg S

   (1993) Condorcet's jury theorem, dependency among jurors

   Social Choice and Welfare 10:87–95.

   https://doi.org/10.1007/BF00187435

   • Google Scholar
6. 1. Boland PJ

   (1989) Majority systems and the condorcet jury theorem

   The Statistician 38:181–189.

   https://doi.org/10.2307/2348873

   • Google Scholar
7. 1. Bousquet CAH
   2. Sumpter DJT
   3. Manser MB

   (2011) Moving calls: a vocal mechanism underlying quorum decisions in cohesive groups

   Proceedings of the Royal Society B: Biological Sciences 278:1482–1488.

   https://doi.org/10.1098/rspb.2010.1739

   • Google Scholar
8. 1. Clément RJG
   2. Wolf M
   3. Snijders L
   4. Krause J
   5. Kurvers RHJM

   (2015) Information transmission via movement behaviour improves decision accuracy in human groups

   Animal Behaviour 105:85–93.

   https://doi.org/10.1016/j.anbehav.2015.04.004

   • Google Scholar
9. 1. Conradt L
   2. List C

   (2009) Group decisions in humans and animals: a survey

   Philosophical Transactions of the Royal Society B: Biological Sciences 364:719–742.

   https://doi.org/10.1098/rstb.2008.0276

   • Google Scholar
10. 1. Conradt L
    2. Roper TJ

    (2003) Group decision-making in animals

    Nature 421:155–158.

    https://doi.org/10.1038/nature01294

    • PubMed
    • Google Scholar
11. 1. Conradt L
    2. Roper TJ

    (2005) Consensus decision making in animals

    Trends in Ecology & Evolution 20:449–456.

    https://doi.org/10.1016/j.tree.2005.05.008

    • PubMed
    • Google Scholar
12. 1. Dmochowski JP
    2. Sajda P
    3. Parra LC

    (2010)

    Maximum likelihood in cost-sensitive learning: model specification, approximations, and upper bounds

    Journal of Machine Learning Research 11:3313–3332.

    • Google Scholar
13. 1. Galton F

    (1907) Vox populi

    Nature 75:450–451.

    https://doi.org/10.1038/075450a0

    • Google Scholar
14. Book

    1. Green DM
    2. Swets JA

    (1966)

    Signal Detection Theory and Psychophysics

    New York, United States: Wiley.

    • Google Scholar
15. 1. Gross M

    (2017) Shining new light on quorum sensing

    Current Biology 27:R1293–R1296.

    https://doi.org/10.1016/j.cub.2017.11.068

    • Google Scholar
16. 1. Hastie R
    2. Kameda T

    (2005) The robust beauty of majority rules in group decisions

    Psychological Review 112:494–508.

    https://doi.org/10.1037/0033-295X.112.2.494

    • PubMed
    • Google Scholar
17. 1. Kao AB
    2. Miller N
    3. Torney C
    4. Hartnett A
    5. Couzin ID

    (2014b) Collective learning and optimal consensus decisions in social animal groups

    PLoS Computational Biology 10:e1003762.

    https://doi.org/10.1371/journal.pcbi.1003762

    • PubMed
    • Google Scholar
18. 1. Kao AB
    2. Couzin ID

    (2014a) Decision accuracy in complex environments is often maximized by small group sizes

    Proceedings of the Royal Society B: Biological Sciences 281:20133305.

    https://doi.org/10.1098/rspb.2013.3305

    • Google Scholar
19. 1. Katsikopoulos KV
    2. King AJ

    (2010) Swarm intelligence in animal groups: when can a collective out-perform an expert?

    PLoS ONE 5:e15505.

    https://doi.org/10.1371/journal.pone.0015505

    • PubMed
    • Google Scholar
20. 1. King AJ
    2. Cowlishaw G

    (2007) When to use social information: the advantage of large group size in individual decision making

    Biology Letters 3:137–139.

    https://doi.org/10.1098/rsbl.2007.0017

    • PubMed
    • Google Scholar
21. 1. Kirstein R
    2. von Wangenheim G

    (2010)

    MAGKS Papers on Economics

    A Generalized Condorcet Jury Theorem with Two Independent Probabilities of Error, MAGKS Papers on Economics, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics.

    • Google Scholar
22. 1. Klein N
    2. Epley N

    (2015) Group discussion improves lie detection

    PNAS 112:7460–7465.

    https://doi.org/10.1073/pnas.1504048112

    • PubMed
    • Google Scholar
23. 1. Koriat A

    (2012) When are two heads better than one and why?

    Science 336:360–362.

    https://doi.org/10.1126/science.1216549

    • PubMed
    • Google Scholar
24. 1. Krawczyk B
    2. Minku LL
    3. Gama J
    4. Stefanowski J
    5. Woźniak M

    (2017) Ensemble learning for data stream analysis: a survey

    Information Fusion 37:132–156.

    https://doi.org/10.1016/j.inffus.2017.02.004

    • Google Scholar
25. 1. Kurvers RHJM
    2. Wolf M
    3. Krause J

    (2014) Humans use social information to adjust their quorum thresholds adaptively in a simulated predator detection experiment

    Behavioral Ecology and Sociobiology 68:449–456.

    https://doi.org/10.1007/s00265-013-1659-6

    • Google Scholar
26. 1. Kurvers RH
    2. Herzog SM
    3. Hertwig R
    4. Krause J
    5. Carney PA
    6. Bogart A
    7. Argenziano G
    8. Zalaudek I
    9. Wolf M

    (2016) Boosting medical diagnostics by pooling independent judgments

    PNAS 113:8777–8782.

    https://doi.org/10.1073/pnas.1601827113

    • PubMed
    • Google Scholar
27. 1. Laan A
    2. Madirolas G
    3. de Polavieja GG

    (2017) Rescuing collective wisdom when the average group opinion is wrong

    Frontiers in Robotics and AI 4:.

    https://doi.org/10.3389/frobt.2017.00056

    • Google Scholar
28. 1. Ladha KK

    (1992) The Condorcet jury theorem, free speech, and correlated votes

    American Journal of Political Science 36:617–634.

    https://doi.org/10.2307/2111584

    • Google Scholar
29. 1. List C

    (2004) Democracy in animal groups: a political science perspective

    Trends in Ecology & Evolution 19:168–169.

    https://doi.org/10.1016/j.tree.2004.02.004

    • PubMed
    • Google Scholar
30. 1. Lorenz J
    2. Rauhut H
    3. Schweitzer F
    4. Helbing D

    (2011) How social influence can undermine the wisdom of crowd effect

    PNAS 108:9020–9025.

    https://doi.org/10.1073/pnas.1008636108

    • PubMed
    • Google Scholar
31. 1. Marshall JAR
    2. Bogacz R
    3. Dornhaus A
    4. Planqué R
    5. Kovacs T
    6. Franks NR

    (2009) On optimal decision-making in brains and social insect colonies

    Journal of the Royal Society Interface 6:1065–1074.

    https://doi.org/10.1098/rsif.2008.0511

    • Google Scholar
32. 1. Marshall JAR
    2. Brown G
    3. Radford AN

    (2017) Individual Confidence-Weighting and group Decision-Making

    Trends in Ecology & Evolution 32:636–645.

    https://doi.org/10.1016/j.tree.2017.06.004

    • PubMed
    • Google Scholar
33. 1. Miller N
    2. Garnier S
    3. Hartnett AT
    4. Couzin ID

    (2013) Both information and social cohesion determine collective decisions in animal groups

    PNAS 110:5263–5268.

    https://doi.org/10.1073/pnas.1217513110

    • PubMed
    • Google Scholar
34. 1. Novaes Tump A
    2. Wolf M
    3. Krause J
    4. Kurvers RHJM

    (2018) Individuals fail to reap the collective benefits of diversity because of over-reliance on personal information

    Journal of the Royal Society Interface 15:20180155.

    https://doi.org/10.1098/rsif.2018.0155

    • Google Scholar
35. 1. Pérez-Escudero A
    2. de Polavieja GG

    (2011) Collective animal behavior from bayesian estimation and probability matching

    PLoS Computational Biology 7:e1002282.

    https://doi.org/10.1371/journal.pcbi.1002282

    • PubMed
    • Google Scholar
36. 1. Pérez-Escudero A
    2. de Polavieja GG

    (2017) Adversity magnifies the importance of social information in decision-making

    Journal of the Royal Society Interface 14:20170748.

    https://doi.org/10.1098/rsif.2017.0748

    • Google Scholar
37. 1. Pratt S
    2. Mallon E
    3. Sumpter D
    4. Franks N

    (2002) Quorum sensing, recruitment, and collective decision-making during colony emigration by the ant leptothorax albipennis

    Behavioral Ecology and Sociobiology 52:117–127.

    https://doi.org/10.1007/s00265-002-0487-x

    • Google Scholar
38. 1. Ross-Gillespie A
    2. Kümmerli R

    (2014) Collective decision-making in microbes

    Frontiers in Microbiology 5:.

    https://doi.org/10.3389/fmicb.2014.00054

    • PubMed
    • Google Scholar
39. 1. Sagi O
    2. Rokach L

    (2018) Ensemble learning: a survey

    Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8:e1249.

    https://doi.org/10.1002/widm.1249

    • Google Scholar
40. 1. Seeley TD
    2. Visscher PK

    (2004) Quorum sensing during nest-site selection by honeybee swarms

    Behavioral Ecology and Sociobiology 56:594–601.

    https://doi.org/10.1007/s00265-004-0814-5

    • Google Scholar
41. Book

    1. Shettleworth SJ

    (2010)

    Cognition, Evolution, and Behavior

    Oxford University Press.

    • Google Scholar
42. 1. Sorkin RD
    2. Hays CJ
    3. West R

    (2001) Signal-detection analysis of group decision making

    Psychological Review 108:183–203.

    https://doi.org/10.1037/0033-295X.108.1.183

    • PubMed
    • Google Scholar
43. 1. Sumpter DJ
    2. Krause J
    3. James R
    4. Couzin ID
    5. Ward AJ

    (2008) Consensus decision making by fish

    Current Biology 18:1773–1777.

    https://doi.org/10.1016/j.cub.2008.09.064

    • PubMed
    • Google Scholar
44. 1. Sumpter DJT
    2. Pratt SC

    (2009) Quorum responses and consensus decision making

    Philosophical Transactions of the Royal Society B: Biological Sciences 364:743–753.

    https://doi.org/10.1098/rstb.2008.0204

    • Google Scholar
45. Book

    1. Surowiecki J

    (2005)

    The Wisdom of Crowds

    Random House LLC.

    • Google Scholar
46. 1. Swets JA

    (1988) Measuring the accuracy of diagnostic systems

    Science 240:1285–1293.

    https://doi.org/10.1126/science.3287615

    • PubMed
    • Google Scholar
47. 1. Trimmer PC
    2. Houston AI
    3. Marshall JAR
    4. Bogacz R
    5. Paul ES
    6. Mendl MT
    7. McNamara JM

    (2008) Mammalian choices: combining fast-but-inaccurate and slow-but-accurate decision-making systems

    Proceedings of the Royal Society B: Biological Sciences 275:2353–2361.

    https://doi.org/10.1098/rspb.2008.0417

    • Google Scholar
48. 1. Trimmer PC
    2. Ehlman SM
    3. McNamara JM
    4. Sih A

    (2017) The erroneous signals of detection theory

    Proceedings of the Royal Society B: Biological Sciences 284:20171852.

    https://doi.org/10.1098/rspb.2017.1852

    • Google Scholar
49. 1. Walker RH
    2. King AJ
    3. McNutt JW
    4. Jordan NR

    (2017) Sneeze to leave: African wild dogs ( Lycaon pictus ) use variable quorum thresholds facilitated by sneezes in collective decisions

    Proceedings of the Royal Society B: Biological Sciences 284:20170347.

    https://doi.org/10.1098/rspb.2017.0347

    • Google Scholar
50. 1. Ward AJ
    2. Sumpter DJ
    3. Couzin ID
    4. Hart PJ
    5. Krause J

    (2008) Quorum decision-making facilitates information transfer in fish shoals

    PNAS 105:6948–6953.

    https://doi.org/10.1073/pnas.0710344105

    • PubMed
    • Google Scholar
51. 1. Ward AJ
    2. Krause J
    3. Sumpter DJ

    (2012) Quorum decision-making in foraging fish shoals

    PLoS ONE 7:e32411.

    https://doi.org/10.1371/journal.pone.0032411

    • PubMed
    • Google Scholar
52. 1. Wolf M
    2. Kurvers RHJM
    3. Ward AJW
    4. Krause S
    5. Krause J

    (2013) Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives

    Proceedings of the Royal Society B: Biological Sciences 280:20122777.

    https://doi.org/10.1098/rspb.2012.2777

    • Google Scholar
53. 1. Wolf M
    2. Krause J
    3. Carney PA
    4. Bogart A
    5. Kurvers RHJM

    (2015) Collective intelligence meets medical decision-making: the collective outperforms the best radiologist

    Plos One 10:e0134269.

    https://doi.org/10.1371/journal.pone.0134269

    • PubMed
    • Google Scholar

Author details

1. James AR Marshall

   Department of Computer Science, University of Sheffield, Sheffield, United Kingdom

   Contribution

   Conceptualization, Formal analysis, Writing—original draft, Writing—review and editing

   For correspondence

   james.marshall@sheffield.ac.uk

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1506-167X

2. Ralf HJM Kurvers

   Centre for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany

   Contribution

   Conceptualization, Writing—review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-3460-0392

3. Jens Krause

   Department of Fish Behavior and Ecology, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany

   Contribution

   Conceptualization, Writing—review and editing

   Competing interests

   No competing interests declared

4. Max Wolf

   Department of Fish Behavior and Ecology, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany

   Contribution

   Conceptualization, Formal analysis, Writing—original draft, Writing—review and editing

   For correspondence

   m.wolf@igb-berlin.de

   Competing interests

   No competing interests declared

• 1,320

  views

• 175

  downloads

• 22

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.