Should you go for coffee with Jules, or go to the movie theater with Jim? Both options require you to make additional decisions, for example, which café would you go to, or what movie could you see? Many of our day-to-day decisions have multiple layers of sub-decisions embedded within them that are not necessarily independent. Our opinions of the cafés in town and the movies showing at the theater may influence our decision over whom to spend the afternoon with.

In 2015, researchers at the Netherlands Institute for Neuroscience performed experiments in macaques to try to work out how the brain makes these decisions. The monkeys learned to choose between two visual stimuli (decision 1). The outcome of decision 1 determined whether the animals then had to make decision 2 or decision 3. The results suggested that the monkeys initially made all three comparisons independently and in parallel, before combining the evidence to select their overall strategy. This process is referred to as hierarchical decision-making. In the analogy above, one would compare the relative merits of Jules versus Jim, café A versus café B, and a horror movie versus a comedy at the same time before deciding what to do.

Hyafil and Moreno Bote have now reanalyzed the data published in 2015 using new computer simulations. This second analysis suggests the results are in fact more consistent with an alternative model of decision-making called a flat model, in which the brain compares all of the final options simultaneously (Jules + café A; Jules + café B; Jim + horror movie; Jim + comedy) before choosing between them.

Making decisions by comparing the final outcomes becomes easier as the brain learns through experience to associate stimuli that often occur together. Hyafil and Moreno Bote hypothesize that in response to a new situation, the brain may sometimes start off by using hierarchical decision-making before switching to a more accurate flat model as experience allows.

In response to the findings of Hyafil and Moreno Bote, the researchers who conducted the work reported in 2015 have also reanalyzed the original data, and carried out a new experiment in human volunteers. They argue that the flat model provides a poor fit to the original data and struggles to explain the new data. Future studies can build on these conflicting findings by further exploring the limits of parallel decision-making, which may help us to understand how the brain is able to make multiple decisions while keeping the future consequences in mind.

Coffee with Jules (if so, which cafe?) or cinema with Jim (if so, what movie?) ? A recent study by Lorteije and colleagues investigated how perceptual mechanisms implement such hierarchically-structured decisions that fill up our daily lives (Lorteije et al., 2015). Monkeys performed saccades to one of 4 possible targets based on the information provided at the primary branching and at secondary branching points (the 'correct' and 'incorrect' branching points) leading to the targets (Figure 1A). Patterns of responses convincingly indicated that monkeys can integrate information from all branching points in parallel. But does the decision space mimic the hierarchical organization, with parallel decisions going on (about Jules/Jim, coffee place, movie) (Figure 1B), or does it directly compare final options (with Jules at Moe's vs. Sicario with Jim vs. etc.)(Figure 1C)? Lorteije and colleagues report two behavioral and two neural effects that they argue speak unanimously in favor of the former hierarchical model against the latter non-hierarchical ‘flat model’ of decision-making. We show in contrast that all four effects can equally (and more parsimoniously) be explained by the flat model of decision-making, which is also perfectly compatible with the new effects from the same dataset described in the companion paper by Zylberberg and colleagues (Zylberberg et al., 2017).

Figure 1

Download asset Open asset

First, stimulus difficulty at the primary branch (L1) was shown to have no influence on the performance at the secondary branch (L2), a result that was replicated by a race-model implementation of hierarchical decision-making, but not of the flat decision-making. However, this result seems a particular feature of their specific choice of implementation of the race model (Vickers, 1979; Drugowitsch et al., 2014), which included neither inhibition between option representations, nor activation rectification (i.e. enforcing non-negative unit activation), two classical ingredients of race models (Usher and McClelland, 2001; Tsetsos et al., 2012; Churchland and Ditterich, 2012). Inhibition between option representations and rectification may underlie the well-known reduction of choice-related neural activity when more choice alternatives are provided (Churchland and Ditterich, 2012). We simulated a race model implementation of the flat model with both rectified activation, cross-inhibition and self-excitation, whereby each of the four options competed during accumulation of evidence (Figure 1D, see Material and methods). Self-excitation models the instability present at the initial point of the race in attractor models of decision-making (Roxin and Ledberg, 2008) and can be at the heart of well-known urgency signals to speed up decisions (Drugowitsch et al., 2012). Noise in the model did not scale with stimulus intensity, in line with recent results suggesting that noise in perceptual accumulation tasks is associated with the accumulation rather than with the sensory process (Drugowitsch et al., 2016), and as it is typically assumed in drift-diffusion models of decision making (Gold and Shadlen, 2007). We used a non-absorbing decision threshold (i.e. activity after the decision as not bounded), but found the same results for simulations with an absorbing bound. Parameters were tuned to reproduce qualitatively the proportion for each of the four possible response types, as well as the impact of each sample in the stream and the psychometric curve for both L1 and L2 decision (Figure 2—figure supplement 2A–D). The impact of L1 difficulty onto L2 for the flat model indeed disappeared when we used this common implementation of the race model (Figure 2—figure supplement 2E).

In the companion paper (Zylberberg et al., 2017), Zylberberg and colleagues question the generality of this finding, arguing that it may not be compatible with the pattern of short reaction times they provide in a new analysis for the same monkey dataset. However, our mathematical analysis (Appendix 1) shows that a clear influence of L1 difficulty onto L2 performance emerges principally when L1 difficulty strongly modulates reaction times, and thus the time of integration of L2 evidence. Short reaction times with low modulation by task difficulty as described in the companion article are thus perfectly compatible with lack of influence of L1 difficulty onto L2 performance. This result was indeed reproduced in a simulation where the threshold was lowered to produce shorter reaction times, and the boundary was eliminated for the first 10 samples (Figure 2A–F). This choice of the threshold implements a form of time-dependent, collapsing bound (Churchland and Ditterich, 2012; Drugowitsch et al., 2012) and it is consistent with the minimum viewing time imposed on monkeys (Figure 2). Without such time-dependent bound, the model can produce either a large proportion of premature responses or very large mean reaction times. The discrepancy between results from these simulations with those performed in the companion paper (who do find dependency of L2 performance on L1 difficulty with short reaction times) may emerge from early responses (<500 ms) in easy trials that are present in their simulations, but prohibited in our simulations by the absence of decision boundary for the first 10 samples.

Figure 2 with 2 supplements see all

Download asset Open asset

The second interesting observation from monkey behavior reported by Lorteije and colleagues was that L1 decisions were biased towards the branch that leads to the easier L2 decision. The effect could only be explained in the hierarchical model by invoking an extra modulatory signal passed on from L2 to L1 that must be carefully tuned (and was not implemented in the race model of [Lorteije et al., 2015]). By contrast, such effect is readily accounted for by the flat model, as a strong signal at a secondary branch will boost the chances of selecting the corresponding final option, and thus bias towards selecting the L1 path leading to this option. Indeed, the effect was reproduced in the simulations (Figure 2G). This relates to one important advantage of the flat model over the hierarchical one: the flat model, by integrating the strength of evidence from each level, takes optimally into account uncertainty in each of the decisions, leading to more accurate responses. By contrast, the hierarchical model integrates decisions at each level independently of the level of evidence in support of each decision. This can only be amended by ad hoc mechanisms such as that proposed by Lorteije and colleagues, that probably do not scale up adequately when more than two options are available at each level.

At the physiological level, neural groups in visual cortex integrated information conveyed at the primary and secondary branching points leading to the path in their receptive field. Selection signals were shown to first differentiate options at the level of secondary branches irrespective of whether the branch is the correct or incorrect one (L2 or L2' branching), and subsequently grow larger for the L2 than for L2' branching. In the hierarchical model, this can only be explained by referring to a modulating signal once decision is reached in L1 that differentially modulates the L2 race signals corresponding to the selected and non-selected L1 branches. In the flat model, such dynamics emerges naturally with activation rectification, because activation corresponding to the two incorrect options in the incorrect L1 branch vanish as L1 signals favor the two alternative options, and so their difference is also reduced (Figure 3A). Indeed, activation rectification plays a role in the dynamics of recorded neural responses, that reach a floor value (0) in the final part of the integration period in non-selected branches (Figure 4 of [Lorteije et al., 2015]). Finally, an interaction between L2 and L2' selection signals was reported: clear evidence in favor of one of the two options in L2 reduced the selection signal in L2', and vice versa. In the hierarchical model, this would require complex modulatory signals passing on from L2 to L1 to L2'. By contrast, both effects are observed in the flat model (Figure 3B): strong evidence for one option will decrease the activation of all other three options through inhibition, increase their chance of collapsing at the zero boundary, and thus reduce the selection signal in the opposite secondary branch.

Figure 3

Download asset Open asset

Overall we show that all four observations that were taken as evidence in favor of the hierarchical model could be accounted for with a standard race-model implementation of the flat model. These observations were robust and did not rely on fine tuning of the parameters. Table 1 summarizes how each of these observed properties depend on the features of the model. Note that all these features are classical constituents of race models and not ad hoc mechanisms. Overall the flat model provides a more parsimonious explanation of the data, as it does not require appending modulatory signals between parallel decisions as the hierarchical model does.

These behavioral and neural effects reported in the original study provide however no univocal evidence in favor of either model. One fundamental difference between the two is indeed that selection signals in the flat model mix evidence from both L1 and L2 branches, while the hierarchical model predicts unmixed selection signals (the influence across branches can only occur posterior to decision). [At this point, an important distinction has to be made in the flat model between localized activations, which indeed mix evidence from both branches, and selection signals at L2, extracted by looking at the difference between two units in the same L1 branch, and which as shown above are largely insensitive to L1 signals]. In the companion paper, Zylberberg et al. now provide data showing that selection signals in at least 3 out of 4 L2 branches mixes evidence from both L1 and L2 branches (their Figure 4). We believe this observation is most compatible with the flat model and by itself rules out the hierarchical model that relies on complete neural segregation of integration of L1 and L2 evidence (although the possibility remains that these level-mixing integrative neural signals are completely non-causal in monkey decisions). Our simulations reproduce these effects: signals in TT is larger for stronger L1 evidence, while signals in DT and DD are weaker (p<10⁻⁹, Figure 3C; see Also Appendix 2). As pointed out by Zylberberg and colleagues, our simulations also display a positive modulation in TD signals, unlike the null effect found in monkey data. However, despite a single source of inhibition, the modulation was not equally strong in all four branches: indeed, it was by far the weakest precisely in the TD branch (Figure 3C). This weaker effect may have explained the lack of significance in monkey data. Moreover, modulation in TD could be reduced or abolished if the flat model implied stronger inhibition between options related to the same L1 branch (TT-TD and DT-DD) than between options related to distinct L1 branches (e.g. TT-DT, TT-DD). Stronger inhibition between local circuits is indeed a general pattern of cortical connectivity (Douglas and Martin, 2004; Lund et al., 2003).

Zylberberg and colleagues produced a last analysis of the original dataset, showing that in the flat model errors in L1 are associated with higher sensitivity at L2, whereas monkeys display no such effect. While we acknowledge that this result challenges the current implementation of the flat model, it is at this point equally unknown whether the hierarchical model proposed by the authors could avoid this feature (the reward maximization introduced in the hierarchical model to account for the influence of L2 bias onto L1 choice may produce the same interaction).

It is unclear whether selection signals recorded in visual cortex are generated locally or reflect feedback process from higher regions where integration of evidence takes place. The flat model is arguably more consistent with the latter hypothesis, as it implies integration from and inhibition across distant locations in visual field, which are more typically associated with higher cortical regions (Wimmer et al., 2015). In any case, these selection signals certainly represent neural markers for a specific integration process along one subbranch.

One important limitation of flat models is they require to learn high-order representations of the environment that can appropriately integrate evidence from all relevant sensory sources, i.e. the structure of connectivity described in Figure 1D may take time to be acquired (Garrard et al., 1997; McClelland and Rumelhart, 1985). Here, monkeys performed ~60,000 trials each, possibly long enough to learn appropriate global representations linking all visual cues relevant to each response. When such time is not afforded, the system may rely on less efficient strategies such as a hierarchical model. Indeed, the comparison of flat and hierarchical models in this visual task sheds new light to the ancient debate of whether information from distinct sources is integrated before or after the decision stage, analogous respectively to the flat and hierarchical strategies. In multimodal integration such as object localization or motion detection, different modalities provide complementary cues about common objects or features of the environment (object motion, phoneme identity, etc.), so over the lifetime the brain can learn the appropriate crossmodal representations and integrate bimodal information directly over those representations. Indeed, in such context both sources of information are merged prior to decision (Körding and Wolpert, 2004). By contrast, when subjects must detect the presence of either a visual or an auditory cue, requires to mapping arbitrarily two distinct unimodal signals into a single response, the relevant crossmodal representations are not formed, and therefore integration could by default only be formed following unimodal decisions, as it has been found experimentally (Otto and Mamassian, 2012). One important hypothesis we make is that the level of integration could strongly depend on experience, gradually switching from post-decision (i.e. hierarchical) to pre-decision (i.e. flat) integration.

Finally, an important distinction has to be made between the hierarchical vs. flat nature of the integration process, and the serial vs. parallel nature of the integration process. While the former has to deal with the level and the structure of the representations at which integration takes place before reaching a decision, the latter corresponds to whether evidence from different sources can be integrated at the same time in these representations. The flat model is perfectly compatible with serial integration of evidence, for example if attentional constraints limit the capacity to process evidence from distant visual locations, such as when subjects must integrate from three distinct levels (Zylberberg et al., 2012).

In summary, despite premature conclusions, we think that the experimental framework developed by Lorteije and colleagues opens a new promising venue for understanding the level at which evidence is accumulated and individual perceptual decisions are taken in ecological settings.

We present here the equations governing the flat race model of decision-making used for simulations. Our implementation of the flat model departs from the one in [Lorteije et al., 2015] in the following aspects, which are typically subsumed in many standard implementations: (1) added noise is constant instead of being proportional to signal strength, (2) there is a positive loop (i.e. negative leak) in the integration process and mutual inhibition between competitors and (3) activities cannot be negative (i.e. activation rectification). The variable $x_t^i$ represents the activity of the activation unit i (from 1 to 4) at sample t, and evolves according to:

where $I_0$ is the constant input, $\alpha$ is the auto-excitation term (negative if leaking, positive if positive loop providing bistability), $\beta$ is the inhibition strength, $I_t^i$ is the sensory evidence at sample t, and the last term represents white noise of variance $\sigma$. Inhibition is homogeneous and all-to-all between activations units, that is, each unit inhibits equally the other unit from the same branch and the two units from the alternative branch. Sensory evidence integrates information from the primary and secondary branching points:

$k_1$ and $k_2$ represent the sensitivity to evidence at level 1 and 2 respectively, $a_t^i$ represents the information provided at level 1 in favor of the corresponding branch (i.e. the difference between instantaneous luminosity in that branch and luminosity in the alternative branch at sample t), $b_t^i$ represents the information provided at level 2 in favor of the corresponding branch.

We enforce non-negative values for unit activity by using rectification:

All units are initiated from the same starting point $x_0^i=0.5$, and the decision is reached whenever any of the units reaches threshold K (Figure 1C). If no decision is reached after presentation of all samples, the response corresponding to the unit with highest final activity is selected. Parameters are: input strength at primary branching point k₁ = 0.016, at secondary branching points k₂ = 0.012, auto-excitation $\alpha$ =0.1, inhibition $\beta$ = 0.07, constant input $I_0$= 0.5, decision threshold = 50, initial values $x_0^i$= 0.5, white noise variance = 1. We chose parameters to reproduce qualitatively the monkey behavioral and neuronal data. We simulated 100,000 trials, which is the same order of magnitude as used in the original experiment. All analyses performed on simulated data strictly reproduced those realized in the original experiment. Results from these analyses are presented in Figure 2—figure supplement 2.

In the simulation set of Figure 2, designed to reproduce the pattern of reaction times described in the companion paper, we lowered the decision threshold to 30 to produce shorter reaction times. We introduced a minimum integration time of 10 samples (i.e. no boundary for the first 10 samples), consistent with the minimal time of 500 ms after stimulus onset monkeys had to wait before performing a responses saccade. We also changed inhibition to 0.05 and auto-excitation to 0.12. All other parameters were unchanged.

A matlab script for all simulations and analyses is available at http://bit.ly/hyafilmoreno2017.

1. 1. Brunton BW
   2. Botvinick MM
   3. Brody CD

   (2013) Rats and humans can optimally accumulate evidence for decision-making

   Science 340:95–98.

   https://doi.org/10.1126/science.1233912

   • PubMed
   • Google Scholar
2. 1. Churchland AK
   2. Ditterich J

   (2012) New advances in understanding decisions among multiple alternatives

   Current Opinion in Neurobiology 22:920–926.

   https://doi.org/10.1016/j.conb.2012.04.009

   • PubMed
   • Google Scholar
3. 1. Douglas RJ
   2. Martin KA

   (2004) Neuronal circuits of the neocortex

   Annual Review of Neuroscience 27:419–451.

   https://doi.org/10.1146/annurev.neuro.27.070203.144152

   • PubMed
   • Google Scholar
4. 1. Drugowitsch J
   2. Moreno-Bote R
   3. Churchland AK
   4. Shadlen MN
   5. Pouget A

   (2012) The cost of accumulating evidence in perceptual decision making

   Journal of Neuroscience 32:3612–3628.

   https://doi.org/10.1523/JNEUROSCI.4010-11.2012

   • PubMed
   • Google Scholar
5. 1. Drugowitsch J
   2. Moreno-Bote R
   3. Pouget A

   (2014) Relation between belief and performance in perceptual decision making

   PLoS One 9:e96511.

   https://doi.org/10.1371/journal.pone.0096511

   • PubMed
   • Google Scholar
6. 1. Drugowitsch J
   2. Wyart V
   3. Devauchelle AD
   4. Koechlin E

   (2016) Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality

   Neuron 92:1398–1411.

   https://doi.org/10.1016/j.neuron.2016.11.005

   • PubMed
   • Google Scholar
7. 1. Garrard P
   2. Perry R
   3. Hodges JR

   (1997) Disorders of semantic memory

   Journal of Neurology, Neurosurgery & Psychiatry 62:431–435.

   https://doi.org/10.1136/jnnp.62.5.431

   • PubMed
   • Google Scholar
8. 1. Gold JI
   2. Shadlen MN

   (2007) The neural basis of decision making

   Annual Review of Neuroscience 30:535–574.

   https://doi.org/10.1146/annurev.neuro.29.051605.113038

   • PubMed
   • Google Scholar
9. 1. Körding KP
   2. Wolpert DM

   (2004) The loss function of sensorimotor learning

   PNAS 101:9839–9842.

   https://doi.org/10.1073/pnas.0308394101

   • PubMed
   • Google Scholar
10. 1. Lorteije JA
    2. Zylberberg A
    3. Ouellette BG
    4. De Zeeuw CI
    5. Sigman M
    6. Roelfsema PR

    (2015) The formation of hierarchical decisions in the visual cortex

    Neuron 87:1344–1356.

    https://doi.org/10.1016/j.neuron.2015.08.015

    • PubMed
    • Google Scholar
11. 1. Lund JS
    2. Angelucci A
    3. Bressloff PC

    (2003) Anatomical substrates for functional columns in macaque monkey primary visual cortex

    Cerebral Cortex 13:15–24.

    https://doi.org/10.1093/cercor/13.1.15

    • PubMed
    • Google Scholar
12. 1. McClelland JL
    2. Rumelhart DE

    (1985) Distributed memory and the representation of general and specific information

    Journal of Experimental Psychology: General 114:159–188.

    https://doi.org/10.1037/0096-3445.114.2.159

    • PubMed
    • Google Scholar
13. 1. Otto TU
    2. Mamassian P

    (2012) Noise and correlations in parallel perceptual decision making

    Current Biology 22:1391–1396.

    https://doi.org/10.1016/j.cub.2012.05.031

    • PubMed
    • Google Scholar
14. 1. Roxin A
    2. Ledberg A

    (2008) Neurobiological models of two-choice decision making can be reduced to a one-dimensional nonlinear diffusion equation

    PLoS Computational Biology 4:e1000046.

    https://doi.org/10.1371/journal.pcbi.1000046

    • PubMed
    • Google Scholar
15. 1. Tsetsos K
    2. Gao J
    3. McClelland JL
    4. Usher M

    (2012) Using Time-Varying Evidence to Test Models of Decision Dynamics: Bounded Diffusion vs. the Leaky Competing Accumulator Model

    Frontiers in Neuroscience 6:79.

    https://doi.org/10.3389/fnins.2012.00079

    • PubMed
    • Google Scholar
16. 1. Usher M
    2. McClelland JL

    (2001) The time course of perceptual choice: the leaky, competing accumulator model

    Psychological Review 108:550–592.

    https://doi.org/10.1037/0033-295X.108.3.550

    • PubMed
    • Google Scholar
17. Book

    1. Vickers D

    (1979)

    Decision Processes in Visual Perception

    Academic Press.

    • Google Scholar
18. 1. Wimmer K
    2. Compte A
    3. Roxin A
    4. Peixoto D
    5. Renart A
    6. de la Rocha J

    (2015) Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT

    Nature Communications 6:6177.

    https://doi.org/10.1038/ncomms7177

    • PubMed
    • Google Scholar
19. 1. Zylberberg A
    2. Lorteije JAM
    3. Ouellette BG
    4. De Zeeuw CI
    5. Sigman M
    6. Roelfsema P

    (2017) Serial, parallel and hierarchical decisionmaking in primates

    eLife 6:e17331.

    https://doi.org/10.7554/eLife.17331

    • Google Scholar
20. 1. Zylberberg A
    2. Ouellette B
    3. Sigman M
    4. Roelfsema PR

    (2012) Decision making during the psychological refractory period

    Current Biology 22:1795–1799.

    https://doi.org/10.1016/j.cub.2012.07.043

    • PubMed
    • Google Scholar

Author details

1. Alexandre Hyafil

   CBC, DTIC, Universitat Pompeu Fabra, Barcelona, Spain

   Contribution

   AH, Conceptualization, Software, Formal analysis, Funding acquisition, Investigation, Visualization, Methodology, Writing—original draft, Project administration, Writing—review and editing

   For correspondence

   alexandre.hyafil@gmail.com

   Competing interests

   The authors declare that no competing interests exist.

   "This ORCID iD identifies the author of this article:" 0000-0002-0566-651X

2. Rubén Moreno-Bote

   1. CBC, DTIC, Universitat Pompeu Fabra, Barcelona, Spain
   2. Serra Húnter Fellow Programme, Universitat Pompeu Fabra, Barcelona, Spain

   Contribution

   RM-B, Supervision, Investigation, Writing—original draft, Project administration, Writing—review and editing

   Competing interests

   The authors declare that no competing interests exist.

• 1,393

  views

• 198

  downloads

• 11

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.