Neural signatures of model-based and model-free reinforcement learning across prefrontal cortex and striatum

Animals use at least two major reinforcement learning (RL) systems for behavioural control in sequential decision-making: a goal-directed or model-based (MB) system and a habitual or model-free (MF) (Dickinson and Balleine, 1994; Daw et al., 2005; Dolan and Dayan, 2013; Dickinson, 1985) system. Both approaches rely on previous experience and converge to the same behaviour given enough practice in a stable environment, but they differ as to how this information is used to infer the values of choices. MB-RL computes estimates prospectively by integrating reward information with knowledge about the state-transition function, which specifies how the state of the world evolves probabilistically given particular choices (Tolman, 1948; Daw and Dayan, 2014). MF-RL, a less flexible but simpler approach, learns without any model of the environment by bootstrapping sampled experience to train cached predictions of long-run rewards via reward prediction errors (RPEs) (Thorndike, 1911; Sutton, 1988).

Neural substrates of reward-based learning and decision-making involve complex anatomical connections between the basal ganglia and the prefrontal cortex (PFC) (Balleine and O’Doherty, 2010; Haber and Behrens, 2014). Lesion studies and human neuroimaging suggest an involvement of the primate putamen in MF-RL (Yin et al., 2004; Tricomi et al., 2009; Wunderlich et al., 2012), whereas more anterior regions of the caudate (Wunderlich et al., 2012; Yin et al., 2005; Tanaka et al., 2008), PFC (Valentin et al., 2007; Gläscher et al., 2010; Daw et al., 2011; Miller et al., 2017; Schuck et al., 2016; Chan et al., 2021; Kolling et al., 2016; Behrens et al., 2018), and hippocampus (Miller et al., 2017; Johnson and Redish, 2007; Boorman et al., 2016) have been implicated in MB-RL. However, there is also much work on the online (Morris and Cushman, 2019; Moran et al., 2021; Sezener et al., 2019) and offline (Antonov et al., 2022; Eldar et al., 2020; Liu et al., 2021; Mattar and Daw, 2018) integration of MF and MB quantities, emphasising a closer interplay between both learning systems. At the single-neuron level, midbrain dopaminergic cells report an RPE that could drive the updating of MF-RL predictions (Schultz et al., 1997; Bayer and Glimcher, 2005). Furthermore, neurons in the striatum have been found to encode cached MF-RL action values at choice time (Lau and Glimcher, 2008), but the paradigms used did not elicit differences between MF and MB computations.

One such task that does enable the dissociation of MB and MF computations is ‘two-step’ task by Daw et al., 2011. It contains a probabilistic transition between task states to uncouple MF learners (who would assign credit to which state was rewarded regardless of the transition) from MB learners (who would appropriately assign credit based on the reward and transition that occurred). Rodents (Miller et al., 2017), monkeys (Miranda et al., 2020), and humans (Daw et al., 2011) all use MB-like behaviour to solve the task. Evidence in rodents suggests dorsal anterior cingulate cortex (ACC) tracks rewards, states, and the probabilistic transition structure, and that ACC is essential in implementing an MB strategy (Akam et al., 2021). Here, we compare primate single-neuron activity of four different subregions implicated in reward-based learning and choice (ACC, dorsolateral PFC [DLPFC], caudate, and putamen) during performance of the classic two-step task, and demonstrate signatures of MB-RL primarily in ACC, and MF-RL signatures most notably in putamen.

Two male rhesus macaques (Macaca mulatta) completed a variation of the classic two-step task (Daw et al., 2011). After an initial eye fixation period, subjects first chose between two first-stage options (A, B, ‘choice 1’), each leading to a different second-stage state (indicated by background colour) (Figure 1A and B). Crucially, the transition between these two stages was probabilistic, with each option transitioning to its preferred second-stage state in 70% (‘common’) of trials and transitioning to the alternative second-stage state in 30% (‘rare’) of trials (Figure 1B). This state-transition allows the dissociation of MF-RL strategies (that estimate state values using bootstrapped sampling of experienced states) from MB-RL strategies (which use the transition structure to assign rewards prospectively to states based on the probability of that state occurring). Both second-stage states then required another choice (‘choice 2’) between two options (C and D, or E and F), leading to one of three possible rewarding outcomes (high, medium, low outcome) indicated by a secondary reinforcer cue (‘feedback’). In our version of the task, the outcome values of each second-stage state randomly and independently changed value every five to nine trials, thus requiring subjects to continue sampling the different second-stage states to determine where the highest rewards were located across trials (Figure 1B). This reward schedule resulted in an advantage of an MB policy over an MF policy, the former of which consistently gained more reward per trial than the latter in simulations (MB gained 1.2979±0.0071 units of reward per trial [n=57 recording sessions, the three reward levels were counted as 0, 1, or 2 AU]; MF 1.2416±0.0069 reward per trial; difference between them: p<0.0001, paired t-test; Figure 1—figure supplement 1). Both strategies were also better than a random policy (0.9953±0.007 reward per trial, vs MF/MB: p<0.001).

Figure 1 with 1 supplement see all

Download asset Open asset

In a prior comprehensive behavioural analysis (Miranda et al., 2020), we demonstrated that both subjects estimated the value of each choice 1 option using a combination of MF and MB-RL algorithms, but with MB dominating (Figure 1C–E). MF-RL does not exploit information about task structure, so it predicts no difference in the probability of repeating choice 1 dependent on common/rare transition, whereas a key signature of MB-RL is just such a difference. We found that both subjects were significantly more likely to repeat choice 1 when a high reward was obtained through a common (rather than rare) transition, with the opposite pattern evident following low rewards (Figure 1C), indicating a strong MB influence on behaviour. Having established the influence of MF- and MB-RL strategies on behaviour (Miranda et al., 2020), we next examined whether signatures of MF- and MB-RL were evident in neuronal activity in four key brain regions implicated in learning and choice: we recorded single-neuron activity from two regions in PFC – dorsal ACC (n=240) and DLPFC (n=187), and two regions in the striatum – caudate (n=115) and putamen (n=119) (see Figures 8 and 9 for recording locations).

As parameters were correlated over trials (e.g. the same choice was made repeatedly when it led to a high reward), there was a risk that intrinsic temporal autocorrelations in firing rate across trials would confound the analysis (Shin et al., 2021; Elber-Dorozko and Loewenstein, 2018; Harris, 2020). We therefore incorporated several nuisance regressors to account for temporal autocorrelations in neuronal activity that may have been present (see Materials and methods, Tables 1–3). To check this was an adequate control, we analysed neural firing rates with respect to a different session’s trial data (i.e. the dependent and independent variables for the regression were from different sessions) (Shin et al., 2021; Elber-Dorozko and Loewenstein, 2018; Harris, 2020). No significant encoding of any of our parameters of interest was observed at either the population- or single-neuron level in such null controls (Figure 2—figure supplement 1). Therefore, temporal autocorrelations in neuronal activity were adequately controlled, and we next turned to examining neural encoding of task parameters.

ACC encoded reward across task events; striatal regions encoded an MF RPE signal

We first examined the extent of reward encoding in each region by examining the response of neurons to the secondary reinforcer cue. At the population level, all regions strongly encoded the value of the secondary reinforcer cue at feedback; furthermore, this encoding persisted throughout a vast period of the following trial – being significantly stronger for ACC during this particular time period (Figure 2A; GLM1). The same was true at the single-neuron level, with large populations of neurons within each region encoding the reward during feedback and long into the following trial (Figure 2B). ACC’s encoding of reward was stronger than all other regions at both the population-level (p<0.05, cluster-based permutation test; Figure 2A) and single-neuron level (p<0.05, chi-square test; Figure 2B). Encoding of the value of the secondary reinforcer peaked earlier in striatal neurons (507.7±19.16 ms) compared to prefrontal neurons (588.5±13.9 ms; p=0.005 independent t-test; Figure 2C). Therefore, all four regions, but especially ACC, encoded the previous trial’s reward throughout the next trial.

Figure 2 with 1 supplement see all

Download asset Open asset

The direction of this reward coding differed between regions from PFC and striatum. The latter encoded the predicted reward (i.e. from secondary reinforcer) positively (i.e. firing rate positively correlated with reward; p<0.05 from 190 and 160 ms for caudate and putamen respectively, one-sample t-test of coefficients against 0, Figure 2D and E), whereas ACC encoded reward with a negative polarity and slower response time (p<0.05 at 410 ms onwards, Figure 2D and E). Additionally, both caudate and putamen encoded the reward from the previous trial negatively during the feedback period of the current trial (Figure 2F and G). Such quantitative features in both striatal areas (i.e. positive coding of current reward and negative encoding of past rewards) are typical of a dopaminergic RPE signal (Schultz et al., 1997; Bayer and Glimcher, 2005; Dolan and Dayan, 2013).

ACC encoded MB state-transition information

Tracking the state-transition structure of the task is imperative for solving the task as an MB learner. All four regions encoded whether the current trial’s first-stage choice transitioned to the common or rare second-stage state (which could be inferred by a change in background colour immediately after choice indicating which second-stage state they had just entered, Figure 1A). This signal was significantly stronger in ACC – where state-transition information was maintained from transition until the feedback epoch (CPD >0.3%, p<0.002, cluster-based permutation test; Figure 3A; GLM1). A similar pattern was also observed at the individual neuron level (Figure 3B).

Figure 3 with 1 supplement see all

Download asset Open asset

As reward expectations would change on rare transitions if using an MB strategy, we next examined whether the encoding of previous trial reward was modulated by current trial’s transition. While all four regions exhibited some selectivity for the interaction between the previous trial’s reward and the current trial’s transition, a larger proportion of ACC neurons encoded this parameter compared to all other regions (p<0.05, chi-squared test, Figure 3C and D). Thus, ACC reward expectancy signals were modulated by the state-transition structure of the task.

All regions, but particularly ACC, encoded a common transition (at the time of transition) similar to a high reward (at the time of feedback), as there was a positive correlation between the coefficients for reward and transition (the transition parameter was signed such that common and rare transitions were equivalent to high and low rewards, respectively) (ACC r=0.4963, DLPFC r=0.3273, caudate r=0.4712, putamen r=0.5052; all p<0.002 except DLPFC, where p=0.006, circular permutation test; Figure 3E, Figure 3—figure supplement 1). As the reward expectation will be higher on common compared to rare trials, this demonstrates that the brain encodes being diverted to an area with a lower reward expectation equivalent to actually receiving a low reward (and vice versa). This signal also reflects awareness of the (MB-like) state-transition structure.

ACC and caudate encoded MB value estimates

Due to this encoding of the state-transition and its interaction with other variables, we next tested whether neurons encoded MB- or MF-derived chosen values of the choice 1 options (using RL computational modelling to estimate these values in a subject- and trial-specific manner; Miranda et al., 2020). At the single-neuron level, ACC was unique in having a large population of neurons that encoded both the MB and MF value of the choice 1 options (Figure 4A and B; GLM2). The encoding of MB emerged 280 ms before the cues were displayed, indicating the subjects were anticipating the upcoming choice, whereas MF-derived chosen value coding only emerged after the cues were presented (170 ms post cue presentation, p<0.05, binomial test). Caudate also had a population of neurons encoding the MB-derived value of choice 1, which emerged 430 ms after choice onset (Figure 4A). At the population level all four regions encoded the MB value estimate of choice 1, with ACC and caudate encoding it more than 500 ms before the options were displayed (Figure 4C). All four regions also encoded MF-RL estimates of choice 1’s value, again with ACC encoding emerging more than 500 ms before choice onset (Figure 4D).

Figure 4 with 1 supplement see all

Download asset Open asset

To further compare this encoding, we examined the peak strength of encoding of each neuron for either of the two choice 1 options (i.e. MB and MF value estimates for PicA/PicB; Table 2) at any point from 500 ms before to 1000 ms after choice 1 was presented. In all regions except DLPFC, peak CPD values across neurons were higher for MB compared to MF estimates (ACC: t=19.8812, p<0.0001, paired t-test; DLPFC: t=−2.1474, p=0.1319; caudate: t=14.659, p<0.0001; putamen: t=6.1911, p<0.0001; Figure 4E). ACC’s peak MB and MF encoding was significantly higher than in all other regions (p<0.001 independent t-test).

Table 2

Regressor	Description	Figure
Constant	To account for the mean effect
Linear effects	A ramping regressor to account for any monotonic autocorrelations across the session
(Linear effects) ^ 2	To account for any polynomial autocorrelations across the session
(Linear effects) ^ 3
(Linear effects) ^ 4
Sin 2	Sin curves of varying frequency (approximately 2, 4, 6, 9 cycles per session) to account for any non-monotonic autocorrelations across the session
Sin 4
Sin 6
Sin 9
Model-based PicA	Model-based derived estimates of the value of each option at choice 1	Figure 4
Model-based PicB
Model-free PicA	Model-free-derived estimates of the value of each option at choice 1
Model-free PicB
Choice 1 picture chosen	Which cue the subject chose on each trial

Examination of the strongest signal observed, ACC encoding of MB Q-values, showed a dynamic pattern, with different neurons encoding the signal during different parts of the epoch (Figure 4—figure supplement 1). When aggregating the number of significant coders throughout the epoch, and examining the specificity of MB versus MF coding, we found that all regions had a significant population of neurons that encoded MB-, but not MF-, derived value (30%, 19%, 23%, and 24% of neurons in ACC, DLPFC, caudate and putamen, respectively; all p<0.0014 binomial test against 10% [as the strongest response to either of the two options was used]; Figure 4F). All regions had a population of neurons that encoded both MB- and MF-derived option value (18%, 4%, 8%, 5% of neurons in ACC, DLPFC, caudate, and putamen, respectively; all p<0.012 binomial test against 1; Figure 4F), but ACC’s population was significantly higher than all other regions (p<0.05, chi-square test, Figure 4F). This dominance of MB encoding in caudate but not putamen at the population level may explain similar dissociations in these two striatal regions observed in human neuroimaging studies (Wunderlich et al., 2012).

Caudate value estimates remapped following a rare transition

The observed prominence of transition-related selectivity, in addition to strong encoding of MB-derived value estimates, led us to next examine how transition type altered the value estimation of the chosen first-stage option. To explore this, we used a combination of MB- and MF-value estimations (or the hybrid estimate) derived from the animal’s behaviour, which better explained their choices during the task than either of the two models in isolation (Miranda et al., 2020) (GLM3). In common trials, caudate was unique in encoding the value of the chosen option more positively than the unchosen option’s value (p<0.025, 280–320 ms post transition revealed, paired t-test; Figure 5). After a rare transition, caudate was again unique in encoding the chosen and unchosen option values differently, but this time it was the unchosen option that was encoded positively and the chosen option negatively (p<0.025, 170–370 ms, paired t-test; Figure 5). Thus, caudate’s encoding of an option’s value also reflected the availability of the option.

Figure 5

Download asset Open asset

Reward-modulated encoding of first-stage choice

We next examined the encoding of the choice 1 option (i.e. the stimulus identity of what they chose). All regions encoded it to an equal extent around the time of choice (difference between regions all p>0.05, cluster-based permutation test; Figure 6A; GLM1). Strikingly, all regions also encoded the interaction of choice 1, transition, and reward from the previous trial during the choice 1 epoch (Figure 6B), with ACC encoding it stronger than DLPFC and putamen (ACC vs DLPFC p=0.006 between –180 and 190 ms, vs putamen p=<0.002 between –150 and 450 ms; cluster-based permutation test; Figure 6B). Importantly, none of the intermediate pairwise interactions were encoded to a similar extent (Figure 6—figure supplement 1). Therefore, both ACC and caudate performed a very specific MB computation of integrating the reward, transition, and choice from the previous trial into a signal which could inform which option to choose on the current trial.

Figure 6 with 1 supplement see all

Download asset Open asset

Furthermore, we also found that choice 1 stimulus identity was encoded by the same neurons that encoded reward and transition. Choice 1 could be decoded from all four neural populations using a support vector machine, with it being most strongly represented in ACC and caudate (Figure 6C). We then performed a median split of neurons within each region depending on the strength with which they encoded reward at feedback and transition at transition (taking the average of the two coefficients). In ACC only, the neurons that were more sensitive to reward and transition encoded choice 1 more strongly than those neurons that were less sensitive to these attributes (Figure 6D–F). Therefore, it was the same subpopulation of neurons within ACC that tracked the different task parameters relevant for guiding MB choice.

Explore/exploit strategy modulates encoding of first-stage choice

In our task, the outcome level (high, medium, low) of each second-stage stimulus remained the same for five to nine trials before potentially changing. This design naturally created periods where subjects could ‘exploit’ the same choice 1 to maximise reward for several trials, and other periods where they had to ‘explore’ different second-stage stimuli to optimise reward (as contingencies shifted). In classical MB-RL, the transition between reward states can be learned by keeping counts of observed transitions from a current state-action pair to a subsequent state, yielding a maximum-likelihood estimate of the environment’s dynamics (Sutton et al., 2018). In fact, knowledge of the reward contingency schedule could support decision-making in both exploitation – by enabling efficient choice when rewards are stable – and exploration – by guiding alternative behaviour most likely to yield improved outcomes (this is different from MF learning, where exploration is more random since the agent lacks explicit state-transition knowledge).

We thus repeated our decoding analysis of choice 1 stimulus identity, but this time limited trials to those where they had not received a high reward for the previous two trials (‘explore’ trials) and those where the previous two rewards had been the highest level (‘exploit’ trials). All regions encoded choice 1 for some duration of the choice epoch for both explore (p<0.002 in all cases, permutation test; Figure 7A) and exploit (p<0.002 in all cases; Figure 7B) conditions, but decoding accuracy was strongest in ACC. Choice 1 was less strongly decoded – particularly in ACC – in the former condition compared to the latter (p<0.002 in all cases, permutation test on differences observed; Figure 7C); and, also during exploitation, the ACC encoded choice 1 before the choice was even presented to the subject (Figure 7—figure supplement 1). This pre-choice ACC encoding in exploit trials may reflect the need to allocate cognitive (or attentive) resources to features – i.e., choice 1 stimulus identity – that are most certain predictors of important outcomes. As a control, we also decoded the direction of the choice 1 (where choice was indicated via joystick movement), which was randomised each trial and therefore orthogonal to the stimulus that was chosen. Again, all four regions encoded its direction in both explore (p<0.002 in all cases; Figure 7D) and exploit (p<0.002 in all cases; Figure 7E). However, there were minimal differences in the strength of the representation between explore and exploit conditions (ACC, p=0.088, cluster-based permutation test; DLPFC p=0.016; caudate p=0.32; putamen p=1; Figure 7F). Therefore, exploit behaviour specifically upregulated relevant task parameters that were worth remembering across trials.

Figure 7 with 1 supplement see all

Download asset Open asset

Despite the extensive work in both MF-RL (or habitual) and MB-RL (or goal-directed) behaviour (Dolan and Dayan, 2013), few studies have shown simultaneous single-neuron signatures of both learning strategies (Akam et al., 2021), let alone across a number of relevant cortical and subcortical regions, and in the primate brain. Using a decision task which formally distinguishes MF and MB values, and by adopting an analytical approach that considered both observable variables and computational measures, we showed that key RL elements underlying choice behaviour were encoded in neurons in the primate PFC and striatum at different time points. The observed widespread and simultaneous representations of MF- and MB-related computations are consistent with the view that these controllers operate in parallel (Daw et al., 2005; Dolan and Dayan, 2013; Daw et al., 2011), but their associated signals are more richly intertwined than had originally been expected. Additionally, we found convincing evidence supporting a prominent role of ACC, and to a lesser extent, the caudate, in MB-RL. The ability of single neurons to encode reward-related parameters has been described throughout the brain (Apicella et al., 1991; Platt and Glimcher, 1999; Roesch and Olson, 2003; Wallis and Kennerley, 2010; Hunt et al., 2018; Muller et al., 2024). Here, we found reward coding at feedback (of MF and MB relevance) to be significantly stronger in ACC. However, at feedback an RPE (a hallmark of MF-RL) can also be computed to support learning. The midbrain dopamine neurons have been strongly implicated in encoding such RPE compared to other areas with similar (but not identical) feedback-related activity (Schultz et al., 1997; Bayer and Glimcher, 2005). Signals in other brain regions, such as PFC, have been found to report differences between received and expected outcome (Apicella et al., 2009; Seo and Lee, 2007; Matsumoto et al., 2007; Kennerley et al., 2011), but in most of these cases either the definition of RPE did not conform to the normative RL theory or the neurons did not exhibit the quantitative properties and negative aspect of the error. In our study, both caudate and putamen neurons clearly responded at feedback with the parametric features of a dopamine-like RPE (Schultz et al., 1997; Bayer and Glimcher, 2005). It is important to underline that, despite being seen as a signal that drives MF-RL, both MF and MB approaches coincide at feedback and use this RPE to update their valuations (Miranda et al., 2020). The data presented here confirms that basal ganglia structures (both dopaminergic and striatal neurons) are unique (relative to PFC) in computing RPE-like signals.

The knowledge of the state-transition structure differentiates MB from MF-RL and hence we investigated neurons that significantly discriminated a common from a rare transition. Our findings are in line with the neural changes seen in the rat ACC when an animal’s belief is modified after an environmental change (Karlsson et al., 2012), and the observation that inhibition of mouse ACC impairs the use of the transition, but not rewards (Akam et al., 2021). Additionally, the selectivity of ACC neurons for detecting state-transition fits well with the neuroimaging signal observed in ACC on a saccadic planning task when an internal model required update (O’Reilly et al., 2013), endorsing its role in dynamically updating behavioural policies. As all regions adjusted their value estimates as a function of the transition type occurred, our simultaneous recordings extend previous findings by supporting that the ubiquitous use of state-transition information across fronto-striatal circuits is likely propagated from (or via) ACC to other (reward-focused) regions.

It is important to highlight that MB-like behaviour can arise from sophisticated MF strategies, such as tracking state-transition probabilities and the reward function (Akam et al., 2021; Akam et al., 2015; Blanco-Pozo et al., 2024). Further work should be undertaken to carefully evaluate whether the MB-like signals seen here are truly derived from an MB-RL mechanism or instead reflect predictions about hidden states that can be used as inputs for learning by MF systems (Blanco-Pozo et al., 2024).

It is well established that ACC neurons can multiplex task variables during decision-making tasks (Kennerley et al., 2011; Hayden et al., 2011; Kennerley and Wallis, 2009; Hunt et al., 2018). The interaction between reward and transition coding is particularly critical for MB-RL valuation and choice in the two-step task. Here, we demonstrate that ACC was the primary region to simultaneously encode key variables of the task structure for MB choice, namely the interaction between reward, transition, and choice. This was also borne out in the computationally derived MB estimates of the first-stage choice, where ACC activity (and to a lesser extent caudate) strongly encoded this information. Another clear distinction between ACC and the other regions when coding either reward or transition information was the timescale of responses. ACC neurons have some of the slowest intrinsic timescales (i.e. the rate of their autocorrelation decay) across the brain (Murray et al., 2014; Cavanagh et al., 2018), which likely underlies the persistent encoding of these task variables. Maintaining these representations over long time periods likely potentiates the ability to contextualise outcomes by state, as well as the monitoring of action-outcome relationships relevant for behavioural adjustment (Hadland et al., 2003; Kennerley et al., 2006; Rudebeck et al., 2008). In fact, the encoding of the next trial’s chosen cue and its expected value was evident in ACC long before the choice epoch even started. Thus, our data provide novel correlates of prospective planning in single-neuron activity (Doll et al., 2015) and contributes to our understanding of the mechanisms underlying MB-RL.

Along with ACC, caudate appears to have an important role in MB behaviour. Choice 1 decoding was most strongly represented by ACC and caudate, and specifically by neurons that tracked both reward and transition information. Furthermore, the distinctive caudate signal of updating (flipping) the value estimates of the currently experienced option on rare trials implies this signal is not a general temporal-difference RPE, but instead more reflective of a state PE (Gläscher et al., 2010) and further supports the role of caudate in MB valuation. Our recordings were predominantly in anterior parts of caudate, which is where others have described flexible value coding (Kim and Hikosaka, 2013), representations of sequences (Seo et al., 2012), and preference for early phases of learning (Pasupathy and Miller, 2005) – features more often embraced by MB-RL. Anatomically, it is also the part of the caudate with the highest afferent projections from ACC – the area we observed to support MB computations (Griggs et al., 2017).

On the other hand, MF-based estimates were neither as striking nor as specific to striatal regions as expected and observed in previous studies (Daw et al., 2011). The monkeys were extensively trained on the task before recordings commenced, which may have caused a shift towards both MB behaviour and MB value representation within the striatum. Alternatively, this training may have allowed more sophisticated representations to occur, such as using latent states to expand the task space (Akam et al., 2015). Despite this, ACC was the only region that significantly encoded both MB and MF values. Having access to both MF and MB values at the time of choice implicates ACC in the arbitration process between the two learning strategies (Lee et al., 2014) that has been considered crucial to guide optimal behaviour. It is also in line with alternative theoretical proposals for ACC that propose a specific role in monitoring the error likelihoods of all possible expected outcomes (Alexander and Brown, 2011). In fact, the use of an RL framework could indeed unify other roles attributed to ACC, in particular conflict resolution (van Veen et al., 2001) and cognitive control (Shenhav et al., 2013).

All data and code to reproduce figures are available at https://github.com/jamesbutler01/TwoStepExperiment, copy archived at Butler, 2026.

1. 1. Akam T
   2. Costa R
   3. Dayan P

   (2015) Simple plans or sophisticated habits? State, transition and learning interactions in the two-step task

   PLOS Computational Biology 11:e1004648.

   https://doi.org/10.1371/journal.pcbi.1004648

   • PubMed
   • Google Scholar
2. 1. Akam T
   2. Rodrigues-Vaz I
   3. Marcelo I
   4. Zhang X
   5. Pereira M
   6. Oliveira RF
   7. Dayan P
   8. Costa RM

   (2021) The anterior cingulate cortex predicts future states to mediate model-based action selection

   Neuron 109:149–163.

   https://doi.org/10.1016/j.neuron.2020.10.013

   • PubMed
   • Google Scholar
3. 1. Alexander WH
   2. Brown JW

   (2011) Medial prefrontal cortex as an action-outcome predictor

   Nature Neuroscience 14:1338–1344.

   https://doi.org/10.1038/nn.2921

   • PubMed
   • Google Scholar
4. 1. Antonov G
   2. Gagne C
   3. Eldar E
   4. Dayan P

   (2022) Optimism and pessimism in optimised replay

   PLOS Computational Biology 18:e1009634.

   https://doi.org/10.1371/journal.pcbi.1009634

   • PubMed
   • Google Scholar
5. 1. Apicella P
   2. Ljungberg T
   3. Scarnati E
   4. Schultz W

   (1991) Responses to reward in monkey dorsal and ventral striatum

   Experimental Brain Research 85:491–500.

   https://doi.org/10.1007/BF00231732

   • PubMed
   • Google Scholar
6. 1. Apicella P
   2. Deffains M
   3. Ravel S
   4. Legallet E

   (2009) Tonically active neurons in the striatum differentiate between delivery and omission of expected reward in a probabilistic task context

   The European Journal of Neuroscience 30:515–526.

   https://doi.org/10.1111/j.1460-9568.2009.06872.x

   • PubMed
   • Google Scholar
7. 1. Balleine BW
   2. O’Doherty JP

   (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

   Neuropsychopharmacology 35:48–69.

   https://doi.org/10.1038/npp.2009.131

   • PubMed
   • Google Scholar
8. 1. Bayer HM
   2. Glimcher PW

   (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal

   Neuron 47:129–141.

   https://doi.org/10.1016/j.neuron.2005.05.020

   • PubMed
   • Google Scholar
9. 1. Behrens TEJ
   2. Muller TH
   3. Whittington JCR
   4. Mark S
   5. Baram AB
   6. Stachenfeld KL
   7. Kurth-Nelson Z

   (2018) What is a cognitive map? Organizing knowledge for flexible behavior

   Neuron 100:490–509.

   https://doi.org/10.1016/j.neuron.2018.10.002

   • PubMed
   • Google Scholar
10. 1. Blanco-Pozo M
    2. Akam T
    3. Walton ME

    (2024) Dopamine-independent effect of rewards on choices through hidden-state inference

    Nature Neuroscience 27:286–297.

    https://doi.org/10.1038/s41593-023-01542-x

    • PubMed
    • Google Scholar
11. 1. Boorman ED
    2. Rajendran VG
    3. O’Reilly JX
    4. Behrens TE

    (2016) Two anatomically and computationally distinct learning signals predict changes to stimulus-outcome associations in hippocampus

    Neuron 89:1343–1354.

    https://doi.org/10.1016/j.neuron.2016.02.014

    • PubMed
    • Google Scholar
12. Software

    1. Butler JL

    (2026) TwoStepExperiment, version swh:1:rev:ef32a40742af500dd55584218007d8f515d6c107

    Software Heritage.

    https://archive.softwareheritage.org/swh:1:dir:8571db215b2de578d656d7726b304d71c6744ad8;origin=https://github.com/jamesbutler01/TwoStepExperiment;visit=swh:1:snp:070f83ab9f17424b006d0452efd29e2f84806742;anchor=swh:1:rev:ef32a40742af500dd55584218007d8f515d6c107
13. 1. Cavanagh SE
    2. Towers JP
    3. Wallis JD
    4. Hunt LT
    5. Kennerley SW

    (2018) Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex

    Nature Communications 9:3498.

    https://doi.org/10.1038/s41467-018-05873-3

    • PubMed
    • Google Scholar
14. 1. Chan SCY
    2. Schuck NW
    3. Lopatina N
    4. Schoenbaum G
    5. Niv Y

    (2021) Orbitofrontal cortex and learning predictions of state transitions

    Behavioral Neuroscience 135:487–497.

    https://doi.org/10.1037/bne0000461

    • PubMed
    • Google Scholar
15. 1. Daw ND
    2. Niv Y
    3. Dayan P

    (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

    Nature Neuroscience 8:1704–1711.

    https://doi.org/10.1038/nn1560

    • PubMed
    • Google Scholar
16. 1. Daw ND
    2. Gershman SJ
    3. Seymour B
    4. Dayan P
    5. Dolan RJ

    (2011) Model-based influences on humans’ choices and striatal prediction errors

    Neuron 69:1204–1215.

    https://doi.org/10.1016/j.neuron.2011.02.027

    • PubMed
    • Google Scholar
17. 1. Daw ND
    2. Dayan P

    (2014) The algorithmic anatomy of model-based evaluation

    Philosophical Transactions of the Royal Society B 369:20130478.

    https://doi.org/10.1098/rstb.2013.0478

    • Google Scholar
18. 1. Dickinson A

    (1985) Actions and habits: the development of behavioural autonomy

    Philosophical Transactions of the Royal Society of London. B, Biological Sciences 308:67–78.

    https://doi.org/10.1098/rstb.1985.0010

    • Google Scholar
19. 1. Dickinson A
    2. Balleine B

    (1994) Motivational control of goal-directed action

    Animal Learning & Behavior 22:1–18.

    https://doi.org/10.3758/BF03199951

    • Google Scholar
20. 1. Dolan RJ
    2. Dayan P

    (2013) Goals and habits in the brain

    Neuron 80:312–325.

    https://doi.org/10.1016/j.neuron.2013.09.007

    • PubMed
    • Google Scholar
21. 1. Doll BB
    2. Duncan KD
    3. Simon DA
    4. Shohamy D
    5. Daw ND

    (2015) Model-based choices involve prospective neural activity

    Nature Neuroscience 18:767–772.

    https://doi.org/10.1038/nn.3981

    • PubMed
    • Google Scholar
22. 1. Elber-Dorozko L
    2. Loewenstein Y

    (2018) Striatal action-value neurons reconsidered

    eLife 7:e34248.

    https://doi.org/10.7554/eLife.34248

    • PubMed
    • Google Scholar
23. 1. Eldar E
    2. Lièvre G
    3. Dayan P
    4. Dolan RJ

    (2020) The roles of online and offline replay in planning

    eLife 9:e56911.

    https://doi.org/10.7554/eLife.56911

    • PubMed
    • Google Scholar
24. 1. Gläscher J
    2. Daw N
    3. Dayan P
    4. O’Doherty JP

    (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning

    Neuron 66:585–595.

    https://doi.org/10.1016/j.neuron.2010.04.016

    • PubMed
    • Google Scholar
25. 1. Griggs WS
    2. Kim HF
    3. Ghazizadeh A
    4. Costello MG
    5. Wall KM
    6. Hikosaka O

    (2017) Flexible and stable value coding areas in caudate head and tail receive anatomically distinct cortical and subcortical inputs

    Frontiers in Neuroanatomy 11:106.

    https://doi.org/10.3389/fnana.2017.00106

    • PubMed
    • Google Scholar
26. 1. Haber SN
    2. Behrens TEJ

    (2014) The neural network underlying incentive-based learning: implications for interpreting circuit disruptions in psychiatric disorders

    Neuron 83:1019–1039.

    https://doi.org/10.1016/j.neuron.2014.08.031

    • PubMed
    • Google Scholar
27. 1. Hadland KA
    2. Rushworth MFS
    3. Gaffan D
    4. Passingham RE

    (2003) The anterior cingulate and reward-guided selection of actions

    Journal of Neurophysiology 89:1161–1164.

    https://doi.org/10.1152/jn.00634.2002

    • PubMed
    • Google Scholar
28. Preprint

    1. Harris KD

    (2020) Nonsense correlations in neuroscience

    bioRxiv.

    https://doi.org/10.1101/2020.11.29.402719

    • Google Scholar
29. 1. Hayden BY
    2. Pearson JM
    3. Platt ML

    (2011) Neuronal basis of sequential foraging decisions in a patchy environment

    Nature Neuroscience 14:933–939.

    https://doi.org/10.1038/nn.2856

    • PubMed
    • Google Scholar
30. 1. Hunt LT
    2. Malalasekera WMN
    3. de Berker AO
    4. Miranda B
    5. Farmer SF
    6. Behrens TEJ
    7. Kennerley SW

    (2018) Triple dissociation of attention and decision computations across prefrontal cortex

    Nature Neuroscience 21:1471–1481.

    https://doi.org/10.1038/s41593-018-0239-5

    • PubMed
    • Google Scholar
31. 1. Johnson A
    2. Redish AD

    (2007) Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point

    The Journal of Neuroscience 27:12176–12189.

    https://doi.org/10.1523/JNEUROSCI.3761-07.2007

    • PubMed
    • Google Scholar
32. 1. Karlsson MP
    2. Tervo DGR
    3. Karpova AYS

    (2012) Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty

    Science 338:135–139.

    https://doi.org/10.1126/science.1226518

    • PubMed
    • Google Scholar
33. 1. Kennerley SW
    2. Walton ME
    3. Behrens TEJ
    4. Buckley MJ
    5. Rushworth MFS

    (2006) Optimal decision making and the anterior cingulate cortex

    Nature Neuroscience 9:940–947.

    https://doi.org/10.1038/nn1724

    • PubMed
    • Google Scholar
34. 1. Kennerley SW
    2. Wallis JD

    (2009) Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus

    Journal of Neurophysiology 102:3352–3364.

    https://doi.org/10.1152/jn.00273.2009

    • PubMed
    • Google Scholar
35. 1. Kennerley SW
    2. Behrens TEJ
    3. Wallis JD

    (2011) Double dissociation of value computations in orbitofrontal and anterior cingulate neurons

    Nature Neuroscience 14:1581–1589.

    https://doi.org/10.1038/nn.2961

    • PubMed
    • Google Scholar
36. 1. Kim HF
    2. Hikosaka O

    (2013) Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values

    Neuron 79:1001–1010.

    https://doi.org/10.1016/j.neuron.2013.06.044

    • PubMed
    • Google Scholar
37. 1. Kolling N
    2. Wittmann MK
    3. Behrens TEJ
    4. Boorman ED
    5. Mars RB
    6. Rushworth MFS

    (2016) Value, search, persistence and model updating in anterior cingulate cortex

    Nature Neuroscience 19:1280–1285.

    https://doi.org/10.1038/nn.4382

    • PubMed
    • Google Scholar
38. 1. Lau B
    2. Glimcher PW

    (2008) Value representations in the primate striatum during matching behavior

    Neuron 58:451–463.

    https://doi.org/10.1016/j.neuron.2008.02.021

    • PubMed
    • Google Scholar
39. 1. Lee SW
    2. Shimojo S
    3. O’Doherty JP

    (2014) Neural computations underlying arbitration between model-based and model-free learning

    Neuron 81:687–699.

    https://doi.org/10.1016/j.neuron.2013.11.028

    • Google Scholar
40. 1. Liu Y
    2. Mattar MG
    3. Behrens TEJ
    4. Daw ND
    5. Dolan RJ

    (2021) Experience replay is associated with efficient nonlocal learning

    Science 372:abf1357.

    https://doi.org/10.1126/science.abf1357

    • Google Scholar
41. 1. Matsumoto M
    2. Matsumoto K
    3. Abe H
    4. Tanaka K

    (2007) Medial prefrontal cell activity signaling prediction errors of action values

    Nature Neuroscience 10:647–656.

    https://doi.org/10.1038/nn1890

    • PubMed
    • Google Scholar
42. 1. Mattar MG
    2. Daw N

    (2018) Prioritized memory access explains planning and hippocampal replay

    Nature Neuroscience 21:1609–1617.

    https://doi.org/10.1038/s41593-018-0232-z

    • PubMed
    • Google Scholar
43. 1. Miller KJ
    2. Botvinick MM
    3. Brody CD

    (2017) Dorsal hippocampus contributes to model-based planning

    Nature Neuroscience 20:1269–1276.

    https://doi.org/10.1038/nn.4613

    • PubMed
    • Google Scholar
44. 1. Miranda B
    2. Malalasekera WMN
    3. Behrens TE
    4. Dayan P
    5. Kennerley SW

    (2020) Combined model-free and model-sensitive reinforcement learning in non-human primates

    PLOS Computational Biology 16:e1007944.

    https://doi.org/10.1371/journal.pcbi.1007944

    • PubMed
    • Google Scholar
45. 1. Moran R
    2. Keramati M
    3. Dolan RJ

    (2021) Model based planners reflect on their model-free propensities

    PLOS Computational Biology 17:e1008552.

    https://doi.org/10.1371/journal.pcbi.1008552

    • PubMed
    • Google Scholar
46. 1. Morris A
    2. Cushman F

    (2019) Model-Free RL or action sequences?

    Frontiers in Psychology 10:2892.

    https://doi.org/10.3389/fpsyg.2019.02892

    • PubMed
    • Google Scholar
47. 1. Muller TH
    2. Butler JL
    3. Veselic S
    4. Miranda B
    5. Wallis JD
    6. Dayan P
    7. Behrens TEJ
    8. Kurth-Nelson Z
    9. Kennerley SW

    (2024) Distributional reinforcement learning in prefrontal cortex

    Nature Neuroscience 27:403–408.

    https://doi.org/10.1038/s41593-023-01535-w

    • PubMed
    • Google Scholar
48. 1. Murray JD
    2. Bernacchia A
    3. Freedman DJ
    4. Romo R
    5. Wallis JD
    6. Cai X
    7. Padoa-Schioppa C
    8. Pasternak T
    9. Seo H
    10. Lee D
    11. Wang X-J

    (2014) A hierarchy of intrinsic timescales across primate cortex

    Nature Neuroscience 17:1661–1663.

    https://doi.org/10.1038/nn.3862

    • PubMed
    • Google Scholar
49. 1. O’Reilly JX
    2. Schüffelgen U
    3. Cuell SF
    4. Behrens TEJ
    5. Mars RB
    6. Rushworth MFS

    (2013) Dissociable effects of surprise and model update in parietal and anterior cingulate cortex

    PNAS 110:1305373110.

    https://doi.org/10.1073/pnas.1305373110

    • Google Scholar
50. 1. Pasupathy A
    2. Miller EKN

    (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum

    Nature 433:873–876.

    https://doi.org/10.1038/nature03287

    • PubMed
    • Google Scholar
51. 1. Platt ML
    2. Glimcher PW

    (1999) Neural correlates of decision variables in parietal cortex

    Nature 400:233–238.

    https://doi.org/10.1038/22268

    • PubMed
    • Google Scholar
52. 1. Roesch MR
    2. Olson CR

    (2003) Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex

    Journal of Neurophysiology 90:1766–1789.

    https://doi.org/10.1152/jn.00019.2003

    • PubMed
    • Google Scholar
53. 1. Rudebeck PH
    2. Behrens TE
    3. Kennerley SW
    4. Baxter MG
    5. Buckley MJ
    6. Walton ME
    7. Rushworth MFS

    (2008) Frontal cortex subregions play distinct roles in choices between actions and stimuli

    The Journal of Neuroscience 28:13775–13785.

    https://doi.org/10.1523/JNEUROSCI.3541-08.2008

    • PubMed
    • Google Scholar
54. 1. Schuck NW
    2. Cai MB
    3. Wilson RC
    4. Niv Y

    (2016) Human orbitofrontal cortex represents a cognitive map of state space

    Neuron 91:1402–1412.

    https://doi.org/10.1016/j.neuron.2016.08.019

    • PubMed
    • Google Scholar
55. 1. Schultz W
    2. Dayan P
    3. Montague PRS

    (1997) A neural substrate of prediction and reward

    Science 275:1593–1599.

    https://doi.org/10.1126/science.275.5306.1593

    • Google Scholar
56. 1. Seo H
    2. Lee D

    (2007) Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game

    The Journal of Neuroscience 27:8366–8377.

    https://doi.org/10.1523/JNEUROSCI.2369-07.2007

    • PubMed
    • Google Scholar
57. 1. Seo M
    2. Lee E
    3. Averbeck BB

    (2012) Action selection and action value in frontal-striatal circuits

    Neuron 74:947–960.

    https://doi.org/10.1016/j.neuron.2012.03.037

    • PubMed
    • Google Scholar
58. 1. Sezener CE
    2. Dezfouli A
    3. Keramati M

    (2019) Optimizing the depth and the direction of prospective planning using information values

    PLOS Computational Biology 15:e1006827.

    https://doi.org/10.1371/journal.pcbi.1006827

    • PubMed
    • Google Scholar
59. 1. Shenhav A
    2. Botvinick MM
    3. Cohen JD

    (2013) The expected value of control: an integrative theory of anterior cingulate cortex function

    Neuron 79:217–240.

    https://doi.org/10.1016/j.neuron.2013.07.007

    • PubMed
    • Google Scholar
60. 1. Shin EJ
    2. Jang Y
    3. Kim S
    4. Kim H
    5. Cai X
    6. Lee H
    7. Sul JH
    8. Lee S-H
    9. Chung Y
    10. Lee D
    11. Jung MW

    (2021) Robust and distributed neural representation of action values

    eLife 10:e53045.

    https://doi.org/10.7554/eLife.53045

    • PubMed
    • Google Scholar
61. 1. Sutton R

    (1988) Learning to predict by the methods of temporal differences

    Machine Learning 3:9–44.

    https://doi.org/10.1023/A:1022633531479

    • Google Scholar
62. Book

    1. Sutton RS
    2. Barto AG
    3. Bach F

    (2018)

    Reinforcement Learning: An Introduction

    The MIT Press.

    • Google Scholar
63. 1. Tanaka SC
    2. Balleine BW
    3. O’Doherty JP

    (2008) Calculating consequences: brain systems that encode the causal effects of actions

    The Journal of Neuroscience 28:6750–6755.

    https://doi.org/10.1523/JNEUROSCI.1808-08.2008

    • PubMed
    • Google Scholar
64. Book

    1. Thorndike E

    (1911) Animal Intelligence: Experimental Studies

    Macmillan Company.

    https://doi.org/10.5962/bhl.title.55072

    • Google Scholar
65. 1. Tolman EC

    (1948) Cognitive maps in rats and men

    Psychological Review 55:189–208.

    https://doi.org/10.1037/h0061626

    • PubMed
    • Google Scholar
66. 1. Tricomi E
    2. Balleine BW
    3. O’Doherty JP

    (2009) A specific role for posterior dorsolateral striatum in human habit learning

    The European Journal of Neuroscience 29:2225–2232.

    https://doi.org/10.1111/j.1460-9568.2009.06796.x

    • PubMed
    • Google Scholar
67. 1. Valentin VV
    2. Dickinson A
    3. O’Doherty JP

    (2007) Determining the neural substrates of goal-directed learning in the human brain

    The Journal of Neuroscience 27:4019–4026.

    https://doi.org/10.1523/JNEUROSCI.0564-07.2007

    • PubMed
    • Google Scholar
68. 1. van Veen V
    2. Cohen JD
    3. Botvinick MM
    4. Stenger VA
    5. Carter CS

    (2001) Anterior cingulate cortex, conflict monitoring, and levels of processing

    NeuroImage 14:1302–1308.

    https://doi.org/10.1006/nimg.2001.0923

    • PubMed
    • Google Scholar
69. 1. Wallis JD
    2. Kennerley SW

    (2010) Heterogeneous reward signals in prefrontal cortex

    Current Opinion in Neurobiology 20:191–198.

    https://doi.org/10.1016/j.conb.2010.02.009

    • PubMed
    • Google Scholar
70. 1. Wunderlich K
    2. Dayan P
    3. Dolan RJ

    (2012) Mapping value based planning and extensively trained choice in the human brain

    Nature Neuroscience 15:786–791.

    https://doi.org/10.1038/nn.3068

    • PubMed
    • Google Scholar
71. 1. Yin HH
    2. Knowlton BJ
    3. Balleine BW

    (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

    The European Journal of Neuroscience 19:181–189.

    https://doi.org/10.1111/j.1460-9568.2004.03095.x

    • PubMed
    • Google Scholar
72. 1. Yin HH
    2. Ostlund SB
    3. Knowlton BJ
    4. Balleine BW

    (2005) The role of the dorsomedial striatum in instrumental conditioning

    The European Journal of Neuroscience 22:513–523.

    https://doi.org/10.1111/j.1460-9568.2005.04218.x

    • PubMed
    • Google Scholar

Author details

1. Bruno Miranda

   1. Institute of Neurology, Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom
   2. Instituto de Fisiologia, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Contributed equally with

   James L Butler

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-4660-6051

2. James L Butler

   Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

   Contribution

   Formal analysis, Visualization, Writing – original draft, Writing – review and editing

   Contributed equally with

   Bruno Miranda

   For correspondence

   james.butler@psy.ox.ac.uk

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5017-6472

3. WM Nishantha Malalasekera

   Institute of Neurology, Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom

   Contribution

   Data curation

   Competing interests

   No competing interests declared

4. Timothy EJ Behrens

   1. Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford, United Kingdom
   2. Sainsbury Wellcome Centre for Neural Circuits and Behaviour College, University College London, London, United Kingdom

   Contribution

   Conceptualization, Supervision, Funding acquisition, Writing – review and editing

   Competing interests

   Editor-in-Chief, eLife

   "This ORCID iD identifies the author of this article:" 0000-0003-0048-1177

5. Peter Dayan

   1. Max Planck Institute for Biological Cybernetics, Tübingen, Germany
   2. University of Tübingen, Tübingen, Germany

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-3476-1839

6. Steven W Kennerley

   1. Institute of Neurology, Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom
   2. Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

   Contribution

   Conceptualization, Data curation, Formal analysis, Supervision, Funding acquisition, Investigation, Visualization, Methodology, Writing – original draft, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5696-7507

• 1,521

  views

• 87

  downloads

• 0

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.