The orbitofrontal cortex (OFC) is often described as signaling either an outcome expectancy, implying a knowledge of the features of the impending outcome (Schoenbaum et al., 1998; Delamater, 2007; Luk and Wallis, 2013; Ostlund and Balleine, 2007; Steiner and Redish, 2012), or a value that exists independent of those features (Levy and Glimcher, 2012; Padoa-Schioppa, 2011). Notably, while support for abstract value encoding comes largely from studies employing economic decision-making procedures (Plassmann et al., 2007; Padoa-Schioppa and Assad, 2006; Levy and Glimcher, 2011), it has been suggested that the 'economic' value revealed by these procedures is the same as that underlying well-controlled OFC-dependent behavioral effects such as reinforcer devaluation or sensory preconditioning (Padoa-Schioppa, 2011). Notably, in these settings, the role of the OFC (West et al., 2011; Jones et al., 2012; Pickens et al., 2003) cannot be strictly explained by signaling of an abstract or outcome-independent value. Indeed, the behavior that requires OFC reflects a value that is inextricably linked to, or an attribute of, the predicted outcome - the reward in the case of reinforcer devaluation or the primary conditioned stimulus in the case of sensory preconditioning. This implies that OFC allows access to a representation of the outcome through which the value can be inferred or calculated on the fly, at the time the behavior is engaged. Thus, while the aforementioned correlates could reflect the final common calculation, it is equally reasonable that they might reflect the specific associative representations that are activated as part of this calculation. Note this is even true of coding changes with subjective preference (Padoa-Schioppa and Assad, 2006, supplemental), since the perception of attributes is influenced by valuation or preference (e.g., a nice warm cup of hot chocolate while skiing in the winter versus the same cup at the beach in the summer).

We recently tried to distinguish between these possibilities by using Pavlovian blocking to strip away or ‘block’ the value portion of the outcome during learning, while leaving unblocked the outcome’s unique sensory and other features (McDannald et al., 2014). We did this by pairing a ‘target’ cue with a rewarding outcome in the presence of a cue that had been previously trained to predict a differently-flavored, but similarly-valued outcome. When this is done, the previously conditioned cue predicts the value that is common to the two outcomes, but does not predict the unique features that distinguish the new outcome (note features are not limited to sensory properties, but might include the outcome timing, location, temperature, size, number, etc.). The target cue acquires associations with the unique features of the new outcome but not with its general value (Burke et al., 2008; Rescorla, 1999). Notably such unblocking is dependent on the OFC, whereas value-based unblocking is not (McDannald et al., 2011). In this prior study, we found that OFC neurons responded strongly to this valueless target cue (McDannald et al., 2014). In fact, they responded as strongly to this cue as they did to a control cue that was paired with an additional reward.

While this result was perhaps not surprising, given prior evidence that OFC neurons signal both value as well as sensory features of predicted rewards (Padoa-Schioppa and Assad, 2006), we were intrigued by the similarity in the strength of these representations. After all, if OFC is primarily concerned with representing value or at least with encoding things with biological significance, then why was the neural representation of a cue signaling a valueless change in reward features so similar to the representation of a cue that actually predicted a more valuable reward? Furthermore, we did not find any correlation in the firing of the value-predicting neurons across other cues present in the task that also predicted reward and value. This suggested to us that the neural activity to this cue was likely not only representing value but instead was representing the unique features that led to the value prediction in the case of that cue; e.g., the additional reward.

Here we have designed a follow-up experiment to test this question. In this experiment, we have copied the design of the prior study (McDannald et al., 2014), recording OFC neurons during unblocking with two changes. The first change was to replace our reward identity shift with a downward shift in value to directly test whether responses to additional reward in the prior study were related to value, which should be reversed in this condition, or some other aspect of the additional reward, which might be unrelated. The second change was to deliver a single drop of reward rather than multiple drops to minimize differences in features between our reward manipulations. We also shifted the size of the outcome rather than the number because number downshift unblocking results in excitatory behavior while a concentration shift does not (Esber and Holland, 2014). The delay to omitted reward in a number downshift condition reduces the downshift association with the cue. In contrast, in concentration downshift conditions, rats immediately know when they have received a reduced concentration and do not exhibit behavioral excitation. We shifted the size because it is more similar to concentration in terms of when rats learn that they have received the downshifted outcome. This change will enable us to differentiate between the upshifted and downshifted cues behaviorally because the former will lead to excitation and the latter inhibition.

Recording in this setting, we replicated our prior results. We found again that many OFC neurons developed responses to the cue that predicted more reward. Interestingly, these neurons again exhibited relatively restricted responses, suggesting they were signaling something unique about this manipulation. Furthermore, they were joined by other neural populations that developed similar responses to the cue that predicted less reward and even, to some extent, to the cue that predicted no change in reward. This pattern of activity is not consistent with coding of value independent of outcome features and instead suggests that much of the neural activity in the OFC is tightly linked to the attributes of impending events: value being one such attribute and outcomes being one such event.

We recorded single-unit activity in the OFC in 18 rats during an odor-based unblocking task (Figure 1a). After implantation of microelectrodes in the lateral OFC, rats were trained to sample an odor in a central port following house light illumination and then respond to a reward well below for a single medium-sized drop of flavored milk. This training was extensive, lasting for at least four days, and was meant to establish the initial odor as a reliable predictor of this specific outcome. Each rat then underwent 1–6 rounds of unblocking.

Figure 1

Download asset Open asset

Each round of unblocking began with two days of training and consisted of four trial types (Figure 1a, compound training). One type was a reminder: the initially trained odor was followed by the expected outcome. On the other three trial types (upshift, downshift, blocked), rats were presented with the initially trained odor, followed immediately by one of three novel odors. On blocked trials, the expected medium-sized drop of milk followed the novel odor. This outcome is fully predicted by the initial odor, thus the novel odor should be blocked from acquiring associative significance. On upshift trials, a noticeably larger drop of milk followed the novel odor. Here the value is increased by the additional amount, with minimal changes in features since the reward identity is the same, as is the location and timing of delivery. Thus the novel target odor should enter into associations with the higher value and possibly with the feature of ‘larger’. On downshift trials, a noticeably smaller drop of milk followed the novel odor. Here the value is decreased by the missing amount, thus the novel odor should enter into associations with the lower value and possibly with the feature of ‘smaller’. We reasoned that if neurons responding to the upshifted odor cue, apparent value coding neurons in the prior study, were in fact signaling value, then they should show less firing to the blocked cue (which predicts no change in value) and even less to the downshifted cue (which predicts diminished value) (Figure 1d, left). On the other hand, if upshift neurons were signaling the attribute of ‘larger’, then we might expect them to be tuned to this particular cue, with additional populations showing selectivity to the cues predicting ‘smaller’ and ‘no change' (Figure 1d, right).

Upshift-responsive neurons in the OFC do not signal value

We recorded 334 single units during the first day of unblocking and 346 units on the second unblocking day in 60 rounds of training across all 18 rats (Figure 1c). Characteristically, cells in these populations fired to all of the events that characterized the trials in our task, including prominently during odor sampling (Figure 1e–g, steepest portion of slope). To address our hypothesis, units with a baseline firing rate below 10 Hz from both unblocking days were screened for phasic responses to one of the four odors using a t-test, which compared firing rates during the ITI and novel odor period (significance level = p<0.01). Excluding non-associative neurons, which fired from the start of training to all of the odor cues (see Figure 2—figure supplement 1 for this and other response types not the focus of this paper and Figure 2—figure supplement 2 for activity of all units with a baseline firing rate below 10 Hz regardless of odor response), this screen identified 120 units (Day 1=65, Day 2=55) that showed a significant increase in firing to at least one of the odor cues but not all four odors similarly.

Our primary goal in the current experiment was to test whether upshift-responsive neurons, identified in our prior study, exhibit a firing pattern consistent with value signaling, or whether they might be signaling features. Thus we focused our initial analysis on the neurons (60/120) that showed a significant phasic response to the upshifted odor. For this analysis, we compared firing to the three novel odor cues via pairwise t-tests between the baseline firing rate in the 2 s prior to the light cue and the firing rate in the 1 s following novel odor onset (p<0.01). We isolated cells that responded significantly above baseline to the upshift odor but not all four odors similarly. Cells that responded to all four odors and were value-coding were included in the upshift-responsive population. This analysis revealed a diverse set of patterns within the upshift-responsive population. While these neurons exhibited a variety of patterns of activity to the other odor cues (Figure 2a–d), as a group they fired most strongly to the upshifted cue (Figure 2e) and this preference was acquired with learning (Figure 2f, g), demonstrating that it was not driven by physical properties of the odors but instead by something the odor cue predicted about reward.

Figure 2 with 2 supplements see all

Download asset Open asset

However, firing across the population did not appear to provide the linear value correlate predicted by a value-coding hypothesis (Figure 1d, left). This was evident in the population response, which while highest for the upshifted cue, did not distinguish between the blocked and downshifted cues (Figure 2e). Of course, the heterogeneity of response types within such a broad population could obscure a simple value correlate, so next we asked whether a subpopulation of the 60 upshift-responsive neurons might show a linear value correlate. Value-coding was operationally defined as an excitatory response to at least one odor and significantly more activity to the upshift cue than the blocked cue as well as significantly less activity to the downshift cue than the blocked cue. Few neurons exhibited firing that clearly rank ordered the cues according to their value (Figure 2k; 5/60; 8%). Though firing in these neurons did provide a linear value correlate (Figure 2p), this proportion was the smallest of the theoretically meaningful patterns that we found and its size was not larger than chance given our analytical approach. We repeated this analysis on all cells regardless of baseline firing rate and without limiting our analysis to odor-responsive cells and found a total of 6 cells, suggesting that our screen had not prevented us from finding value-coding cells.

On the other hand, we found many neurons that showed isolated firing to the upshifted odor alone (Figure 2h; 19/60; 32%). Indeed this was our largest subgroup. In these cells, we were only able to accurately classify the upshift trial type above chance, and no other trial type, suggesting that these cells only contain information about the upshift cue (Figure 2q). Additionally we saw an almost equally large population of neurons that fired similarly to the upshifted and downshifted cues (Figure 2i; 16/60; 27%). This population was reminiscent of the neurons in our prior study that fired similarly to the two cues predicting changes in the outcome. And finally we found a substantial number of upshift-responsive neurons that were equally responsive to the other two novel odor cues (Figure 2j; 10/60; 17%), replicating the finding of salience or novelty encoding in our prior study. Notably, none of these populations showed a relationship between firing to the upshift and downshift cues consistent with value coding across them (Figures 2m-o). Thus, if we disregard the very small number of neurons showing linear coding of value, our results here largely replicated what we observed when unblocking learning by shifting value versus identity (Figure 2l). Reducing the feature space by moving to a single bolus of reward and including an explicit bidirectional value shift did not reveal frank value coding in a large proportion of the neurons.

OFC neurons respond independently to the upshifted, downshifted, and blocked cues

The upshift-responsive population may respond in a non-linear fashion to value. It is not immediately clear why that would be, since rats have been exposed to the different valued rewards in the session, and thus firing in the OFC should scale to reflect the full range that is available (Padoa-Schioppa, 2009). While this is difficult to rule out, one way to address this is to turn to our alternative account, which is that the firing of these neurons represents the attribute or feature that is predicted by the cue, which is the increase in the size of the reward. If this is what is signaled by these neurons, then one would expect there to be similar populations firing to the cue predicting a decrease in reward size and perhaps to the cue signaling no change. These neurons would not have passed our first screen, since we specifically searched for upshift-responsive neurons. So to capture these, we returned to our original data set and repeated the analysis above, but this time examining firing to the other cues. This approach found cells that exhibited isolated firing to the downshifted cue (n = 16, Figure 3a, d, g) and to the blocked cue (n = 16, Figure 3b, e, h). These populations were similar to the neurons that showed isolated firing to the upshift cue (n = 19, Figure 3c, f, i). The difference in the number of cells that responded to each cue was not significant, chi-squared = 0.353, p = 0.84. In each case, cue-evoked firing developed with training (Figure 3j, k). This impression was confirmed by an ensemble analysis that tested how well these neurons could identify the trial type; this analysis found that classification accuracy by these cells during the odor-sampling period improved significantly from day 1 (n = 20, Figure 3l) to day 2 (n = 29, Figure 3m). Within day 1, classification accuracy was at chance during the odor-sampling period. There was significant correlation between trial number and classification (rho = 0.85, p<0.001) but accuracy did not increase to above chance until day 2, beginning on the first ten-trial window shown and remaining above chance for the majority of trial windows. The chance decoding in the first day reflects the finding that the responses of these neurons were not stable across these trials (Figure 3n). Further, high decoding accuracy on the second day suggests that a downstream structure could decode the trial type from these tuned cells. Though the magnitude of the change in firing rate between baseline and the preferred cue is ~2 spikes/second in tuned cells, the high decoding accuracy suggests that the variance in firing plays a significant role in downstream decoding. High decoding accuracy on day 2 suggests that there is low variance in this response, allowing a downstream structure to interpret a relatively small change in firing magnitude.

Figure 3 with 1 supplement see all

Download asset Open asset

Finally we also examined whether firing activity in these populations might reflect value or at least vigor of responding to that specific cue. For this we compared firing in each neuron in each population to the speed of responding on those trials and also on trials involving the other two cues. Although a handful of individual neurons showed a significant correlation with response latency, there was no systematic relationship ( Figure 3—figure supplement 1).

Value coding in OFC neurons is not ‘blocked’ by our task

We conducted our original study to isolate the role of OFC in representing valueless information about predicted outcomes. We used blocking and identity unblocking as a way to prevent information about general value from becoming associated with our cue of interest. We found that OFC neurons developed firing to a cue predicting a valueless change in reward identity. However, many OFC neurons also fired when we increased the amount of reward. Here we conducted a follow-up experiment designed to test whether these responses are signaling value or whether they might be signaling the feature of the larger reward. We found primarily the latter, with very few neurons showing linear value correlates.

However, one flaw in our design is that, of course, we intentionally used blocking to reduce or eliminate associations with general value. Our apparent failure to observe linear value correlates could be secondary to this manipulation. To address this, we ran a control experiment in which the trial structure was identical to the above experiment except that in compound training, rats did not receive training with novel cues in compound with the initial odor cue (Figure 4a). This change essentially eliminated the blocking of value, without changing any of the other aspects of the task. Accordingly, we found that the rats again showed evidence of learning about the reward predicted by the three cues, spending more time in the fluid well following sampling of the large-predictive odor and less time following sampling of the small-predictive odor, versus the medium-predictive cue, as if expecting more and less reward respectively after sampling these cues (Figure 4b).

Figure 4

Download asset Open asset

Against this backdrop, we then asked whether this modified design had uncovered the linear value coding that seemed to be missing in our unblocking design. We recorded 93 single units during the first day and 78 units on the second day in 16 rounds of training across all 7 rats (Figure 4c). As in our first experiment, these neurons exhibited firing to all of the events that described a trial, including prominently to the odor-sampling period (Figures 4f–h). Twenty-six units (Day 1=16, Day 2=10) showed a significant increase in firing to at least one but not all four of the odor cues similarly. We found no evidence of linear value coding (0/26, 0%); instead neurons in this population again tended to develop firing that was tuned to the individual cues (14/26, 54% including 8 small, 2 medium, and 4 large). Although we had a much smaller sample, the activity of these neurons looked similar to the activity of the tuned populations identified in our first experiment inasmuch as these neurons exhibited high firing to one of the odors and equally low firing to the other two (Figures 4d, e).

Neural signals in the OFC are often described as representing either outcome expectancies or abstract value. Although many studies have argued for one or the other, few have used behavioral designs that clearly dissociate predictions of these two hypotheses. Previously we addressed this question by using an unblocking procedure to strip away or ‘block’ the abstract value of the outcome during learning, while leaving unblocked – free to enter into associations – the outcome’s sensory and other unique features. This approach revealed that many neurons in the OFC developed firing to target cues that predicted a valueless change in the identity of the predicted outcome. This sort of associative encoding is consistent with the involvement of the orbitofrontal cortex in a variety of behaviors that require direct access to specific information about the outcome's sensory features (McDannald et al., 2005; 2011; Ostlund and Balleine, 2007; Howard et al., 2015).

However, we also found that many neurons developed firing to a cue that simply predicted the delivery of an additional drop of reward. Although this condition was included as a positive control condition (in the event we saw no change in activity to our other cue), we were intrigued because the firing to this cue was not substantially stronger or in any other way more robust than that to the cue that predicted an unambiguous change in reward identity. This seemed at odds with proposals that the primary or at least a major role of the OFC is to encode the abstract or general value that cues predicting reward acquire (i.e., independent of outcome features). We wondered whether firing to this cue was simply reflecting some unique feature of additional reward (the concept of more, its timing, the unique presence of a third drop) and not the abstract increase in value it represented.

Here we tested this hypothesis by repeating our prior study, to identify neurons that fired to the cue predicting more reward, while adding a condition in which a cue predicted less reward. We reasoned that if the neurons firing to more reward were signaling abstract value, independent of any features unique to that condition, then neurons that fire more to a cue predicting more reward should fire less to a cue predicting less reward (versus the cue predicting no change). On the other hand, if these neurons were signaling a unique feature of the larger reward - essentially something contributing to its identity - then these neurons would not become selective for the other cues.

Our results were clearly in line with the latter prediction. We saw only a small population of neurons that exhibited firing consistent with abstract value coding across the three cues (upshift, downshift, blocked) and much larger groups of neurons that became responsive to each cue with training. Notably this was true in both the first experiment, in which we used a pre-trained ‘blocking’ cue as we had in the first study, and it was also true in the second experiment, in which we eliminated this pre-training. Thus, we do not believe that our failure to observe abstract or general value coding across cues was due to blocking of value by the pre-trained cue.

The firing that develops to these cues may reflect associations with novel features of the outcome delivered on these trial types (larger, smaller, unchanged). Such associations are known to develop when additional rewards are delivered, even during unblocking (Holland, 1984). This information encoded across an ensemble of neurons would clearly be relevant to determining the value of the impending outcome. However, our findings are not consistent with the proposal that these neurons directly signal the abstract or linear value that is acquired across cues at the single unit level. Indeed we saw almost no neurons in the OFC that fit this profile in our experiments.

So what does our failure to find these correlates mean? Many prior studies have reported value coding in the OFC (Plassmann et al., 2007; Padoa-Schioppa and Assad, 2006; Levy and Glimcher, 2011; Padoa-Schioppa, 2009; Tremblay and Schultz, 1999; Padoa-Schioppa and Assad, 2008; Padoa-Schioppa, 2013; Cai and Padoa-Schioppa, 2014; McNamee et al., 2013; Hare et al., 2008; 2010; Plassmann et al., 2010; Kahnt et al., 2014; Strait et al., 2014). How do we reconcile our results with these prior reports? A number of possible ways exist to understand this apparent discrepancy. The first is that it represents a species difference. Most of the aforementioned studies have focused on ‘economic’ value and decision-making. These models are largely applied to humans and monkeys. It is possible that the increased size and complexity of the OFC in primate species has led to the development of this abstract value coding. This is possible, though we believe it is unlikely, since most of the anatomical and functional data across species is roughly similar (Rudebeck and Murray, 2014; Wallis, 2012; Stalnaker, et al., 2015). This is especially true of the importance of the OFC to reinforcer devaluation (Gottfried, 2003; Gallagher et al., 1999; Izquierdo et al., 2004), a function first identified in rodents. To the extent economic value is sensitive to devaluation (Padoa-Schioppa, 2011; Padoa-Schioppa and Schoenbaum, 2015), this suggests that the role of the OFC in contributing to its representation is not unique to primates.

A second possibility relating to species differences is the accompanying differences in task design. Studies claiming value effects on neural activity in OFC have typically tested in a situation requiring choices between outcomes. This may make value information particularly salient compared to our task where explicit comparisons are not required. We view this as unlikely, since a core requirement of value signaling is automaticity (Lebreton et al., 2015). Nevertheless, this does not preclude differences in how this tracked value is represented in neural activity in the OFC. While the OFC may perform the same function between Pavlovian and choice tasks, there may be differences in how each is encoded.

A third possibility is that we are in the wrong part of the OFC to find these correlates. We are recording primarily in the lateral OFC. This is an area distinguished by a pattern of connectivity with amygdala, striatal and cortical areas that is qualitatively most similar to what is termed the lateral orbital network in primates (Ongur, 2000; Price, 2007). Most of the studies that have identified abstract value correlates in the OFC found them in more medial areas (Plassmann et al., 2007; 2010; Levy and Glimcher, 2011; McNamee et al., 2013; Hare et al., 2008; 2010; Strait et al., 2014). Further, studies in primates that have tried to distinguish medial versus lateral functions have assigned valuation based functions more to medial areas and functions such as credit assignment and representation of reward identity to more lateral areas (Noonan et al., 2010; 2012). It may be that the sort of linear value correlates that we were looking for are to be found in medial OFC. Indeed recent work in humans has shown that OFC represents specific outcome features and that more lateral orbital areas represent those outcomes in a way that is dependent upon prior cues (Klein-Flugge et al., 2013). In this regard, it is worth noting that the lateral part of the OFC is not normally necessary for behaviors in which value independent of outcome features is required. For example, the OFC is not required for simple Pavlovian or instrumental conditioning (Ostlund and Balleine, 2007; Gallagher et al., 1999; Gremel and Costa, 2013; Izquierdo et al., 2004) discrimination learning (Izquierdo et al., 2004; Walton et al., 2010; McDannald et al., 2005), extinction by reward omission (Takahashi et al., 2009), transfer (Ostlund and Balleine, 2007), and even perhaps reversal learning (Rudebeck et al., 2013), all of which can be accomplished without reference to specific information about predicted outcomes. Similarly, both blocking and unblocking – when it can be accounted for by value – do not require the OFC (Burke et al., 2008; McDannald et al., 2011). However, the OFC is necessary for superficially similar behaviors (Pavlovian or instrumental responding, discriminations, even learning) when they require knowledge of the outcome features in order to recognize errors or to derive or infer a value (Ostlund and Balleine, 2007; Gallagher et al., 1999; Gremel and Costa, 2013; Izquierdo et al., 2004; McDannald et al., 2011). This is even true in unblocking, where we have shown that the OFC is required for the development of conditioned responding to a target cue paired with a shift in outcome identity but not to a target cue paired with additional outcome (McDannald et al., 2011).

A fourth possible explanation for the apparent discrepancy between our results and many of these studies lies in the difference in measures. The vast majority of these studies have reported that BOLD signal correlates with abstract value (Plassmann et al., 2007; 2010; Levy and Glimcher, 2011; McNamee et al., 2013; Hare et al., 2008; 2010). BOLD signal differs from single unit spiking in at least three important ways that are relevant to this question. First, BOLD signal obviously reflects the summed activity of many neurons. Thus it identifies what the global summed processing across a large ensemble is tracking. This might be very different from what is encoded by groups of individual single units. Without knowing more about specific connectivity between neurons within an area, it is impossible to know if one or the other result is more relevant to the function of the area. But at a minimum, this might lead to very different results. The second way these two measures differ is that BOLD signal is more likely to reflect neural processing other than what is output from an area. This is because it reflects energy usage, which is a measure that does not distinguish between local and long distance communication. On the other hand, extracellular recording electrodes are heavily biased to record from large, regular spiking neurons (McCormick et al., 1985), which are likely to be the output neurons, at least of cortical regions. Again this might cause very different signals to be detected using the two methods. Finally the third and related difference between these two measures is that fMRI can be heavily influenced by input to an area (Logothetis et al., 2001; Logothetis and Pfeuffer, 2004; Logothetis and Wandell, 2004; Logothetis, 2008). Again this is because it measures energy usage and thus will be sensitive to the EPSP's (and IPSP's) due to afferent input. Note this is true even if these inputs do not lead to summation and action potential generation at the downstream hillock of a large output neuron (a very energy efficient event). For all these reasons, it is to be expected that fMRI studies and single unit studies may often report divergent findings.

Finally it is worth noting that neuroeconomic value may not be the same sort of value identified in our lexicon as pure, abstract, general or cached (i.e., independent of outcome features). This has been our reading of the original reports (Padoa-Schioppa and Assad, 2006; Levy and Glimcher, 2011; Plassmann et al., 2007). However one of the authors of these studies has suggested more recently that neuroeconomic value is in fact tied closely to and derived from knowledge of the features of the predicted outcome (Padoa-Schioppa and Schoenbaum, 2015). He states unequivocally that the economic value that correlates with firing in the OFC in monkeys is the same value that changes with devaluation and that it is tied inextricably to the identity of the predicted outcome. Indeed devaluation-sensitivity is taken to be an iconic feature of decisions based on economic value (Padoa-Schioppa, 2011). Notably, this definition would align our concept of signaling of outcome features with signaling of economic value, since economic value would then be just another attribute of the outcome. Of course, we would not expect it to be shared across outcomes or even necessarily across cues, particularly in a very lightly trained animal such as that in our current study. In this context, we would predict relatively specific representations and relatively little representation of this value across outcomes. This is what we have observed here. It is possible with extended training, such as that in studies reporting large populations of single unit coding value (Padoa-Schioppa and Assad, 2006; 2008; Padoa-Schioppa, 2009; 2013; Cai and Padoa-Schioppa, 2014), that the representations generalize as the rewards used become part of a ‘goods space’ as it is defined in these prior studies. As long as what is being signaled by such activity remains an attribute or feature of the predicted outcome - and sensitive to changes in that feature without further learning - then this would be consistent with our current data.

1. 1. Burke KA
   2. Franz TM
   3. Miller DN
   4. Schoenbaum G

   (2008) The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards

   Nature 454:340–344.

   https://doi.org/10.1038/nature06993

   • Google Scholar
2. 1. Cai X
   2. Padoa-Schioppa C

   (2014) Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation

   Neuron 81:1140–1151.

   https://doi.org/10.1016/j.neuron.2014.01.008

   • Google Scholar
3. 1. Combrisson E
   2. Jerbi K

   (2015) Exceeding chance level by chance: the caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy

   Journal of Neuroscience Methods 250:126–136.

   https://doi.org/10.1016/j.jneumeth.2015.01.010

   • Google Scholar
4. 1. Delamater AR

   (2007) The role of the orbitofrontal cortex in sensory-specific encoding of associations in pavlovian and instrumental conditioning

   Annals of the New York Academy of Sciences 1121:152–173.

   https://doi.org/10.1196/annals.1401.030

   • Google Scholar
5. 1. Esber GR
   2. Holland PC

   (2014) The basolateral amygdala is necessary for negative prediction errors to enhance cue salience, but not to produce conditioned inhibition

   The European Journal of Neuroscience 40:3328–3337.

   https://doi.org/10.1111/ejn.12695

   • Google Scholar
6. 1. Gallagher M
   2. McMahan RW
   3. Schoenbaum G

   (1999)

   Orbitofrontal cortex and representation of incentive value in associative learning

   Journal of Neuroscience 19:6610–6614.

   • Google Scholar
7. 1. Gottfried JA
   2. O'Doherty J
   3. Dolan RJ

   (2003) Encoding predictive reward value in human amygdala and orbitofrontal cortex

   Science 301:1104–1107.

   https://doi.org/10.1126/science.1087919

   • Google Scholar
8. 1. Gremel CM
   2. Costa RM

   (2013) Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions

   Nature Communications 4:2264.

   https://doi.org/10.1038/ncomms3264

   • Google Scholar
9. 1. Hare TA
   2. Camerer CF
   3. Knoepfle DT
   4. Rangel A

   (2010) Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition

   Journal of Neuroscience 30:583–590.

   https://doi.org/10.1523/JNEUROSCI.4089-09.2010

   • Google Scholar
10. 1. Hare TA
    2. O'Doherty J
    3. Camerer CF
    4. Schultz W
    5. Rangel A

    (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors

    Journal of Neuroscience 28:5623–5630.

    https://doi.org/10.1523/JNEUROSCI.1309-08.2008

    • Google Scholar
11. 1. Holland PC

    (1984) Unblocking in pavlovian appetitive conditioning

    Journal of Experimental Psychology. Animal Behavior Processes 10:476–497.

    https://doi.org/10.1037/0097-7403.10.4.476

    • Google Scholar
12. 1. Howard JD
    2. Gottfried JA
    3. Tobler PN
    4. Kahnt T

    (2015) Identity-specific coding of future rewards in the human orbitofrontal cortex

    Proceedings of the National Academy of Sciences of the United States of America 112:5195–5200.

    https://doi.org/10.1073/pnas.1503550112

    • Google Scholar
13. 1. Izquierdo A
    2. Suda RK
    3. Murray EA

    (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency

    Journal of Neuroscience 24:7540–7548.

    https://doi.org/10.1523/JNEUROSCI.1921-04.2004

    • Google Scholar
14. 1. Jones JL
    2. Esber GR
    3. McDannald MA
    4. Gruber AJ
    5. Hernandez A
    6. Mirenzi A
    7. Schoenbaum G

    (2012) Orbitofrontal cortex supports behavior and learning using inferred but not cached values

    Science 338:953–956.

    https://doi.org/10.1126/science.1227489

    • Google Scholar
15. 1. Kahnt T
    2. Park SQ
    3. Haynes J-D
    4. Tobler PN

    (2014) Disentangling neural representations of value and salience in the human brain

    Proceedings of the National Academy of Sciences of the United States of America 111:5000–5005.

    https://doi.org/10.1073/pnas.1320189111

    • Google Scholar
16. 1. Klein-Flügge MC
    2. Barron HC
    3. Brodersen KH
    4. Dolan RJ
    5. Behrens TE

    (2013) Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex

    Journal of Neuroscience 33:3202–3211.

    https://doi.org/10.1523/JNEUROSCI.2532-12.2013

    • Google Scholar
17. 1. Lebreton M
    2. Abitbol R
    3. Daunizeau J
    4. Pessiglione M

    (2015) Automatic integration of confidence in the brain valuation signal

    Nature Neuroscience 18:1159–1167.

    https://doi.org/10.1038/nn.4064

    • Google Scholar
18. 1. Levy D
    2. Glimcher P

    (2011) Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain

    Journal of Neuroscience 31:14693–14707.

    https://doi.org/10.1523/JNEUROSCI.2218-11.2011

    • Google Scholar
19. 1. Levy DJ
    2. Glimcher PW

    (2012) The root of all value: a neural common currency for choice

    Current Opinion in Neurobiology 22:1027–1038.

    https://doi.org/10.1016/j.conb.2012.06.001

    • Google Scholar
20. 1. Logothetis NK
    2. Pauls J
    3. Augath M
    4. Trinath T
    5. Oeltermann A

    (2001) Neurophysiological investigation of the basis of the fMRI signal

    Nature 412:150–157.

    https://doi.org/10.1038/35084005

    • Google Scholar
21. 1. Logothetis NK
    2. Pfeuffer J

    (2004) On the nature of the BOLD fMRI contrast mechanism

    Magnetic Resonance Imaging 22:1517–1531.

    https://doi.org/10.1016/j.mri.2004.10.018

    • Google Scholar
22. 1. Logothetis NK
    2. Wandell BA

    (2004) Interpreting the BOLD signal

    Annual Review of Physiology 66:735–769.

    https://doi.org/10.1146/annurev.physiol.66.082602.092845

    • Google Scholar
23. 1. Logothetis NK

    (2008) What we can do and what we cannot do with fMRI

    Nature 453:869–878.

    https://doi.org/10.1038/nature06976

    • Google Scholar
24. 1. Luk C-H
    2. Wallis JD

    (2013) Choice coding in frontal cortex during stimulus-guided or action-guided decision-making

    Journal of Neuroscience 33:1864–1871.

    https://doi.org/10.1523/JNEUROSCI.4920-12.2013

    • Google Scholar
25. 1. McCormick DA
    2. Connors BW
    3. Lighthall JW
    4. Prince DA

    (1985)

    Comparative electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex

    Journal of Neurophysiology 54:782–806.

    • Google Scholar
26. 1. McDannald M
    2. Lucantonio F
    3. Burke K
    4. Niv Y
    5. Schoenbaum G

    (2011) Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning

    Journal of Neuroscience 31:2700–2705.

    https://doi.org/10.1523/JNEUROSCI.5499-10.2011

    • Google Scholar
27. 1. McDannald MA
    2. Saddoris MP
    3. Gallagher M
    4. Holland PC

    (2005) Lesions of orbitofrontal cortex impair rats' differential outcome expectancy learning but not conditioned stimulus-potentiated feeding

    Journal of Neuroscience 25:4626–4632.

    https://doi.org/10.1523/JNEUROSCI.5301-04.2005

    • Google Scholar
28. 1. McDannald MA
    2. Esber GR
    3. Wegener MA
    4. Wied HM
    5. Liu TL
    6. Stalnaker TA
    7. Jones JL
    8. Trageser J
    9. Schoenbaum G

    (2014) Orbitofrontal neurons acquire responses to 'valueless' pavlovian cues during unblocking

    eLife 3:e02653.

    https://doi.org/10.7554/eLife.02653

    • Google Scholar
29. 1. McNamee D
    2. Rangel A
    3. O'Doherty JP

    (2013) Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex

    Nature Neuroscience 16:479–485.

    https://doi.org/10.1038/nn.3337

    • Google Scholar
30. 1. Noonan MP
    2. Walton ME
    3. Behrens TEJ
    4. Sallet J
    5. Buckley MJ
    6. Rushworth MFS

    (2010) Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex

    Proceedings of the National Academy of Sciences of the United States of America 107:20547–20552.

    https://doi.org/10.1073/pnas.1012246107

    • Google Scholar
31. 1. Noonan MP
    2. Kolling N
    3. Walton ME
    4. Rushworth MF

    (2012) Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement

    The European Journal of Neuroscience 35:997–1010.

    https://doi.org/10.1111/j.1460-9568.2012.08023.x

    • Google Scholar
32. 1. Ongur D

    (2000) The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans

    Cerebral Cortex 10:206–219.

    https://doi.org/10.1093/cercor/10.3.206

    • Google Scholar
33. 1. Ostlund S
    2. Balleine B

    (2007) Orbitofrontal cortex mediates outcome encoding in pavlovian but not instrumental conditioning

    Journal of Neuroscience 27:4819–4825.

    https://doi.org/10.1523/JNEUROSCI.5443-06.2007

    • Google Scholar
34. 1. Padoa-Schioppa C
    2. Assad JA

    (2006) Neurons in the orbitofrontal cortex encode economic value

    Nature 441:223–226.

    https://doi.org/10.1038/nature04676

    • Google Scholar
35. 1. Padoa-Schioppa C
    2. Assad JA

    (2008) The representation of economic value in the orbitofrontal cortex is invariant for changes of menu

    Nature Neuroscience 11:95–102.

    https://doi.org/10.1038/nn2020

    • Google Scholar
36. 1. Padoa-Schioppa C
    2. Schoenbaum G

    (2015) Dialogue on economic choice, learning theory, and neuronal representations

    Current Opinion in Behavioral Sciences 5:16–23.

    https://doi.org/10.1016/j.cobeha.2015.06.004

    • Google Scholar
37. 1. Padoa-Schioppa C

    (2009) Range-adapting representation of economic value in the orbitofrontal cortex

    Journal of Neuroscience 29:14004–14014.

    https://doi.org/10.1523/JNEUROSCI.3751-09.2009

    • Google Scholar
38. 1. Padoa-Schioppa C

    (2011) Neurobiology of economic choice: a good-based model

    Annual Review of Neuroscience 34:333–359.

    https://doi.org/10.1146/annurev-neuro-061010-113648

    • Google Scholar
39. 1. Padoa-Schioppa C

    (2013) Neuronal origins of choice variability in economic decisions

    Neuron 80:1322–1336.

    https://doi.org/10.1016/j.neuron.2013.09.013

    • Google Scholar
40. 1. Pickens CL
    2. Saddoris MP
    3. Setlow B
    4. Gallagher M
    5. Holland PC
    6. Schoenbaum G

    (2003)

    Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task

    Journal of Neuroscience 23:11078–11084.

    • Google Scholar
41. 1. Plassmann H
    2. O'Doherty J
    3. Rangel A

    (2007) Orbitofrontal cortex encodes willingness to pay in everyday economic transactions

    Journal of Neuroscience 27:9984–9988.

    https://doi.org/10.1523/JNEUROSCI.2131-07.2007

    • Google Scholar
42. 1. Plassmann H
    2. O'Doherty JP
    3. Rangel A

    (2010) Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making

    Journal of Neuroscience 30:10799–10808.

    https://doi.org/10.1523/JNEUROSCI.0788-10.2010

    • Google Scholar
43. 1. Price JL

    (2007) Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions

    Annals of the New York Academy of Sciences 1121:54–71.

    https://doi.org/10.1196/annals.1401.008

    • Google Scholar
44. 1. Rescorla R

    (1999) Learning about qualitatively different outcomes during a blocking procedure

    Animal Learning & Behavior 27:140–151.

    https://doi.org/10.3758/BF03199671

    • Google Scholar
45. 1. Rudebeck PH
    2. Murray EA

    (2014) The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes

    Neuron 84:1143–1156.

    https://doi.org/10.1016/j.neuron.2014.10.049

    • Google Scholar
46. 1. Rudebeck PH
    2. Saunders RC
    3. Prescott AT
    4. Chau LS
    5. Murray EA

    (2013) Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating

    Nature Neuroscience 16:1140–1145.

    https://doi.org/10.1038/nn.3440

    • Google Scholar
47. 1. Schoenbaum G
    2. Chiba AA
    3. Gallagher M

    (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning

    Nature Neuroscience 1:155–159.

    https://doi.org/10.1038/407

    • Google Scholar
48. 1. Stalnaker TA
    2. Cooch NK
    3. Schoenbaum G

    (2015) What the orbitofrontal cortex does not do

    Nature Neuroscience 18:620–627.

    https://doi.org/10.1038/nn.3982

    • Google Scholar
49. 1. Steiner AP
    2. Redish AD

    (2012) The road not taken: neural correlates of decision making in orbitofrontal cortex

    Frontiers in Neuroscience 6:131.

    https://doi.org/10.3389/fnins.2012.00131

    • Google Scholar
50. 1. Strait CE
    2. Blanchard TC
    3. Hayden BY

    (2014) Reward value comparison via mutual inhibition in ventromedial prefrontal cortex

    Neuron 82:1357–1366.

    https://doi.org/10.1016/j.neuron.2014.04.032

    • Google Scholar
51. 1. Takahashi YK
    2. Roesch MR
    3. Stalnaker TA
    4. Haney RZ
    5. Calu DJ
    6. Taylor AR
    7. Burke KA
    8. Schoenbaum G

    (2009) The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

    Neuron 62:269–280.

    https://doi.org/10.1016/j.neuron.2009.03.005

    • Google Scholar
52. 1. Tremblay L
    2. Schultz W

    (1999) Relative reward preference in primate orbitofrontal cortex

    Nature 398:704–708.

    https://doi.org/10.1038/19525

    • Google Scholar
53. 1. Wallis J

    (2012) Cross-species studies of orbitofrontal cortex and value-based decision-making

    Nature Neuroscience 15:13–19.

    https://doi.org/10.1038/nn.2956

    • Google Scholar
54. 1. Walton ME
    2. Behrens TE
    3. Buckley MJ
    4. Rudebeck PH
    5. Rushworth MF

    (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning

    Neuron 65:927–939.

    https://doi.org/10.1016/j.neuron.2010.02.027

    • Google Scholar
55. 1. West EA
    2. DesJardin JT
    3. Gale K
    4. Malkova L

    (2011) Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques

    Journal of Neuroscience 31:15128–15135.

    https://doi.org/10.1523/JNEUROSCI.3295-11.2011

    • Google Scholar

Author details

1. Nina Lopatina

   1. Intramural Research Program, National Institute on Drug Abuse, Baltimore, United States
   2. Program in Neuroscience, University of Maryland School of Medicine, Baltimore, United States

   Contribution

   NL, conceived and designed the experiment; acquired the data; analyzed and interpreted the data; drafted the manuscript; approved the submitted manuscript, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

2. Michael A McDannald

   Department of Psychology, Boston College, Boston, United States

   Contribution

   MAMCD, conceived and designed the experiment; acquired the data; analyzed and interpreted the data; critically revised the manuscript; approved the submitted manuscript, Conception and design, Analysis and interpretation of data

   Competing interests

   The authors declare that no competing interests exist.

3. Clay V Styer

   Intramural Research Program, National Institute on Drug Abuse, Baltimore, United States

   Contribution

   CVS, acquired the data; critically revised the manuscript; approved the submitted manuscript, Acquisition of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

4. Brian F Sadacca

   Intramural Research Program, National Institute on Drug Abuse, Baltimore, United States

   Contribution

   BFS, acquired the data; analyzed and interpreted the data; critically revised the manuscript; approved the submitted manuscript, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

5. Joseph F Cheer

   Department Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, United States

   Contribution

   JFC, critically revised the manuscript; approved the submitted manuscript, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

6. Geoffrey Schoenbaum

   1. Intramural Research Program, National Institute on Drug Abuse, Baltimore, United States
   2. Department of Neuroscience, Johns Hopkins University, Baltimore, United States
   3. Department Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, United States

   Contribution

   GS, conceived and designed the experiment; analyzed and interpreted the data; drafted the manuscript; approved the submitted manuscript, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   For correspondence

   geoffrey.schoenbaum@nih.gov

   Competing interests

   The authors declare that no competing interests exist.

• 1,207

  views

• 262

  downloads

• 34

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.