Using knowledge of the structure of the world to infer value is at the heart of model-based reasoning, and relies on a circuit that includes the orbitofrontal cortex (OFC) (Stalnaker et al., 2015; Rudebeck and Murray, 2014; Wallis, 2011). When OFC is intact, rats and primates can use the causal structure of their environment to infer the value of elements on-the-fly. With OFC inactivated or lesioned, they cannot. This is evident in a variety of situations (Gallagher et al., 1999; Izquierdo et al., 2004; Reber et al., 2017; Gremel and Costa, 2013; West et al., 2011; Takahashi et al., 2009; McDannald et al., 2005; Walton et al., 2010), however it is perhaps most striking during sensory preconditioning. Here, inactivation of the OFC entirely and selectively impairs the use of previously acquired stimulus-stimulus associations to guide responding when one of the cues later comes to predict food (Jones et al., 2012).

How might the OFC support such inference? Some proposals focus on the ability of OFC neurons to respond to cues based on their acquired biological significance or value (Padoa-Schioppa and Assad, 2006; Padoa-Schioppa, 2011; Rolls, 1996; Levy and Glimcher, 2012; Rolls et al., 1996; Rolls and Grabenhorst, 2008; Kringelbach, 2005). The loss of such signaling is proposed to affect value-guided behavior. However, inactivation or lesions of OFC typically only affect value-guided behavior that requires inference or model-based processing (Schoenbaum et al., 2011). If the value can be derived from direct experience, the OFC is not normally necessary. This raises the possibility that the OFC is required for representing the model and perhaps not, uniquely, for encoding value (Wilson et al., 2014; Schuck et al., 2016). A clear distinction between these two accounts comes when there are associations to be learned among neutral or valueless cues. If the core function of the OFC is to represent associative information that has biological significance or value, then this area should not represent such neutral associations until they have acquired some significance. On the other hand, if the core function of the OFC is to represent the causal structure of the world, then one might expect to see these relationships represented in some manner, even before they have any significance.

Here we directly tested these predictions by recording OFC neurons in rats during sensory preconditioning (Brogden, 1939). In this task, hungry rats are initially exposed to pairs of neutral cues (A->B, C->D). In subsequent conditioning sessions, the second cue in each pair is presented, one of which predicts a food reward (B->US, D). Finally responding to the first cue in each pair is assessed in an unrewarded probe test (A, C). As noted above, inactivation of the OFC in the probe test abolishes the normal increase in responding to A without affecting responding to B (Jones et al., 2012). If this is because of a role for the OFC in representing value, either independent of or combined with associative structure, then neural activity will reflect the significance of A and its relationship to subsequent events only in the probe test. By contrast, if this is because of a role for OFC in representing associative structure, independent of value, then neural activity in the OFC should reflect the relationship of A (and C) to subsequent events in both the probe test and the initial preconditioning phase.

We trained 21 rats with recording electrodes implanted in the OFC in a sensory-preconditioning task similar to the one used in our prior study (Jones et al., 2012). In the initial phase, rats learned to associate two pairs of 10 s auditory cues (A->B; C->D) in the absence of reward. As there was no reward, rats showed no significant responding at the food cup and no differences among the different cues (one-way ANOVA, F(3, 80)=0.54, p=0.66; Figure 1A). In the second phase, rats learned that one of the auditory cues (B) predicted reward and the other (D) did not. Learning during conditioning was reflected in an increase in responding at the food cup during presentation of B, but not D (two-way ANOVA, main effect of cue: F(1, 246)=46.95, p<0.001, main effect of session: F(5, 246)=11.75, p<0.001 interaction: F(5, 246)=3.49 p=0.0046; Figure 1B). In the final phase of the task, the rats were again presented with the four auditory cues, beginning with reminder trials of cue B and D followed by unrewarded presentations of cues A and C. As expected, the rats responded at the food cup significantly more to cue B than D (Figure 1C, left panel; t-test_BD: t(20) = 8.23) and more during presentation of A, the cue that predicted B, than during presentation of C, the cue that predicted D (Figure 1C, central panel; ANOVA, main effect of cue: F(1, 251)=5.79, df = 1, p=0.017; t-test_AC: t(20) = 2.15, df = 1, p=0.044).

Figure 1

Download asset Open asset

Orbitofrontal neurons acquire ability to distinguish cue pairs during preconditioning

We recorded 266 neurons from OFC during the two preconditioning days (an average of 6 neurons per subject per day). Of these, 42% (112/266) significantly increased firing to at least one of the cues during preconditioning (right-tailed rank-sum between baseline and cue response, p<0.05), while 15% significantly decreased firing (40/266; left-tailed rank-sum, p<0.05). Overall, the prevalence of modulated firing to each of the individual cues was roughly equivalent (excited: 20% A, 18% B, 20% C, 13% D; inhibited: 7% A, 7% B, 4% C, 2% D).

This population included some neurons responding to one or both cue pairs, and such correlates were over-represented in the population of neurons responding to at least one of the cues, with elevated firing to both cues of a pair (A and B or C and D, 45/112) more common than elevated firing to cues of different pairs (A and D or B and C, 23/112; chi-squared test for independence, X² = 10.2; p=0.0014). This pattern is evident in Figure 2A, which plots the average (AUC) normalized responding of each of the 266 neurons to each preconditioned pair, ordered by how distinctly neurons responded to the initial cue in each preconditioned pair. This plot shows that those neurons that respond to one cue of a pair (e.g., cue A) have a strong tendency to respond to the other cue of a pair (e.g. B), confirming the pattern seen in individual neurons (Figure 2B). If this pattern was merely the result of neurons having a general sensitivity to auditory cues, we would expect the neurons that fired to one cue pair to also fire to the other cue pair. However, the strength of response to one cue pair (e.g., A and B) tended to not be strongly predictive of a response to the other cue pair (e.g., C and D). To test whether this pattern was statistically reliable, we examined the relationship between the mean spiking above baseline to each cue between the paired cues and between the cues that were not paired for all 266 neurons recorded in both days. As illustrated in Figure 2C, we found that OFC neurons were much more likely to have a similar response to paired cues (AB or CD) than to unpaired cues (CB, AD). This was true across all neurons (n = 266 rho_AB=0.74 and rho_CB = 0.16, Zr1-r2 = 9.05, p<10⁻¹⁶; rho_CD = 0.75, rho_AD = 0.23, Zr1-r2 = 8.59, p<10⁻¹⁶). Thus, OFC neurons tended to respond similarly to the paired auditory cues and distinctly to each of the pairs.

Figure 2 with 1 supplement see all

Download asset Open asset

We next tested if the correlated firing during the contiguous cues was merely the result of their temporal adjacency. If this is the cause, then nearby bins should be more correlated than temporally distant bins. The supplement to Figure 2 tests this, comparing the mean correlation between activity in bins early (first half) and late (last half) in one cue of a pair to activity in the other cue of the pair. While there is an overall lower correlation (owing to more bin-to-bin variation in firing rates of individual neurons), the influence of timing on correlation is, at best, surprisingly modest, and formally there is no significant difference between the strength of these correlations calculated with the early versus the late bins for either set of cues on either day. These results suggest that mere temporal contiguity of the time bins does not account for the correlated firing observed in OFC during the cues in preconditioning.

To say that this correlation is a measure of the association of the cues, however, something about this correlation should grow or change across preconditioning. To assess this, we examined how these correlations evolved during learning in neurons from rats that demonstrated they learned the relevant sensory association by responding more to cue A than to cue C in the final probe test (n = 203 from 14/21 rats). The outcome of this analysis is displayed in Figure 3A. As expected, there was a strong positive relationship between firing to the paired cues (AB and CD), and no relationship between firing to the unpaired cues (AD and CB). Furthermore, the pattern of this correlation differed across days: on day 1, the correlations were strongest on the same trial for each cue of a pair, weaker for adjacent trials of that pair, and negligible between the early trials of one cue of the pair and the late trials of the other cue of the pair. This pattern of relatively restricted correlation is consistent with the contiguity explanation – correlations do not reflect a consistent representation of the pair but are merely caused by a subset of neurons that happen to be activated by adjacent sounds at a particular time. However on day 2, following a full day of preconditioning and time to consolidate associations, the correlations between cues of a pair encompass most of the 6 trials of the opposite pair of each cue, forming more of a checkerboard pattern, as if a reliable response is evoked to each cue of a pair. The across-trial reliability of the evoked response is consistent with identification of the cue pairs as a reliable feature of the environment in these rats.

Figure 3

Download asset Open asset

If OFC responses to paired, innocuous cues become more reliably similar, we should be able to identify OFC’s response to one pair of cues on a given trial better on the second preconditioning day than on the first, when the correlation among trials is less consistent. For example, Figure 3B displays the relationship in firing within the neurons recorded in a single session for presentations of each cue, plotted as the first two principal components of the population response on each of the two preconditioning days. On day one the ability to classify trials as B (black grid background) or D (grey grid background) does not discriminate the paired cues (A and C) very well, whereas the ability to classify B and D on day two is nearly perfect at telling their paired partners apart.

To test this quantitatively, we generated pseudo-ensembles for each preconditioning day. We modeled the population response with a simple linear discriminant classifier trained on all but one response to each of the cues and then tested the ability of this model to classify the held-out presentation of each cue. The held-out trials (one each of A, B, C, and D) could then be labeled as having come from any one of the cues. To establish the reliability of this classification, this analysis was repeated on 6 sets of cue presentations, and on resampled ensembles (with replacement) of size equal to the population recorded that day from rats that learned the task (89 neurons for day 1 and 114 neurons for day 2) one thousand times. Figure 3C illustrates the average output of this classifier as a confusion matrix, with ‘correct’ classification (responses to a cue labeled as that cue) on the main diagonal, and different kinds of mis-classification along the other diagonals, with trials sometimes categorized as a ‘within-pair’ error (e.g., labeling an A trial as coming from cue B), or a ‘between-pair’ error (e.g., labeling an A trial as coming from cue C or D). While between pair errors were relatively rare, it appears that on average there is a substantial increase in within-pair errors from day 1 to day 2. When the output of these classifiers are aggregated by response (correct, or within and between pair errors), displayed in Figure 3D, the population response showed a decline in self-classification and an increase in within-pair classification across the two preconditioning days. This shift in the distribution of errors in classification is consistent with the expectation that if cues of a pair are being represented more similarly across trials, there should be an increase in within-pair misclassification. To test whether a shift this large could have occurred by chance, we performed a permutation test where the distribution of the shift in between-type errors from day 1 to 2 was computed across all resampled ensembles. According to this approach, which allows the direct calculation of a p-value for the specific difference that was observed, the shift in within-pair classification across days was unlikely to occur by chance (p=0.009, Figure 3E, top panel). A similar permutation test on the difference between the within pair and between pair classification on day two found that this difference was also unlikely to occur by chance (p=0.0001, Figure 3D, top right panel).

Finally to control for baseline differences between trials, as some neurons distinguish AB trial blocks from CD trial blocks, we repeated this classification analysis, either by simply by subtracting baseline firing on individual trials from the cue responses on that trial as a first control dataset or by fitting a regression model to the relationship between cue firing on a given trial and firing at baseline on that trial and using the residuals from that regression a second control dataset and classifying both control datasets as above. In both, we again observed an increase in within-pair classification from day 1 to day 2 (p_subtraction = 0.001; p_residual = 0.007) and a greater within-pair than between pair classification on day 2 (p_subtraction = 0.011; p_residual = 0.038).

Orbitofrontal neurons acquire the ability to predict reward during pavlovian conditioning

As noted earlier, one hallmark of OFC neurons is they acquire responses to cues that have biological significance or value through pairing with reward. Accordingly, we found that activity to B increased significantly in the 683 neurons recorded over the course of 6 days of conditioning. The evolution of this increase can be seen in the average (AUC) normalized responding of these neurons to cues B and D shown in Figure 4A and B. Firing to cues B and D is initially very similar, however over the 6 days of training, cue B comes to evoke a larger neural response than cue D. Although firing to B is contaminated by the delivery of reward at several points within the cue, the increased firing is also evident in many neurons at the outset of cue B. On the final conditioning day, twice as many neurons fired above baseline in the first 2 s of cue B, before reward onset, than did so at the outset of cue D (17%, 17/101 vs 7%, 7/101; X² = 4.73, p=0.03). In addition, the prevalence of such neurons increased significantly over the course of conditioning for rewarded cue B (17% or 17/101 on day 6 vs 8% or 10/128 on day 1; X² = 4.41, p=0.036) vs cue D (7% or 7/101 on day 6 vs 6% or 8/128 on day 1; X² = 0.04, p=0.84). This increase is similar to what we have observed previously in similar settings (Takahashi et al., 2013; Lucantonio et al., 2014).

Figure 4

Download asset Open asset

Orbitofrontal neurons exhibit ability to infer reward in the probe test

Given the increase in the fraction of neurons firing to B across conditioning, we wondered whether the pattern of neural activity to the other cues paired with them in preconditioning might also change. This would be consistent with a role for OFC in dynamically representing the current cognitive map (rather than some prior, static one). To examine this, we plotted the activity of the 205 neurons (averaging 9.8 neurons per subject) recorded in the probe session. Recall that during the probe test in the current experiment, we presented cues B and D in a reminder phase with reward given, and then followed this with unrewarded presentations of the paired cues, A and C. Consistent with the conditioning data, a larger fraction of neurons again exhibited increased activity to the rewarded cue B than cue D (31% vs 8%; one-way sign-test baseline vs. cue, Figure 5A). However, in addition, the fraction of neurons responding above baseline to the preconditioned cues (A and C) also increased significantly (Figure 5A). Notably, although the firing to each remained largely segregated, the increase was seen to both cues, with 37% of neurons elevating their firing rate to cue A and 35% of neurons elevating their firing rate to cue C (across first 3 trials of each for comparison with B/D fractions, one-way sign-test, baseline vs. cue, p<0.05), with roughly the same fraction inhibited as in preconditioning (6% for cue A and 7% for cue C). While some of this increase may reflect generalization, the reorganization favored the promotion of firing correlates that reflected the earlier learning. This is evident in Figure 5B and C, which plot the mean normalized response of the ten percent of neurons with the largest difference in responding to cue A over C (Figure 5B) or vice versa (Figure 5C). In neurons with the stronger response to A, there is a strong and prolonged response to cue B (and reward), whereas in neurons with the stronger response to C, there was only a modest response to cue B, and this response is primarily observed only after reward delivery begins. These distinctions hold for both more selective and permissive comparisons of A vs. C responding.

Figure 5

Download asset Open asset

The increase in the fraction of neurons responding to cues A and C, which had not been presented since preconditioning, coupled with the preserved relationship between firing to cues A and B, shows that the activity of OFC neurons integrates associations formed in preconditioning and conditioning in the probe test. As noted earlier, conditioned responding in this phase to cue A is OFC-dependent (Jones et al., 2012). To test whether the neural reorganization might be related to this dependence, we divided the recording data based on whether the rats showed evidence of preconditioning in the probe test. Figure 6A displays the relative activity between cues for the 150 neurons recorded in rats that responded more to cue A than to cue C. These neurons showed stronger correlated firing between formerly paired cues than between cues that had never been paired (n = 150, rho_AB = 0.43 and rho_CB = 0.19, Z_r1-r2= 2.27, p=0.023; rho_CD = 0.37, rho_AD = 0.12, Z_r1-r2 = 2.36, p=0.018). By contrast, Figure 6B displays the mean activity of 55 neurons recorded in rats that showed either no preference in responding to cues A and C or responded more to cue C than cue A. These neurons showed correlated firing between the unpaired cues that was as strong or stronger than that between the formerly paired cues (n = 55, rho_AB = 0.45 and rho_CB = 0.59, Z_r1-r2 = 0.90, p=0.36; rho_CD = 0.12, rho_AD = 0.14, Z_r1-r2 = 0.13, p=0.89).

Figure 6

Download asset Open asset

To the confirm the robustness of the distinct patterns of correlations across trials and through time, we created another simple linear discriminant classifier, using pseudo-ensembles of 205 neurons, equal to the population recorded for that day, and trained using the mean activity evoked by the cues on A and C trials. We then asked this A/C classifier to identify activity during presentation of B or D to test whether firing to the preconditioned cues was, in essence, representing the subsequent cue in each pair. Because B had two phases, one before and one after the delivery of reward began, we conducted this analysis on segments of the trial, a 1 s window moved in 250 ms steps and iterated 1000x on resampled ensembles. The mean classification success was then compared to a null distribution created from the same classifier, with shuffled cue labels; classification better than 95% of the shuffled examples was labeled as significant (p>0.05). The result, plotted separately for the neurons recorded in good (Figure 6C) and poor (Figure 6D) performers, shows that above-chance classification (e.g. B = A and D = C) was only observed in ensembles composed of neurons from good performers. Further, the significant increase in correct classification came during the period when cue B overlapped with reward and was consistent through this period. This indicates not only that the ensembles reorganized in the good performers as a result of conditioning, but that they reorganized such that activity during A was best correlated with the middle and later sections of B, when reward could be expected to come. This is consistent with the idea that activity during A is directly signaling B and is association with reward, even though A was never presented with reward.

The OFC has long been implicated in our ability to respond adaptively and flexibly to obtain reward (Gallagher et al., 1999; Izquierdo et al., 2004; Reber et al., 2017; Gremel and Costa, 2013; West et al., 2011; Takahashi et al., 2009; McDannald et al., 2005; Walton et al., 2010; Jones et al., 2012). Traditionally this involvement has been linked to representing associative information of biological significance (Rolls, 1996; Rolls et al., 1996; Rolls and Grabenhorst, 2008; Kringelbach, 2005). More recently, research has emphasized the importance of the OFC to encoding the value or utility of available options, allowing decisions between them that reflect meaningful or idiosyncratic real-time changes in their desirability (Padoa-Schioppa and Assad, 2006; Padoa-Schioppa, 2011; Levy and Glimcher, 2011; Plassmann et al., 2007; Padoa-Schioppa, 2009; Padoa-Schioppa, 2013; Tremblay and Schultz, 1999; Kobayashi et al., 2010; O'Neill and Schultz, 2010). Together, these ideas have promoted the core function of the OFC as transforming information into an expectation of value (Padoa-Schioppa, 2011; Levy and Glimcher, 2012). However, an alternative view is that the OFC’s core function is to represent a structure among environmental features, of which value is merely one of many features (Stalnaker et al., 2015; Wilson et al., 2014; Schuck et al., 2016; Wikenheiser et al., 2017). Here we tested between these different perspectives by examining the representation of associative information in OFC neurons and ensembles both before and after those associations had acquired biological significance. To do this, we recorded single unit activity in OFC during an OFC-dependent sensory preconditioning task (Jones et al., 2012). Activity was recorded during the initial preconditioning phase, while rats were exposed to neutral cue pairs, and subsequently during the probe test, when the same cues were presented after one had been paired with reward. As expected, we found that associative neural activity in the OFC was heavily driven by reward; the cue that had been paired with reward was strongly represented by the population. In addition, probe test firing to cues paired in preconditioning was strongly correlated, particularly in rats that showed evidence of preconditioning. However, while the OFC’s response to these cues was robust once they were tied to an expectation of value, the response represented a modification of neural correlates of the arbitrary cue pairs evident and in fact acquired during the initial phase of training.

That OFC acquires neural representations of the arbitrary cue pairs in the initial phase of preconditioning, prior to the introduction of reward, suggests that the OFC builds associative representations even for information that does not have clear biological significance or value. While the implicit learning of statistical relationships between visual (Turk-Browne et al., 2009) or auditory cues (McNealy et al., 2006) has been reported in sensory cortices, it’s striking that more frontal regions like OFC have access to these associations. In this regard, the OFC joins a growing number of associative regions, including hippocampal, retrosplenial, striatal, and even midbrain areas (Cerri et al., 2014; Robinson et al., 2014; Sharpe et al., 2017a; Wimmer and Shohamy, 2012), that appear to be involved in and even required for stimulus-stimulus learning.

But what is the actual role of these representations - if OFC is not simply signaling value, what does it signal? One possibility suggested by recent computational accounts is that correlates like these reflect a role in maintaining so called successor representations. These representations capture the expectation of moving to one state from another, independent of value, but stop short of encoding a full task model (Gershman et al., 2012). Successor representations have been applied to interpret neural activity in hippocampus (Stachenfeld et al., 2017), and aspects of these models would account for the apparent associative activity observed to the predictive cues (A and C) in preconditioning. While appealing, if OFC represents the matrix of future expected states, it is not clear why this activity changes as a result of conditioning to B. In simple versions of this model, an established matrix is not affected except by direct experience; A and C were not experienced again until the probe test, and yet the pattern of activity to cues A and C changed from preconditioning to probe. Alternatively, activity in OFC to A and C could reflect the product of their successor representation matrices and the value of the downstream states. This would explain the dramatic change in neural activity to A across conditioning, since the value of B was presumably altered by pairing with reward. However, responding to A does not seem to be fundamentally based on value cached in B, since that responding is affected by spontaneous changes in the value of the actual food (Sharpe et al., 2017a). Further, recent evidence shows that cue A in our design will not serve as a conditioned reinforcer, whereas a second-order cue will do so (Sharpe et al., 2017b). These data provide direct evidence that a preconditioned cue, at least in our design, is not accessing cached value by any common definition. While these disparate findings can perhaps be reconciled with successor representations models that incorporate off-line rehearsal or other additional processing steps, the activity we observe here seems more consistent with the proposal that the OFC encodes a fuller cognitive ‘state’ map (Stalnaker et al., 2015; Wilson et al., 2014; Lopatina et al., 2017).

Finally, it is worth noting that the current results are consistent with data showing that the OFC is necessary for performance in the final phase of training in this task, when information must be integrated to predict the reward. Neural activity in the probe test to the preconditioned cues clearly differed between pairs, and activity in the first cue of a pair appeared to encode the second cue, particularly for the critical AC cue pair. Activity to A was most similar to activity during the rewarded portions of B, and this coding was strongest in the rats that showed strong responding to A.

However, these data do not address whether the encoding of these associations in OFC during the preconditioning phase is necessary for performance in the final phase of training. The correlates in OFC may be merely a reflection of processing in other brain regions, such as the hippocampus and retrosplenial cortex, which are necessary in these earlier phases (Robinson et al., 2014). Consistent with this idea, the OFC receives strong input from hippocampus, which has a specific influence on the encoding in OFC in real time (Wikenheiser et al., 2017). In this case, temporary inactivation of OFC during the preconditioning phase should not affect inference in the final test. By contrast, representation of this information in OFC may be necessary in the preconditioning phase, perhaps to allow proper updating or integration with the new learning. If this is the case, then inactivation should affect later responding. Regardless, the identification of sensory-sensory representations in the OFC prior to their endowment with biological significance substantially expands the potential role of this area in this very simple and other more complex settings.

1. 1. Brogden WJ

   (1939) Sensory pre-conditioning

   Journal of Experimental Psychology 25:323–332.

   https://doi.org/10.1037/h0058944

   • Google Scholar
2. 1. Cerri DH
   2. Saddoris MP
   3. Carelli RM

   (2014) Nucleus accumbens core neurons encode value-independent associations necessary for sensory preconditioning

   Behavioral Neuroscience 128:567–578.

   https://doi.org/10.1037/a0037797

   • PubMed
   • Google Scholar
3. 1. Gallagher M
   2. McMahan RW
   3. Schoenbaum G

   (1999)

   Orbitofrontal cortex and representation of incentive value in associative learning

   Journal of Neuroscience 19:6610–6614.

   • PubMed
   • Google Scholar
4. 1. Gershman SJ
   2. Moore CD
   3. Todd MT
   4. Norman KA
   5. Sederberg PB

   (2012) The successor representation and temporal context

   Neural Computation 24:1553–1568.

   https://doi.org/10.1162/NECO_a_00282

   • PubMed
   • Google Scholar
5. 1. Gremel CM
   2. Costa RM

   (2013) Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions

   Nature Communications 4:2264.

   https://doi.org/10.1038/ncomms3264

   • PubMed
   • Google Scholar
6. 1. Izquierdo A
   2. Suda RK
   3. Murray EA

   (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency

   Journal of Neuroscience 24:7540–7548.

   https://doi.org/10.1523/JNEUROSCI.1921-04.2004

   • PubMed
   • Google Scholar
7. 1. Jones JL
   2. Esber GR
   3. McDannald MA
   4. Gruber AJ
   5. Hernandez A
   6. Mirenzi A
   7. Schoenbaum G

   (2012) Orbitofrontal cortex supports behavior and learning using inferred but not cached values

   Science 338:953–956.

   https://doi.org/10.1126/science.1227489

   • PubMed
   • Google Scholar
8. 1. Kobayashi S
   2. Pinto de Carvalho O
   3. Schultz W

   (2010) Adaptation of reward sensitivity in orbitofrontal neurons

   Journal of Neuroscience 30:534–544.

   https://doi.org/10.1523/JNEUROSCI.4009-09.2010

   • PubMed
   • Google Scholar
9. 1. Kringelbach ML

   (2005) The human orbitofrontal cortex: linking reward to hedonic experience

   Nature Reviews Neuroscience 6:691–702.

   https://doi.org/10.1038/nrn1747

   • PubMed
   • Google Scholar
10. 1. Levy DJ
    2. Glimcher PW

    (2011) Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain

    Journal of Neuroscience 31:14693–14707.

    https://doi.org/10.1523/JNEUROSCI.2218-11.2011

    • PubMed
    • Google Scholar
11. 1. Levy DJ
    2. Glimcher PW

    (2012) The root of all value: a neural common currency for choice

    Current Opinion in Neurobiology 22:1027–1038.

    https://doi.org/10.1016/j.conb.2012.06.001

    • PubMed
    • Google Scholar
12. 1. Lopatina N
    2. Sadacca BF
    3. McDannald MA
    4. Styer CV
    5. Peterson JF
    6. Cheer JF
    7. Schoenbaum G

    (2017) Ensembles in medial and lateral orbitofrontal cortex construct cognitive maps emphasizing different features of the behavioral landscape

    Behavioral Neuroscience 131:201–212.

    https://doi.org/10.1037/bne0000195

    • PubMed
    • Google Scholar
13. 1. Lucantonio F
    2. Takahashi YK
    3. Hoffman AF
    4. Chang CY
    5. Bali-Chaudhary S
    6. Shaham Y
    7. Lupica CR
    8. Schoenbaum G

    (2014) Orbitofrontal activation restores insight lost after cocaine use

    Nature Neuroscience 17:1092–1099.

    https://doi.org/10.1038/nn.3763

    • PubMed
    • Google Scholar
14. 1. McDannald MA
    2. Saddoris MP
    3. Gallagher M
    4. Holland PC

    (2005) Lesions of orbitofrontal cortex impair rats' differential outcome expectancy learning but not conditioned stimulus-potentiated feeding

    Journal of Neuroscience 25:4626–4632.

    https://doi.org/10.1523/JNEUROSCI.5301-04.2005

    • PubMed
    • Google Scholar
15. 1. McNealy K
    2. Mazziotta JC
    3. Dapretto M

    (2006) Cracking the language code: neural mechanisms underlying speech parsing

    Journal of Neuroscience 26:7629–7639.

    https://doi.org/10.1523/JNEUROSCI.5501-05.2006

    • PubMed
    • Google Scholar
16. 1. O'Neill M
    2. Schultz W

    (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value

    Neuron 68:789–800.

    https://doi.org/10.1016/j.neuron.2010.09.031

    • PubMed
    • Google Scholar
17. 1. Padoa-Schioppa C
    2. Assad JA

    (2006) Neurons in the orbitofrontal cortex encode economic value

    Nature 441:223–226.

    https://doi.org/10.1038/nature04676

    • PubMed
    • Google Scholar
18. 1. Padoa-Schioppa C

    (2009) Range-adapting representation of economic value in the orbitofrontal cortex

    Journal of Neuroscience 29:14004–14014.

    https://doi.org/10.1523/JNEUROSCI.3751-09.2009

    • PubMed
    • Google Scholar
19. 1. Padoa-Schioppa C

    (2011) Neurobiology of economic choice: a good-based model

    Annual Review of Neuroscience 34:333–359.

    https://doi.org/10.1146/annurev-neuro-061010-113648

    • PubMed
    • Google Scholar
20. 1. Padoa-Schioppa C

    (2013) Neuronal origins of choice variability in economic decisions

    Neuron 80:1322–1336.

    https://doi.org/10.1016/j.neuron.2013.09.013

    • PubMed
    • Google Scholar
21. Book

    1. Paxinos G
    2. Watson C

    (2009)

    The Rat Brain in Stereotaxic Coordinates

    New York: Academic Press.

    • Google Scholar
22. 1. Plassmann H
    2. O'Doherty J
    3. Rangel A

    (2007) Orbitofrontal cortex encodes willingness to pay in everyday economic transactions

    Journal of Neuroscience 27:9984–9988.

    https://doi.org/10.1523/JNEUROSCI.2131-07.2007

    • PubMed
    • Google Scholar
23. 1. Reber J
    2. Feinstein JS
    3. O'Doherty JP
    4. Liljeholm M
    5. Adolphs R
    6. Tranel D

    (2017) Selective impairment of goal-directed decision-making following lesions to the human ventromedial prefrontal cortex

    Brain 140:1743–1756.

    https://doi.org/10.1093/brain/awx105

    • PubMed
    • Google Scholar
24. 1. Robinson S
    2. Todd TP
    3. Pasternak AR
    4. Luikart BW
    5. Skelton PD
    6. Urban DJ
    7. Bucci DJ

    (2014) Chemogenetic silencing of neurons in retrosplenial cortex disrupts sensory preconditioning

    Journal of Neuroscience 34:10982–10988.

    https://doi.org/10.1523/JNEUROSCI.1349-14.2014

    • PubMed
    • Google Scholar
25. 1. Rolls ET
    2. Critchley HD
    3. Mason R
    4. Wakeman EA

    (1996) Orbitofrontal cortex neurons: role in olfactory and visual association learning

    Journal of Neurophysiology 75:1970–1981.

    https://doi.org/10.1152/jn.1996.75.5.1970

    • PubMed
    • Google Scholar
26. 1. Rolls ET
    2. Grabenhorst F

    (2008) The orbitofrontal cortex and beyond: from affect to decision-making

    Progress in Neurobiology 86:216–244.

    https://doi.org/10.1016/j.pneurobio.2008.09.001

    • PubMed
    • Google Scholar
27. 1. Rolls ET

    (1996) The orbitofrontal cortex

    Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 351:1433–1443.

    https://doi.org/10.1098/rstb.1996.0128

    • PubMed
    • Google Scholar
28. 1. Rudebeck PH
    2. Murray EA

    (2014) The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes

    Neuron 84:1143–1156.

    https://doi.org/10.1016/j.neuron.2014.10.049

    • PubMed
    • Google Scholar
29. Software

    1. Sadacca BF

    (2018) OFC_SPC_17, version ac151e1

    Github.

    https://github.com/sadacca/OFC_SPC_17
30. 1. Schoenbaum G
    2. Takahashi Y
    3. Liu TL
    4. McDannald MA

    (2011) Does the orbitofrontal cortex signal value?

    Annals of the New York Academy of Sciences 1239:87–99.

    https://doi.org/10.1111/j.1749-6632.2011.06210.x

    • PubMed
    • Google Scholar
31. 1. Schuck NW
    2. Cai MB
    3. Wilson RC
    4. Niv Y

    (2016) Human orbitofrontal cortex represents a cognitive map of state space

    Neuron 91:1402–1412.

    https://doi.org/10.1016/j.neuron.2016.08.019

    • PubMed
    • Google Scholar
32. 1. Sharpe MJ
    2. Batchelor HM
    3. Schoenbaum G

    (2017b) Preconditioned cues have no value

    eLife 6:e28362.

    https://doi.org/10.7554/eLife.28362

    • PubMed
    • Google Scholar
33. 1. Sharpe MJ
    2. Chang CY
    3. Liu MA
    4. Batchelor HM
    5. Mueller LE
    6. Jones JL
    7. Niv Y
    8. Schoenbaum G

    (2017a) Dopamine transients are sufficient and necessary for acquisition of model-based associations

    Nature Neuroscience 20:735–742.

    https://doi.org/10.1038/nn.4538

    • PubMed
    • Google Scholar
34. 1. Stachenfeld KL
    2. Botvinick MM
    3. Gershman SJ

    (2017) The hippocampus as a predictive map

    Nature Neuroscience 20:1643–1653.

    https://doi.org/10.1038/nn.4650

    • PubMed
    • Google Scholar
35. 1. Stalnaker TA
    2. Cooch NK
    3. Schoenbaum G

    (2015) What the orbitofrontal cortex does not do

    Nature Neuroscience 18:620–627.

    https://doi.org/10.1038/nn.3982

    • PubMed
    • Google Scholar
36. 1. Takahashi YK
    2. Chang CY
    3. Lucantonio F
    4. Haney RZ
    5. Berg BA
    6. Yau HJ
    7. Bonci A
    8. Schoenbaum G

    (2013) Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning

    Neuron 80:507–518.

    https://doi.org/10.1016/j.neuron.2013.08.008

    • PubMed
    • Google Scholar
37. 1. Takahashi YK
    2. Roesch MR
    3. Stalnaker TA
    4. Haney RZ
    5. Calu DJ
    6. Taylor AR
    7. Burke KA
    8. Schoenbaum G

    (2009) The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

    Neuron 62:269–280.

    https://doi.org/10.1016/j.neuron.2009.03.005

    • PubMed
    • Google Scholar
38. 1. Tremblay L
    2. Schultz W

    (1999) Relative reward preference in primate orbitofrontal cortex

    Nature 398:704–708.

    https://doi.org/10.1038/19525

    • PubMed
    • Google Scholar
39. 1. Turk-Browne NB
    2. Scholl BJ
    3. Chun MM
    4. Johnson MK

    (2009) Neural evidence of statistical learning: efficient detection of visual regularities without awareness

    Journal of Cognitive Neuroscience 21:1934–1945.

    https://doi.org/10.1162/jocn.2009.21131

    • PubMed
    • Google Scholar
40. 1. Wallis JD

    (2011) Cross-species studies of orbitofrontal cortex and value-based decision-making

    Nature Neuroscience 15:13–19.

    https://doi.org/10.1038/nn.2956

    • PubMed
    • Google Scholar
41. 1. Walton ME
    2. Behrens TE
    3. Buckley MJ
    4. Rudebeck PH
    5. Rushworth MF

    (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning

    Neuron 65:927–939.

    https://doi.org/10.1016/j.neuron.2010.02.027

    • PubMed
    • Google Scholar
42. 1. West EA
    2. DesJardin JT
    3. Gale K
    4. Malkova L

    (2011) Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques

    Journal of Neuroscience 31:15128–15135.

    https://doi.org/10.1523/JNEUROSCI.3295-11.2011

    • PubMed
    • Google Scholar
43. 1. Wikenheiser AM
    2. Marrero-Garcia Y
    3. Schoenbaum G

    (2017) Suppression of Ventral Hippocampal Output Impairs Integrated Orbitofrontal Encoding of Task Structure

    Neuron 95:1197–1207.

    https://doi.org/10.1016/j.neuron.2017.08.003

    • PubMed
    • Google Scholar
44. 1. Wilson RC
    2. Takahashi YK
    3. Schoenbaum G
    4. Niv Y

    (2014) Orbitofrontal cortex as a cognitive map of task space

    Neuron 81:267–279.

    https://doi.org/10.1016/j.neuron.2013.11.005

    • PubMed
    • Google Scholar
45. 1. Wimmer GE
    2. Shohamy D

    (2012) Preference by association: how memory mechanisms in the hippocampus bias decisions

    Science 338:270–273.

    https://doi.org/10.1126/science.1223252

    • PubMed
    • Google Scholar

Author details

1. Brian F Sadacca

   Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States

   Contribution

   Conceptualization, Data curation, Formal analysis, Investigation, Writing—original draft, Writing—review and editing

   Competing interests

   No competing interests declared

2. Heather M Wied

   Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States

   Contribution

   Conceptualization, Investigation, Methodology

   Competing interests

   No competing interests declared

3. Nina Lopatina

   Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States

   Contribution

   Conceptualization, Investigation, Methodology

   Competing interests

   No competing interests declared

4. Gurpreet K Saini

   Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States

   Contribution

   Investigation, Methodology

   Competing interests

   No competing interests declared

5. Daniel Nemirovsky

   Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States

   Contribution

   Investigation, Methodology

   Competing interests

   No competing interests declared

6. Geoffrey Schoenbaum

   1. Intramural Research program of the National Institute on Drug Abuse, NIH, Baltimore, United States
   2. Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, United States
   3. Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, United States

   Contribution

   Conceptualization, Supervision, Funding acquisition, Writing—original draft, Project administration, Writing—review and editing

   For correspondence

   geoffrey.schoenbaum@nih.gov

   Competing interests

   Reviewing editor, eLife

   "This ORCID iD identifies the author of this article:" 0000-0001-8180-0701

• 2,914

  views

• 434

  downloads

• 78

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.