Neuronal activity in dorsal anterior cingulate cortex during economic choices under variable action costs
Abstract
The role of the dorsal anterior cingulate cortex (ACCd) in decision making has often been discussed but remains somewhat unclear. On the one hand, numerous studies implicated this area in decisions driven by effort or action cost. On the other hand, work on economic choices between goods (under fixed action costs) found that neurons in ACCd encoded only post-decision variables. To advance our understanding of the role played by this area in decision making, we trained monkeys to choose between different goods (juice types) offered in variable amounts and with different action costs. Importantly, the task design dissociated computation of the action cost from planning of any particular action. Neurons in ACCd encoded the chosen value and the binary choice outcome in several reference frames (chosen juice, chosen cost, chosen action). Thus, this area provided a rich representation of post-decision variables. In contrast to the OFC, neurons in ACCd did not represent pre-decision variables such as individual offer values in any reference frame. Hence, ongoing decisions are unlikely guided by ACCd. Conversely, neuronal activity in this area might inform subsequent actions.
Introduction
Economic choices involve the computation and comparison of subjective values, which integrate multiple attributes relevant to the decision. In particular, obtaining rewards often entails different levels of physical effort. We previously showed that economic choices under variable action costs can take place in a non-spatial representation (goods space) (Cai and Padoa-Schioppa, 2019). In the experiments, monkeys chose between two juices offered in variable amounts and associated with different action costs. The task design dissociated the computation of action costs from action planning. A group of neurons in the orbitofrontal cortex (OFC) encoded offer values integrating juice type, juice amount, and action cost. Moreover, another group of cells in OFC encoded the choice outcome before presentation of the action targets, indicating that decisions were made in goods space. However, it remained unclear whether and how other brain regions contribute to this choice process. When goods available for choice bear different action costs, the decision necessarily reflects aspects of the action. Thus, signals from the ACCd and/or other motor regions could provide an input for the computation of subjective values in OFC.
There are several reasons to believe that ACCd might be important for choices under variable action cost. First, both anatomic and functional evidence indicates that ACCd plays a role in linking actions and rewards (Bush et al., 2000; Calderazzo et al., 2021; Hadland et al., 2003; Hayden and Platt, 2010; Paus, 2001). Second, ACCd has been implicated in various types of decision making, including strategic decision making, social decision making, and cost-benefit tradeoff (Klein-Flügge et al., 2016; Lee et al., 2007; Rangel and Hare, 2010; Rushworth et al., 2007; Wallis and Kennerley, 2010). More importantly, previous studies found that neuronal activity in this region integrates expected reward and action cost (Kennerley et al., 2009; Kennerley and Wallis, 2009) and reflects effort-discounted subjective values in an action-based reference frame (Hosokawa et al., 2013). In rodent studies, lesion of the medial frontal cortex (MFC, roughly homologous to the primate ACCd) disrupts reward and effort tradeoff (Rudebeck et al., 2006; Walton et al., 2002). Furthermore, a recent study showed that chemogenetic modulation of the rodent ACCd altered the animal’s effort-exerting behavior in conditions that require a choice, but not when choices were not required (Hart et al., 2020).
In a previous study, we examined the neuronal representations in ACCd in an economic choice task with equal action cost (Cai and Padoa-Schioppa, 2012). We discovered that ACCd only encodes post-decision variables, namely, chosen value, chosen juice, and movement direction. However, based on its engagement in effort-based decisions, ACCd may contribute to the computation of offer values when decisions require the integration of action costs. To test this possibility, in the current study, we introduced action cost as an additional value attribute. The action cost was operationalized by requiring the animal to make saccades of different amplitudes. Critically, the action cost was indicated to the animal before and independent of the instruction associated with any particular action. We report the following results. First, action costs taxing the oculomotor system induced appreciable behavioral effects as the animals ‘choices’ were biased in favor of low-cost offers. Second, contrary to reasonable predictions, neurons in ACCd did not encode offer values in any reference frame. Conversely, neurons in ACCd encoded different variables associated with the choice outcome. Third, in contrast to OFC, where neurons encoded choice outcome in juice-based and cost-based reference frames with approximately equal strength, the encoding of choice outcomes in ACCd was heavily skewed in favor of a cost-based reference frame. Taken together, these results extend previous observations on good-based choices and support the hypothesis that ACCd plays a role in guiding motor systems based on reward and effort associated with the choice outcomes.
Results
Experimental design and choice patterns
We designed a task in which animals chose between two juices offered in variable amounts (Figure 1A). We dissociated the spatial location of the offers from the saccades necessary to obtain them and we introduced a delay between the presentation of the offers and the saccade targets. Moreover, offers were associated with radial eye movements in different directions, and different saccade amplitudes imposed variable action costs. We reasoned that if the initial fixation point is straight ahead of the subject, the action cost associated with an eye movement is essentially independent of the saccade direction (isotropic) and only depends on the saccade amplitude. Each offer provided information about all the attributes of value – juice type indicated by symbol color, quantity indicated by symbol number, and action cost indicated by symbol shape, while the animal was unable to plan the saccade necessary to obtain the offer. For any given trial, one ‘offer’ was defined by a juice type, its quantity, and its action cost. An ‘offer type’ was defined by two offers (e.g., [1A+:3B−]), in which ‘−’ indicates high action cost (long saccade) and ‘+’ indicates low action cost (short saccade). A ‘trial type’ was defined by two offers and a choice (e.g., [1A+:3B−, A]). Thus, a ‘trial type’ encompasses task-related factors in goods space.
Behavioral evidence indicated that the experimental manipulation was effective in producing an action cost. Figure 1B illustrates the choice pattern recorded in one representative session. Trials were divided into two groups depending on whether juice A was offered at low cost or at high cost. The gray sigmoid is displaced to the right, indicating that the relative value of juice A was higher when juice A was offered at low cost. This effect was consistent across sessions for both monkeys. Our data set included 141 behavioral sessions from both animals. For a quantitative analysis of choice patterns, we constructed a logistic model that provided measures for the relative value of the two juices (ρ), the difference in action cost (ξ), the choice hysteresis related to the chosen juice (η) and to the chosen cost (φ), and the spatial biases related to the offer position (δ) and to the target position (ε) (see Materials and methods, Equation 1). We first examined the distribution of ξ across sessions. The difference in saccade amplitude had a significant effect on choices in both animals (Figure 1C, monkey B, median(ξ) = 0.27, p < 10–9; monkey L, median(ξ) = 0.18, p < 10–11; Wilcoxon signed-rank test).
In addition to action cost, we also examined other factors that may influence the animal’s choice. These factors include the monkey’s tendency to choose on any given trial the same juice chosen and received in the previous trial (choice hysteresis; Padoa-Schioppa, 2013). This effect was quantified by the normalized coefficient η (Equation 1, median(η) = 0.16, p < 10–19; Wilcoxon signed-rank test). We also tested whether the animal’s choices were affected by the cost associated with the offer chosen in previous trial (cost hysteresis). This effect, quantified by the normalized coefficient φ (Equation 1), was not significant across sessions (median(φ) = –0.008, p = 0.58; Wilcoxon signed-rank test). The normalized coefficient δ quantified offer-based spatial biases. Across sessions, this effect was statistically significant but rather small (median(δ) = –0.03, p = 0.04; Wilcoxon signed-rank test). Finally, the normalized coefficient ε quantified target-based spatial biases. Similar to offer-based spatial biases, this effect was statistically significant but very small in amplitude (median(δ) = 0.017, p = 0.027; Wilcoxon signed-rank test).
Encoding of choice outcome in multiple reference frames
We recorded and analyzed the activity of 688 neurons from the ACCd of two monkeys (B, 318 cells; L, 370 cells). Firing rates were analyzed in nine time windows aligned with different behavioral events (see Materials and methods). A ‘neuronal response’ was defined as the activity of one neuron in one time window as a function of the trial type. Inspection of individual responses (see Materials and methods) indicated that many neurons in ACCd were modulated by the trial type and encoded variables related to the choice outcome. More specifically, ACCd neurons appeared to encode decision variables in the frames of reference defined by the juice type and/or the action cost. Figure 2 illustrates a few examples. The response in Figure 2A, recorded in the late-delay time window, varied as a linear function of the value of chosen offer (variable chosen value). Conversely, the response in Figure 2B, recorded in the post-juice time window, was low when the animal chose juice A and high when the animal chose juice B (variable chosen juice). Meanwhile, the response in Figure 2C, recorded in the pre-go time window, was modulated by both chosen cost (low when the animal chooses the low-cost offer and high when the animal chooses the high-cost offer) and the location of chosen target (low when the chosen target is in the ipsilateral hemifield and high when the chosen target is in the contralateral hemifield).
Further inspection revealed that many neurons in the ACCd also encode the spatial location of the chosen offer or chosen target. For example, the response in Figure 3A (late-delay time window) was roughly binary – low when the chosen offer was located on the ipsilateral hemifield and high the chosen offer was on the contralateral hemifield. It did not vary significantly with the chosen value. Similarly, the response in Figure 3B (pre-go time window) was low when the chosen target was in the ipsilateral hemifield and high when the chosen target was in the contralateral hemifield, and did not vary with the chosen value.
For a population level statistical analysis, we proceeded in steps. First, we submitted each neuronal response to two 3-way analysis of variances (ANOVAs) (factors [trial type × offer A location × target A location]; factors [trial type × chosen offer location × chosen target location]; see Materials and methods). We imposed a significance threshold p < 0.001. Responses that passed this criterion for at least one factor in one of the two ANOVAs were identified as ‘task related’ and included in subsequent analyses. As detailed in Table 1, out of the total 688 neurons, many neurons were modulated by the trial type (172 cells = 25%) and/or the chosen target location (108 cells = 15.7%), while few cells were modulated by the spatial configuration of the offers (5 cells = 0.7%), the spatial configuration of the targets (25 cells = 3.6%), or the location of the chosen offer (38 cells = 5.5%). Overall, 249 cells (36.2%) were modulated by at least one factor. Subsequent analyses were restricted to this population.
In a previous study that did not dissociate the spatial location of the offer from that of the saccade target, we found that neuronal responses in ACCd often reflected the chosen side (Cai and Padoa-Schioppa, 2012). Here, we examined neuronal responses with a series of analysis of covariances (ANCOVAs), continuous variables were those defined in Table 2. As a covariate, we used the side of the chosen offer for early time windows (from post-offer to pre-target time windows), and the side of the chosen target for late time windows (from post-target to post-juice time windows; see Materials and methods).
Figure 4 illustrates the results obtained for the population. Figure 4A indicates the number of responses explained by each variable in each time window. Notably, each response could be explained by more than one variable and thus could contribute to multiple bins in this panel. Figure 4B illustrates a complementary account. Here, each response was assigned to the variable that provided the best fit. In early time windows, the dominant variables were chosen value and chosen location only. In late time windows, after target presentation and upon juice delivery, the dominant variables were chosen value, chosen target location only, chosen cost, and chosen juice. Two procedures – stepwise and best-subset – were used to identify the variables that best explained the neuronal data set (see Materials and methods). Variables were selected across all time windows. Both procedures selected variables chosen value, chosen cost, chosen juice, chosen offer location only, and chosen target location only. Figure 5 illustrates the percentage of neurons encoding each of the selected variables across different time windows.
To summarize, confirming previous results, neurons in ACCd encoded task-relevant variables associated with only the choice outcome. Remarkably, choice outcomes were encoded in all relevant reference frames – juice-, cost-, spatial-, and action-based. In contrast, we did not find any offer value signal in any of these reference frames. Moreover, few neurons in ACCd encode variables value ratio and cost/benefit conflict, which essentially capture the decision difficulty. Overall, this result demonstrates a prevailing coding of post-decision variables in ACCd.
Dimensional integration in chosen value cells
Previous studies showed that the activity of chosen value cells in the OFC depended on the juice type, juice quantity, and action costs. Here, we assessed whether chosen value cells in the ACCd also reflected the action cost. To do so, we defined two variants of the variable chosen value (juice) – one cost-affected and one cost-independent. We assessed which variant better fit neuronal responses. For each response, we considered the two R2 and we computed the difference ΔR2 = R2 cost-affected−R2cost-independent and we examined the distribution for ΔR2 across the population. We did not want to bias the results in favor of either variant. Thus, for this analysis we identified neuronal responses encoding the chosen value (juice) as follows. For each response, we considered the two R2 obtained from the two variants of chosen value variable both ‘explaining’ the response (p < 0.05), and we assigned the maximum R2 to the response. We then assigned each response to one of the selected variables accordingly. As any neuron could be tuned in multiple time windows, we aggregated the chosen value (juice) coding responses in the time windows after offer presentation but before target presentation (including post-offer, late-delay, mem-delay, and pre-target windows) and those in the time windows after target presentation (including post-target, pre-go, pre-juice, and post-juice windows) and analyzed them separately.
We discovered that in the post-offer time windows, the distribution of ΔR2 was significantly biased toward negative values (mean(ΔR2) = –0.013, p < 10–8; Figure 6). Thus, chosen value responses integrate only juice type and quantity in the early time windows before action planning is possible. In the post-target time windows, the distribution of ΔR2 was not statistically different from 0 (mean(ΔR2) = –0.0034, p = 0.25; Figure 6). Therefore, we cannot reject the null hypothesis that action cost was integrated by chosen value responses later in the trial. Moreover, the difference across early and late time windows was marginally significant (mean (ΔR2) = 0.0096, p = 0.052; Figure 6). Such pattern is consistent with our previous observation in the OFC where chosen value responses progressed from integrating only juice type and quantity to integrating all three determinants.
Contrasting neuronal encoding in OFC and ACCd
Since our previous study on the OFC used the same choice task and animals used here, we had the opportunity to contrast the neuronal representations in OFC and ACCd, to probe the differential contributions of these two areas to economic choice under variable action costs. First, we noted that neurons in the OFC and ACCd do not encode the same set of variables. In particular, pre-decision variables such as offer value (juice) and offer value (cost) are encoded in the OFC but not the ACCd (Figure 5). On the other hand, choice outcome in spatial (chosen offer location) and action-based (chosen target location) reference frames were represented in ACCd but not in OFC (Figure 5). We thus compared the strength and pattern of the variables that are commonly encoded in these two brain regions, including chosen value, chosen cost, and chosen juice.
We first examined the encoding of chosen value. We compared the percentage of chosen value coding neurons across the time windows of a trial. In the post-offer window, the percentage of neurons encoding the chosen value was strikingly similar with 13.7% and 14.3% in ACCd and OFC, respectively (Figure 7A). However, ACCd demonstrated more sustained encoding of chosen value with significantly higher percentage of neurons in late-delay (ACCd: 20.1%, OFC: 10.5%; p < 0.005, χ2-test), mem-delay (ACCd: 13.3%, OFC: 6.3%; p < 0.01, χ2-test), and pre-target (ACCd: 11.2%, OFC: 3.2%; p < 0.001, χ2-test) windows (Figure 7A).
We next compared the percentage of chosen cost and chosen juice coding neurons between the two areas. Encoding of the variable chosen cost was stronger in the OFC in post-juice window (Figure 7B, OFC: 10.9%, ACCd: 1.9%; p < 10–4, χ2-test). Moreover, encoding of the variable chosen juice was substantially higher in OFC in all time windows following target presentation including post-target (Figure 7C, ACCd: 1.6%, OFC: 9.0%; p < 10–3, χ2-test), pre-go (OFC: 7.5%, ACCd: 0.31%; p < 10–4, χ2-test), pre-juice (OFC: 21.8%, ACCd: 2.2%; p < 10–10, χ2-test), and post-juice (OFC: 19.6%, ACCd: 3.4%; p < 10–7, χ2-test) windows. However, within each brain region, encoding of the binary choice outcome signal differs. In OFC, it was skewed toward the juice-based reference frame (variable chosen juice); in ACCd, it was heavily skewed toward the cost-based reference frame (variable chosen cost).
In summary, our analyses revealed that in the cost-involved decision making, ACCd is primarily engaged in encoding the value and cost associated with the chosen option.
Discussion
In our experiments, monkeys performed economic choices between goods defined by three attributes – juice type (flavor), quantity, and action cost. Importantly, all three attributes were communicated to the animal early in the trial during offer presentation, well before the location of the eventual saccade targets were indicated. Hence animals could make their decision independently of action planning. Choices were influenced by all three attributes. In a previous study using the same choice task, the analysis of neuronal responses in central OFC (area 13 m) provided evidence that decisions were resolved in goods space. In the current study, we investigated the contribution of ACCd to this decision process. Neuronal coding in ACCd was consistent with good-based decisions. Furthermore, neurons in ACCd did not encode offer values and did not appear to encode any variable that would provide input to the ongoing decision. Conversely, neurons in ACCd encoded variables related to the choice outcome in multiple reference frames, including variables chosen value, chosen offer location only, chosen target location only, chosen cost, and chosen juice. Consistent with other reports on the prefrontal cortex (Barak et al., 2013; Fusi et al., 2016; Mashhoori et al., 2018; Rigotti et al., 2013), multiplexing of information on chosen cost and chosen target location was observed in time windows after target presentation. Importantly, chosen offer location only and chosen target location only were not significantly represented in the OFC. Thus, ACCd may function as a hub for broadcasting information associated with the choice outcome given its simultaneous representation of decision output in juice-based, cost-based, spatial-based, and action-based reference frames.
In our earlier study of anterior cingulate cortex (Cai and Padoa-Schioppa, 2012), we tested the role of this region in economic choice by recording neuronal activity from both the dorsal and ventral bank of the cingulate sulcus while monkeys performed a choice task involving the tradeoff of juice type and quantity. We observed that neurons in both regions encode only post-decision variables, namely chosen value and chosen juice, but not pre-decision variables in either juice-based or spatial reference frames. The results indicated that ACCd does not contribute to economic choice in the context of goods consisting of the attribute of juice taste and quantity. However, the outcome does not exclude a possible contribution of ACCd in other choice contexts, especially when choices are affected by action costs. One prevailing hypothesis is that when action costs are integrated into the computing of subjective value for choice, the neural locus of value computation and choice may include ACCd (Hosokawa et al., 2013). Our present results do not support this proposal. Indeed, activity in ACCd did reflect the aspect of physical effort, but not in such a form that would contribute to the encoding of subjective value.
Our results provide additional evidence disambiguating the role of ACCd in effort-based decisions. One proposal argues that in effort-based decisions, ACCd integrates multiple attributes and compute the utility of an offer to guide optimal choice. This view has gained support from both single-unit studies in non-human primates and human fMRI studies (Croxson et al., 2009; Hosokawa et al., 2013; Kennerley et al., 2009; Prévost et al., 2010). However, a non-human primate study showed that lesion of ACCd did not affect the animal’s choices, which require the integration of reward and effort (Kennerley et al., 2006). Moreover, rodent electrophysiological studies demonstrated that encoding of effort was immediately preceding the execution of effort rather than at the time of decision (Cowen et al., 2012; Hashemnia et al., 2020). Consistent with this outcome, rats with ACCd lesions were far more likely than controls to abort a high reward/high effort choice at the point where they encountered the ramp (Holec et al., 2014). These results suggest that the ACCd is needed when deciding to maintain a previously chosen course of action against a high effort. In this view, the ACCd encodes the amount of effort necessary to achieve a particular goal thus provides a signal that allows other systems to prepare for exerting the appropriate amount of effort. In our study, we did not find a systematic representation of the action cost associated with any specific offer. However, there was significant encoding of the chosen action cost in the late phase of the trial, when actions were planned. Moreover, chosen value signals were much more sustained in ACCd. Taken together, our findings support the view of ACCd in regulating downstream systems based on the outcome of decisions.
The fact that we used the same experimental design and the same animals in our OFC and ACCd studies allowed us to compare the contribution of OFC and ACCd to economic choice under variable action costs. We first discovered that the percentage of task-relevant cells was strikingly similar in these two regions. At the same time, these two regions did not encode the same set of task-relevant variables. Most importantly, OFC but not ACCd encoded offer values in cost-based reference frame, integrating all choice-relevant attributes. In our paradigm, goods and values might be represented in the frames of reference defined by the juice type, the action cost, the spatial location of the offer, or the spatial location of the saccade target. Neurons in ACCd did not appear to encode offer values in any of these reference frames. On the other hand, neurons in ACCd encoded the saccadic action of the animal – a signal that was absent in OFC. Furthermore, encoding of the chosen value was more sustained in the ACCd as indicated by significantly higher percentage of neurons encoding this variable in late-delay and mem-delay time windows. Lastly, in ACCd, percentage of neurons encoding the chosen cost was substantially higher than that encoding the chosen juice. Our discovery that few neurons in ACCd encode chosen juice likely reflects the fact that it is the OFC but not the ACCd that receives direct input from the primary taste cortex (Rolls, 2015). Overall, these findings differentiate the ACCd from the OFC in informing the motor system of the expected reward and cost associated with the outcome of the choice.
In another study, Hosokawa et al., 2013, compared the neuronal coding in ACCd and OFC in a choice task involving cost-benefit tradeoff. Our findings differ in two aspects. First, Hosokawa et al., 2013, reported contralateral action value coding in ACCd while we did not discover significant offer value coding in either spatial- or action-based reference frames in our ACCd recordings. Second, they reported that there was no action-based value representation in the OFC, therefore concluded that OFC does not integrate action cost in economic choice. Two elements may help explain the discrepancies between our findings in ACCd and OFC (Cai and Padoa-Schioppa, 2019) and those of Hosokawa et al., 2013. First, we recall that Hosokawa et al., 2013, only tested value-related variables such as the benefit, cost, and discounted value in action-based reference frame. Most importantly, they did not test the variable that is related to the saccade direction, which is highly correlated with the spatial value signal. As a consequence, contralateral value signal may not be significant if chosen target location was included in their regression analysis. Indeed, in our analysis, saccade direction (or chosen target location) was identified as one of the variables that explained a significant portion of neuronal activity in ACCd (Cai and Padoa-Schioppa, 2012; Cai and Padoa-Schioppa, 2019). The second and often overlooked aspect is that value may be encoded in schemes other than the action-based reference frame. In their study, each unique combination of reward quantity and cost was presented by a unique picture. Thus, information on good attributes were conveyed to the animal with an ‘integrated’ visual representation. Accordingly, a distinct group of neurons may have been recruited to encode the reward and cost conjunctively represented by a unique fractal, which would result in 16 groups of offer value coding neurons.
To conclude, our findings demonstrate that choice outcome coding dominates the neuronal representations in ACCd during economic choice. When action cost was introduced as an additional value attribute, neurons in ACCd expanded the scope of encoding incorporating choice outcome signals in the cost-based reference frame. However, the detailed neuronal reorganization during this process remains unknown. To uncover this process, future studies should monitor the same population of neurons during such reorganization.
Materials and methods
Choice task
Request a detailed protocolAll experimental procedures conformed to the NIH Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committee (IACUC) at Washington University.
The experimental procedures were as described in a previous study (Cai and Padoa-Schioppa, 2019). Briefly, two rhesus monkeys (B, male, 9.0 kg; L, female, 6.5 kg) participated in the experiments. Before training, a head-restraining device and an oval recording chamber were implanted on the skull under general anesthesia, as previously described (Cai and Padoa-Schioppa, 2014). During the experiments, animals sat in an electrically insulated enclosure (Crist Instruments) with their head restrained. The eye position was monitored with an infrared video camera (Eyelink; SR Research). The behavioral task was controlled through a custom-written software (http://www.monkeylogic.net/) based on Matlab (MathWorks).
Figure 1 illustrates the choice task. In each session, an animal chose between two juices offered in variable amounts and at different action cost. The offers were represented by sets of color symbols, with the color indicating the juice type, the number of symbols indicating juice amount, and the shape of the symbols indicating the action cost (cross for low cost; diamond for high cost). The trial started with the animal fixating the center fixation point. After 1.5 s, two offers appeared on the two sides of the fixation point. The offers remained on the monitor for 1 s. The monkey continued fixating for another 1 s, after which two saccade targets appeared. The two saccade targets, represented by two dots with matching color to that of the two offers, were located on two concentric rings centered on the fixation point. The radius for low-cost targets was 3.5–4°; the radius for high-cost targets was 10–16°. In each trial, one of the saccade targets was placed on the low-cost (small radius) ring while the other saccade target was placed on high-cost (large radius) ring. The two targets were always placed on opposite sides of the center fixation point. The angle defining their position was selected on every trial among four possible values, corresponding to 22.5°, 157.5°, 202.5°, and 337.5° from azimuth. Thus, for each juice, there were eight possible saccade target positions (2 distances × 4 angles). The monkey maintained center fixation for a randomly variable delay (0.6–1.2 s), at the end of which the fixation point was extinguished (go signal). At that point, the animal was allowed to indicate its choice with a saccade. The animal had to maintain peripheral fixation for an additional 0.75 s, at the end of which the chosen juice was delivered. In each session, the two juice quantities varied pseudo-randomly from trial to trial. The spatial positions of the offers, the action costs, and the angle of the saccade targets varied pseudo-randomly and were counter-balanced across trials. Different pairs of juices were used across sessions.
Neuronal recordings
Request a detailed protocolProcedures for surgery, neuronal recordings, and spike sorting were similar to those described previously (Cai and Padoa-Schioppa, 2014). In brief, the recording chamber (main axes, 50 mm × 30 mm) was centered on stereotaxic coordinates (A30, L0), with the longer axis parallel to the coronal plane. Neuronal recordings were guided by structural MRI obtained for each animal before and after the implant. In monkey B, we recorded from both hemispheres and recording locations ranged A28-A38 in the anterior-posterior direction (with the corpus callosum extending anteriorly to A36). In monkey L, we recorded from the left hemisphere and recording locations ranged A25-A34 in the anterior-posterior direction (with the corpus callosum extending anteriorly to A31). Tungsten electrodes (125 µm diameter, FHC) were advanced using custom-built motorized micro-drives, with a 2.5 µm resolution. We typically used four electrodes in each session. Electrical signals were amplified and band-passed filtered (high pass: 300 Hz, low pass: 6 kHz; Lynx 8, Neuralynx, Inc). Action potentials were detected on-line and waveforms were saved to disk (25 kHz sampling rate; Power 1401, Spike 2; Cambridge Electronic Design). Spike sorting was conducted off-line (Spike 2; Cambridge Electronic Design) and only cells that appeared well isolated and stable throughout the session were included in the analysis.
Analysis of choice patterns
Request a detailed protocolAll analyses were conducted in Matlab (MathWorks). On any given trial, one ‘offer’ was defined by a juice type, its quantity, and its action cost (e.g., 3B−). An ‘offer type’ was defined by two offers (e.g., [1A+:3B−]). In this notation, ‘−’ indicates high action cost (long saccade) and ‘+’ indicates low action cost (short saccade). A ‘trial type’ was defined by two offers and a choice (e.g., [1A+:3B−, A]). Of note, the position of each saccade target was defined by a distance (two possible values) and an angle (four possible values). For the purpose of all the analyses, we turned the angle into a binary variable corresponding to whether the target associated to juice A or the chosen target was placed in the contralateral or ipsilateral hemifield.
In the behavioral analysis, we examined several factors that could affect choices, including the juice quantity, the action cost, the outcome of the previous trial (choice hysteresis), a term capturing a visual side bias, and a term capturing a saccade side bias. We thus constructed the following logistic model:
where choice B = 1 if the animal chose juice B and 0 otherwise; qJ was the quantity of juice J offered (with J = A, B); δjuice J, + = 1 if juice J was offered at low cost and 0 otherwise; δn−1, J = 1 if in the previous trial the animal had chosen and received juice J and 0 otherwise; δcost of J, cost n−1 = 1 if the cost of J is the same as that chosen in the previous trial and 0 otherwise; δoffer J, left = 1 if the offer of juice J was placed to the left of the center fixation and 0 otherwise; and δtarget J, left = 1 if the saccade target associated with juice J was placed in the left hemifield and 0 otherwise. For each session, the logistic regression provided a measure for the relative value of the two juices (ρ = a1/a0), for the difference in action cost (ξ = a2/a0), for the choice hysteresis related to the chosen juice (η = a3/a0) and to the chosen cost (φ = a4/a0), and for the spatial biases related to the offer position (δ = a5/a0) and to the target position (ε = a6/a0). In this formulation, each factor (action cost, hysteresis, spatial biases) is quantified as a value term, and all values are expressed in units of juice B. The relative value (ρ) is essentially the quantity ratio qB/qA that makes the animal indifferent between the two juices. The factor a1 can be thought of as an inverse temperature capturing the steepness of the sigmoid once all the effects included in the logistic regression are accounted for.
Task-related responses
Request a detailed protocolEach cell was analyzed in relation to the choice pattern recorded in the same session. In each trial, the neuronal activity was analyzed in nine time windows aligned with different behavioral events: pre-offer (0.5 s before the offer), post-offer (0.5 s after offer on), late-delay (0.5–1.0 s after offer on), mem-delay (0–0.5 s after offer off), pre-target (0.5 s before target on), post-target (0.5 s after target on), pre-go (0.5 s before the ‘go’), pre-juice (0.5 s before juice delivery), and post-juice (0.5 s after juice delivery). To identify task-related responses, each neuronal response was submitted to two 3-way ANOVAs (factors [trial type × offer A location × target A location]; factors [trial type × chosen offer location × chosen target location]). We imposed a significance threshold p < 0.001. Responses that passed this criterion for at least one factor in either ANOVA were identified as ‘task related’ and included in subsequent analyses.
Variable selection analysis
Request a detailed protocolOur goal was to identify the variables encoded in ACCd during economic choices under variable action cost. Our strategy was to define a large number of variables neurons in this area might conceivably encode, and to use procedures for variable selection to identify a small subset of variables that would best explain the population. Our analysis combined the approach previously adopted to analyze ACCd activity during standard choices (Cai and Padoa-Schioppa, 2012) with that adopted for OFC activity recorded using the present task (Cai and Padoa-Schioppa, 2019). The variables defined Table 2 and included in the analysis are essentially the same defined for OFC activity in this task (Cai and Padoa-Schioppa, 2019). Concurrently, earlier work found that many neurons in ACCd encoded the movement direction, alone or convolved with other decision variables (Cai and Padoa-Schioppa, 2012). Importantly, the present task dissociated the spatial location of the chosen offer from the movement direction. Thus, neuronal responses were analyzed with an ANCOVA using the variables defined in Table 2 as regressors and grouping data by whether the chosen offer or chosen target was on the ipsilateral or on the contralateral hemifield (binary variable). For early time windows (before target presentation), the categorical variable included in the ANCOVA was chosen offer location; for late time windows (after target presentation), the categorical variable was chosen target location.
In preliminary analyses, we tested different models of the ANCOVA and observed that the interaction term was rarely significant. We thus report here only the results obtained using the ‘parallel lines’ model. This model assumes that the encoding of the factor (variable) and the group (chosen location) are statistically independent. A variable was said to ‘explain’ the response if the factor in the ANCOVA was significant (p < 0.05). For each variable, the ANCOVA also provided an R2. This was the total R2, computed including both the factor and the group. For variables that did not explain the response, we arbitrarily set R2 = 0. These criteria were used for all variables except the chosen offer location only and chosen target location only, for which we proceeded as follows. We computed the ‘horizontal lines’ model of the ANCOVA, which assumes that firing rates only depend on the group (chosen offer location or chosen target location). Variables chosen offer location only or chosen target location only were thus said to explain the response if the group was significant in this analysis (p < 0.05). From this model, we also obtained the R2, which was set equal to 0 if the group was not significant. Importantly, the R2 thus obtained for the variable chosen offer location only or chosen target location only was always lower than that obtained for any other variable that explained the response. Hence, for any given response, chosen offer location only or chosen target location only provided the best explanation (highest R2) only if it provided the sole explanation.
The detailed procedures used in the variable selection analysis were described in previous reports (Cai and Padoa-Schioppa, 2012; Cai and Padoa-Schioppa, 2019) and we performed the analysis based on the neuronal activity across all nine time windows. Two procedures – stepwise and best-subset – identified a small number of variables that best explained the neuronal data set. The ‘explanatory power’ of any subset of variables was defined as the total number of responses collectively explained by those variables. In the stepwise procedure, we selected at each step the variable that provided the highest number of best fits within any time window. We then removed from the data set all the responses explained by this variable and we repeated the procedure on the residual data. The procedure was repeated until when the marginal explanatory power of any additional variable fell <2%. In the best-subset procedure, we identified the subset of n variables that collectively provided the highest explanatory power. Importantly, the best-subset procedure warrants optimality and the two procedures applied to our data set provided identical results.
Dimensional integration in chosen value signals
Request a detailed protocolWe examined the integration of multiple determinants into single value signals for chosen value (juice) responses. To do so, we defined two variants of the variable chosen value (juice) – one cost-affected and one cost-independent. We sought to assess which variant better fit neuronal responses. For each response, we considered the two R2 and we computed the difference ΔR2 = R2cost-affected – R2cost-independent and we examined the distribution for ΔR2 across the population. We did not want to bias the results in favor of either variant. Thus, for this analysis we identified neuronal responses encoding the chosen value (juice) as follows. For each response and each value variable, we considered the two R2 obtained from the two variants, and we assigned the maximum R2 to the response. We then assigned each response to one of the selected variables accordingly.
Data availability
All data generated or analysed during this study are included in the manuscript and supporting files.
References
-
The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-offThe Journal of Neuroscience 33:3844–3856.https://doi.org/10.1523/JNEUROSCI.2753-12.2013
-
Cognitive and emotional influences in anterior cingulate cortexTrends in Cognitive Sciences 4:215–222.https://doi.org/10.1016/S1364-6613(00)01483-2
-
Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortexThe Journal of Neuroscience 32:3791–3808.https://doi.org/10.1523/JNEUROSCI.3864-11.2012
-
Distribution and overlap of entorhinal, premotor, and amygdalar connections in the monkey anterior cingulate cortexThe Journal of Comparative Neurology 529:885–904.https://doi.org/10.1002/cne.24986
-
Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequencesJournal of Neurophysiology 107:2393–2407.https://doi.org/10.1152/jn.01012.2011
-
Effort-based cost-benefit valuation and the human brainThe Journal of Neuroscience 29:4531–4541.https://doi.org/10.1523/JNEUROSCI.4515-08.2009
-
Why neurons mix: high dimensionality for higher cognitionCurrent Opinion in Neurobiology 37:66–74.https://doi.org/10.1016/j.conb.2016.01.010
-
The anterior cingulate and reward-guided selection of actionsJournal of Neurophysiology 89:1161–1164.https://doi.org/10.1152/jn.00634.2002
-
Neurons in anterior cingulate cortex multiplex information about reward and actionThe Journal of Neuroscience 30:3339–3346.https://doi.org/10.1523/JNEUROSCI.4874-09.2010
-
Not all effort is equal: the role of the anterior cingulate cortex in different forms of effort-reward decisionsFrontiers in Behavioral Neuroscience 8:12.https://doi.org/10.3389/fnbeh.2014.00012
-
Single-neuron mechanisms underlying cost-benefit analysis in frontal cortexThe Journal of Neuroscience 33:17385–17397.https://doi.org/10.1523/JNEUROSCI.2221-13.2013
-
Optimal decision making and the anterior cingulate cortexNature Neuroscience 9:940–947.https://doi.org/10.1038/nn1724
-
Neurons in the frontal lobe encode the value of multiple decision variablesJournal of Cognitive Neuroscience 21:1162–1178.https://doi.org/10.1162/jocn.2009.21100
-
Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variablesThe European Journal of Neuroscience 29:2061–2073.https://doi.org/10.1111/j.1460-9568.2009.06743.x
-
Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-offThe Journal of Neuroscience 36:10002–10015.https://doi.org/10.1523/JNEUROSCI.0292-16.2016
-
Functional specialization of the primate frontal cortex during decision makingThe Journal of Neuroscience 27:8170–8173.https://doi.org/10.1523/JNEUROSCI.1561-07.2007
-
Primate anterior cingulate cortex: where motor control, drive and cognition interfaceNature Reviews. Neuroscience 2:417–424.https://doi.org/10.1038/35077500
-
Separate valuation subsystems for delay and effort decision costsThe Journal of Neuroscience 30:14080–14090.https://doi.org/10.1523/JNEUROSCI.2752-10.2010
-
Neural computations associated with goal-directed choiceCurrent Opinion in Neurobiology 20:262–270.https://doi.org/10.1016/j.conb.2010.03.001
-
Taste, olfactory, and food reward value processing in the brainProgress in Neurobiology 127:64–90.https://doi.org/10.1016/j.pneurobio.2015.03.002
-
Separate neural pathways process different decision costsNature Neuroscience 9:1161–1168.https://doi.org/10.1038/nn1756
-
Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviourTrends in Cognitive Sciences 11:168–176.https://doi.org/10.1016/j.tics.2007.01.004
-
Heterogeneous reward signals in prefrontal cortexCurrent Opinion in Neurobiology 20:191–198.https://doi.org/10.1016/j.conb.2010.02.009
-
The role of rat medial frontal cortex in effort-based decision makingThe Journal of Neuroscience 22:10996–11003.https://doi.org/10.1523/JNEUROSCI.22-24-10996.2002
Article and author information
Author details
Funding
National Institute of Mental Health (R01-DA032758)
- Camillo Padoa-Schioppa
National Natural Science Foundation of China (grants 31571102 and 91632106)
- Xinying Cai
Ministry of Education Program of Introducing Talents of Discipline to Universities (Base B16018)
- Xinying Cai
NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai (Joint Research Institute Seed Grants for Research Collaboration)
- Xinying Cai
Science and Technology Commission of Shanghai Municipality (grants 15JC1400104 and 16JC1400101)
- Xinying Cai
The Shanghai Municipal Science and Technology Major Project (2018SHZDZX05)
- Xinying Cai
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
We thank Heide Schoknecht for help with animal training.
Ethics
All experimental procedures conformed to the NIH Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committee (IACUC) at Washington University.
Copyright
© 2021, Cai and Padoa-Schioppa
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 896
- views
-
- 138
- downloads
-
- 6
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Developmental Biology
- Neuroscience
The blood-brain barrier (BBB) controls the movement of molecules into and out of the central nervous system (CNS). Since a functional BBB forms by mouse embryonic day E15.5, we reasoned that gene cohorts expressed in CNS endothelial cells (EC) at E13.5 contribute to BBB formation. In contrast, adult gene signatures reflect BBB maintenance mechanisms. Supporting this hypothesis, transcriptomic analysis revealed distinct cohorts of EC genes involved in BBB formation and maintenance. Here, we demonstrate that epigenetic regulator’s histone deacetylase 2 (HDAC2) and polycomb repressive complex 2 (PRC2) control EC gene expression for BBB development and prevent Wnt/β-catenin (Wnt) target genes from being expressed in adult CNS ECs. Low Wnt activity during development modifies BBB genes epigenetically for the formation of functional BBB. As a Class-I HDAC inhibitor induces adult CNS ECs to regain Wnt activity and BBB genetic signatures that support BBB formation, our results inform strategies to promote BBB repair.
-
- Computational and Systems Biology
- Neuroscience
Hypothalamic kisspeptin (Kiss1) neurons are vital for pubertal development and reproduction. Arcuate nucleus Kiss1 (Kiss1ARH) neurons are responsible for the pulsatile release of gonadotropin-releasing hormone (GnRH). In females, the behavior of Kiss1ARH neurons, expressing Kiss1, neurokinin B (NKB), and dynorphin (Dyn), varies throughout the ovarian cycle. Studies indicate that 17β-estradiol (E2) reduces peptide expression but increases Slc17a6 (Vglut2) mRNA and glutamate neurotransmission in these neurons, suggesting a shift from peptidergic to glutamatergic signaling. To investigate this shift, we combined transcriptomics, electrophysiology, and mathematical modeling. Our results demonstrate that E2 treatment upregulates the mRNA expression of voltage-activated calcium channels, elevating the whole-cell calcium current that contributes to high-frequency burst firing. Additionally, E2 treatment decreased the mRNA levels of canonical transient receptor potential (TPRC) 5 and G protein-coupled K+ (GIRK) channels. When Trpc5 channels in Kiss1ARH neurons were deleted using CRISPR/SaCas9, the slow excitatory postsynaptic potential was eliminated. Our data enabled us to formulate a biophysically realistic mathematical model of Kiss1ARH neurons, suggesting that E2 modifies ionic conductances in these neurons, enabling the transition from high-frequency synchronous firing through NKB-driven activation of TRPC5 channels to a short bursting mode facilitating glutamate release. In a low E2 milieu, synchronous firing of Kiss1ARH neurons drives pulsatile release of GnRH, while the transition to burst firing with high, preovulatory levels of E2 would facilitate the GnRH surge through its glutamatergic synaptic connection to preoptic Kiss1 neurons.