We first sought to establish whether plasticity-related activity differs in dSPNs and iSPNs during the acquisition of goal-directed action. Based on the above considerations, we used a pathway-specific approach, targeting dSPNs and iSPNs by injecting the retrograde tracers, fluorogold (FG) and cholera-toxin B (CTB) subunit, bilaterally into the direct monosynaptic targets of each pathway, the SNr and GPe, respectively (Figure 1A and B). The spread of these injection sites is presented in Figure 1—figure supplement 1A. We were able to identify dSPNs and iSPNs in the same animal; Figure 1C (also see Figure 1—figure supplement 1B) shows triple labelling of SPNs with either FG or CTB or both, co-labelled with DARPP-32, a marker for SPNs. There was no difference in the number of cells/mm² labelled with FG or CTB (Figure 1—figure supplement 1C; F(1,20)=1.26, p=0.276) or in the mean percentage of neurons co-labelled with either FG or CTB (Figure 1D; F(1,20)=1.2, p=0.279), suggesting that this approach was not significantly biased towards one or other population of SPNs. Overall, we were able to target approximately half of all SPNs identified with DARPP-32. There was a substantial population of cells labelled with both tracers, suggesting a degree of overlap in collaterals projecting from both cell types to the SNr and GPe (estimated at 16–17% of the total number of retrogradely labelled cells).

Figure 1 with 1 supplement see all

Download asset Open asset

Next, we assessed the learning-related activity in dSPNs and iSPNs induced by instrumental training. Half of the rats were given training over 4 days, in which pressing a lever (located to the left or right side of a central food magazine) delivered grain pellets on increasing random interval schedules of reinforcement (see Materials and methods). A second, control group, received yoked training in which grain pellets were delivered at the same average interval as the instrumentally trained rats, but independently of lever pressing. This resulted in robust acquisition of the instrumental response for rats in Group Instrumental, but not for rats in Group Yoked (Figure 1E; F(1,20)=90.73, p<0.001). Both groups entered the magazine to retrieve the food, and although Group Instrumental had numerically higher entry rates, this difference was not significant (Figure 1F; F(1,20)=4.04, p=0.058).

Ninety minutes after the beginning of the final training session, rats were perfused and we used immunofluorescence to measure the expression of the immediate-early gene Zif268 (also known as EGR1), widely used as an activity marker for learning-related plasticity in the striatum (Moratalla et al., 1992; Knapska and Kaczmarek, 2004; Hernandez et al., 2006; Maroteaux et al., 2014), in dSPNs and iSPNs in the pDMS. There was robust labelling of Zif268 in both groups (Figure 1G and H). Figure 1I and J shows the percentage of dSPNs and iSPNs co-labelled with Zif268, respectively. Rats in Group Instrumental had significantly higher Zif268 expression in dSPNs than rats in Group Yoked (F(1,20)=4.52, p=0.046). In contrast, analysis of Zif268 expression in iSPNs revealed no significant differences between Group Instrumental and Group Yoked in overall levels of Zif268 expression (F(1,20)=1.43, p=0.245).

We next sought to assess the causal involvement of dSPNs and iSPNs in goal-directed learning. To induce prolonged suppression of neuronal activity across each day of instrumental training, chemogenetic (DREADDs) inhibition is optimal; however, the use of DREADDs in striatal neurons has been a topic of debate in the literature with concerns raised over its effectiveness. Although there have been successful reports of the use of DREADDs in striatal neurons (Ferguson et al., 2011; Ferguson et al., 2013; Carvalho Poyraz et al., 2016), SPNs have a relatively low number of G-protein-activated inwardly rectifying potassium channels (Lovinger, 2010). We therefore assessed whether hM4Di DREADDs expressed on SPNs could suppress cortically evoked neuronal firing following the application of the synthetic ligand of the hM4Di receptor, clozapine-N-oxide (CNO; RTI International).

We used a dual-virus approach (Figure 2A; Marchant et al., 2016; Hart et al., 2018a) to express hM4Di receptors on dSPNs or iSPNs, by bilaterally infusing a retrogradely transported AAV-virus expressing Cre recombinase (rAAV5.CMV.HI.eGFP-CRE.WPRE.SV40; retro-Cre) into either the SNr or GPe to express Cre on dSPNs or iSPNs, respectively. We then injected a Cre-dependent hM4Di DREADDs virus (rAAV5/hSyn-DIO-hM4D(Gi)-mCherry; DIO-hM4D) or a control fluorophore lacking the hM4D construct (rAAV5/hSyn-DIO-mCherry; DIO-mCherry) into the pDMS, to selectively express DIO-hM4D or DIO-mCherry on either dSPNs or iSPNs. To induce neuronal firing in SPNs, we injected channelrhodopsin (AAV5-CaMKIIa-hChR2(H134R)-eYFP; ChR2-eYFP) into the prelimbic cortex (Figure 2A), which has dense bilateral glutamatergic projections onto dSPNs and iSPNs (Hart et al., 2018a; Wall et al., 2013). We used ex vivo slice electrophysiology to assess the effect of CNO on the cortically evoked firing of dSPNs and iSPNs expressing hM4Di DREADDs.

Figure 2 with 1 supplement see all

Download asset Open asset

Figure 2B shows the raw trace recorded from a dSPN (top) and an iSPN (bottom) expressing DIO-hM4D; action potentials were elicited by pulsing an LED light onto ChR2-containing corticostriatal axons (blue bars). Group CNO control consisted of cells expressing DIO-mCherry on dSPNs as well as iSPNs. A raw trace recorded from a dSPN expressing DIO-mCherry and confocal images of one representative recorded neuron from each group are presented in Figure 2—figure supplement 1A and B. We compared action potential frequency at three time points; 1 min prior, immediately prior, and 1 min after CNO application. There was no difference in any group in action potential frequency between the two pre-CNO time points (Figure 2C; pairwise comparisons, highest F(1,21)=2.68, p=0.117, iSPN hM4D). Following the administration of CNO, action potential frequency was significantly reduced relative to the time point immediately prior to CNO application in Group dSPN hM4D and iSPN hM4D but not in Group CNO control (pairwise comparisons, Group CNO control, F(1,21)=0.93, p=0.346; Group dSPN hM4D, F(1,21)=8.96, p=0.007; Group iSPN hM4D, F(1,21)=15.8, p=0.001).

Having established that hM4Di DREADDs expressed on SPNs could suppress cortically evoked firing, we next assessed whether this activity was necessary for goal-directed learning. We used the same dual-virus approach to express either DIO-hM4D or DIO-mCherry on dSPNs (Figure 2D) or iSPNs (Figure 2F). Example images of retro-Cre expression in the SNr (left) and DIO-hM4D expression on pDMS dSPNs (right) are presented in Figure 2E. The same is shown for retro-Cre expression in the GPe and DIO-hM4D expression on pDMS iSPNs in Figure 2G. The spread of retro-Cre and DIO-hM4D expression in the SNr or GPe and pDMS respectively in all included animals is presented in Figure 2—figure supplement 1C–E.

We quantified the mean number of cells expressing DIO-hM4D or DIO-mCherry for each animal (Figure 2—figure supplement 1F); there was no difference in the number of dSPNs or iSPNs expressing DIO-hM4D (dSPN+VEH versus dSPN+CNO, F(1,46)=0.06, p=0.808; iSPN+VEH versus iSPN+CNO, F(1,46)=0.52, p=0.473). There were, however, significantly more SPNs (half dSPNs and half iSPNs) expressing DIO-mCherry (Group CNO control versus rest; F(1,46)=23.93, p<0.001). We compared the rate of infection achieved with this dual-virus approach in dSPNs to that obtained using retrograde tracing (see Figure 1—figure supplement 1C); retro-Cre in combination with DIO-hM4D achieved 38–42% of the labelling achieved with FG, and this percentage is consistent with that previously reported using the same viral combination in the PL-pDMS pathway, and compared to FG (e.g., 30–45% estimated by Hart et al., 2018a). For iSPNs, retro-Cre in combination with DIO-hM4D achieved 81–102% of the labelling that was achieved with CTB. The substantially higher infection rate from GPe is likely to be due to a combination of two things; fewer CTB labelled neurons compared to FG (see Figure 1—figure supplement 1C) and more efficient retro-Cre transport in striatopallidal neurons, possibly due to the shorter pathway length.

For behavioural testing there were five groups of rats, summarized in Figure 2H. There were three control groups: (1) Group CNO control, expressed DIO-mCherry on SPNs (half on dSPNs and half on iSPNs which were pooled based on a lack of significant differences in press rate during training or test; Fs <3.69, lowest p=0.081); (2) Group dSPN+VEH, and (3) Group iSPN+VEH both of which expressed DIO-hM4D on dSPNs or iSPNs, respectively, and received injections of vehicle throughout training. The two experimental groups expressed DIO-hM4D and received injections of CNO throughout training to suppress the activity of dSPNs (Group dSPN+CNO) or iSPNs (Group iSPN+CNO).

A summary of the behavioural design is presented in Figure 2I. Rats were given 5 days of instrumental pre-training with two levers for a common outcome (Oc) after which they received 2 days of instrumental training with the same two levers paired with new, distinct outcomes (R1–O1 and R2–O2). Intraperitoneal injections of CNO or VEH were administered 1 hr prior to these training sessions. All rats acquired the instrumental response across pre-training (Figure 2—figure supplement 1G). Across 2 days of instrumental training (Figure 2J), all rats increased their response rates from Day 1 to Day 2 (F(1,46)=83.28, p<0.001) and there were no significant differences between groups in overall response rates (Bonferroni, k=3, highest F(1,46)=4.86, p=0.099, dSPN+CNO versus controls).

In order to assess goal-directed learning, we gave rats a choice extinction test, drug free, immediately after specific satiety-induced outcome devaluation. Prior to the test rats were allowed freely to consume one of the previously earned outcomes (O1 or O2) for 1 hr after which they were returned to the operant chambers and tested with both levers available in a choice extinction test. This test was conducted twice so that each outcome was devalued across tests and each lever served as the devalued lever, allowing the effect of devaluation to be assessed within subjects. Goal-directed action is demonstrated by rats showing a reduction in responding on the lever previously associated with the now-devalued outcome relative to the other lever, indicating their ability to integrate prior action-outcome learning with the current value of the outcome. Responding on the valued and devalued levers is presented for each rat in each group in Figure 2K. We compared responding in Groups dSPN+CNO or iSPN+CNO against the three control groups; dSPN+VEH, iSPN+VEH, and CNO control. We also compared the control groups that received vehicle to the CNO control. There were no differences between groups in overall rates of responding (main effect of group, highest F(1,46)=2.01, p=0.163, VEH controls versus CNO control). There was a main effect of devaluation (Bonferroni, k=3, F(1,46)=28.73, p<0.001) and, more importantly, a significant interaction between group (dSPN+CNO versus controls) and devaluation (Bonferroni, k=3, F(1,46)=6.93, p=0.012) indicating that whereas the control groups showed a reliable devaluation effect, this was abolished in rats that had dSPNs suppressed during instrumental training. Importantly, there was no difference in the magnitude of the devaluation effect in Group iSPN+CNO relative to the three control groups, or between the two vehicle groups and Group CNO control (Fs <1, lowest p=0.677). These results suggest that dSPN activity in the pDMS is necessary during training for animals to encode the specific action-outcome associations necessary to acquire goal-directed instrumental actions.

In a subset of rats, we examined whether this effect was attributable to a diminished capacity to discriminate between responses or outcomes, when dSPNs or iSPNs were suppressed. To achieve this, rats were injected with CNO or vehicle (group allocations were unchanged) and given a rewarded choice test following outcome devaluation (Figure 2—figure supplement 1H), in which each lever delivered its respective outcome as in training, negating the need for animals to use prior action-outcome learning to bias actions. Under these circumstances, all rats were able to respond according to current outcome value, and there were no differences between groups (main effect of lever, Bonferroni, k=3, F(1,18)=53.24, p<0.001; no effect of group and no group × lever interactions, Fs <1.6, lowest p=0.224). This result demonstrates that, under dSPN (or iSPN) inhibition, rats can discriminate the lever press actions and outcomes and that specific satiety-induced outcome devaluation is effective in biasing choice when animals are given feedback on those choices, confirming that the effects of dSPN inactivation were specific to goal-directed learning.

Finally, although we saw no evidence for motor disruption across training or during the rewarded choice test when activity in dSPNs or iSPNs was suppressed, we also assessed whether suppressing activity in pDMS SPNs induces a locomotor impairment more directly using a rotarod test (Hamm et al., 1994). Rats infused with DIO-hM4D were all tested twice; once under vehicle and once under CNO. As shown in Figure 2—figure supplement 1I, suppressing SPNs did not affect the amount of time spent on the rotarod for rats in any group (pairwise comparisons between CNO and VEH in each group, highest F=2.7, dSPN+VEH). Therefore, although it remains possible that fine motor movements were affected by inactivation, the suppression of dSPNs or iSPNs produced no overt impairments detectable using this test.

The results so far suggest that goal-directed learning relies on dSPN plasticity during training; however, it is possible that learning was disrupted because dSPN inhibition altered the relative output of dSPNs and iSPNs to favour heightened iSPN output. To assess this possibility, we tested whether increased iSPN activity could similarly affect goal-directed learning. We used a Cre-dependent hM3Dq DREADD (rAAV5/hSyn-DIO-hM3D-mCherry; DIO-hM3D) to stimulate the activity of iSPNs targeted using the same retro-Cre approach described (Figure 3A and B). We used slice electrophysiology to verify that CNO increased neuronal firing in iSPNs expressing Cre-dependent hM3Dq DREADDs; application of CNO produced a significant increase in the number of action potentials recorded from iSPNs expressing DIO-hM3D, relative to baseline (Figure 3D and E; paired t-test, p=0.049). A confocal image of a representative recorded neuron is presented in Figure 3—figure supplement 1A.

Figure 3 with 1 supplement see all

Download asset Open asset

Next, we assessed the functional consequences of iSPN stimulation on goal-directed learning using the behavioural design described previously (Figure 2I). Expression of retro-Cre in the GPe and DIO-hM3D in the pDMS is shown in Figure 3B and the location and spread of virus expression for all included animals are shown in Figure 3—figure supplement 1B and C. The mean number of cells expressing DIO-mCherry or DIO-hM3D was quantified and is presented in Figure 3—figure supplement 1D; there were no significant differences between groups or viruses (highest F(1,17)=2.51, p=0.131). There were three groups of rats, summarized in Figure 3C: all groups received retro-Cre into the GPe. There were two control groups; one expressed DIO-mCherry on iSPNs and received CNO injections throughout training and the other expressed DIO-hM3D on iSPNs and received VEH injections throughout training. The third group expressed DIO-hM3D on iSPNs and received CNO injections throughout training to stimulate the activity of iSPNs in the pDMS. We found no effect of stimulating the activity of iSPNs during instrumental training on press rates (Figure 3F; Fs <1, lowest p=0.391), and found that this stimulation did not affect goal-directed learning (Figure 3G; main effect of devaluation, F(1,17)=34.561, p<0.001; no devaluation × group interaction, Fs <1.1, lowest p=0.313). Therefore, neither inhibition nor stimulation of iSPNs during instrumental training produced any detectable effect on goal-directed learning.

Having confirmed that goal-directed learning is dependent on the activity of dSPNs but not iSPNs, we assessed the involvement of these pathways in goal-directed performance. Here we employed an optogenetic approach for temporally precise inhibition of projection neurons during action performance. We used the same dual-virus approach described above to express halorhodopsin (AAV5-EF1a-DIO-eNpHR3.0-eYFP; DIO-eNpHR) on dSPNs (Figure 4A,D, and E) or iSPNs (Figure 4A,F, and G) so as to allow the selective inhibition of each pathway using light from a 625 nm wavelength LED delivered to the pDMS. Using ex vivo slice electrophysiology, we validated the effectiveness of this inhibition. Following the delivery of LED light, action potential frequency of SPNs (dSPNs and iSPNs were pooled due to little or no difference in their response to LED illumination) was significantly reduced in DIO-eNpHR labelled SPNs (Figure 4B and C; pairwise comparison, F(1,8)=11.76, p=0.009) but not in non-labelled SPNs (Figure 4—figure supplement 1A and B; pairwise comparison, F(1,4)=1.4, p=0.306). Confocal images of a representative recorded dSPN, iSPN, and an unlabelled SPN are presented in Figure 4—figure supplement 1C–E.

Figure 4 with 2 supplements see all

Download asset Open asset

Groups used in this experiment are summarized in Figure 4H. Rats in the eYFP control group expressed a control fluorophore (AAV5-EF1a-DIO-eYFP; DIO-eYFP) in either dSPNs or iSPNs, serving as a control for LED light delivery. Due to the lack of eNpHR construct, this group of rats retained intact pathway function and, indeed, as there were no differences in their press rates during training and no differential effects of LED light presentation on test (Fs <1), they were pooled into a single control group. Example images of retro-Cre expression and the extent of virus spread in the SNr and GPe are presented in Figure 4—figure supplement 2A–D. We quantified the mean number of cells in the pDMS expressing DIO-eYFP or DIO-eNpHR (Figure 4—figure supplement 2E); animals that received DIO-eYFP had greater expression than those that received DIO-eNpHR (F(1,22)=12.8, p=0.002), but there were no differences between Groups dSPN eNpHR and iSPN eNpHR, nor any differences between hemispheres in any group (Fs <1.5, lowest p=0.234). The spread of pDMS virus expression and location of optogenetic cannula tips for each group are presented in Figure 4—figure supplement 2F.

A summary of the behavioural design is shown in Figure 4I. Instrumental pre-training and training sessions were conducted without LED light delivery. For analysis of the optogenetic inhibition experiments, we compared the control group against the two experimental groups, dSPN eNpHR and iSPN eNpHR as well as comparing Groups dSPN eNpHR and iSPN eNpHR against each other. All groups acquired the instrumental response across pre-training (Figure 4—figure supplement 2G). Across four days of instrumental training (Figure 4J), all groups increased their press rates (F(1,22)=188.35, p<0.001) and there were no differences between groups in overall response rates (all Fs <1.1). We tested that actions were goal directed using outcome devaluation followed by a choice extinction test. The test was 7.5 min long; dSPNs or iSPNs in the pDMS were inhibited with orange LED light during the first and last 2.5 min, separated by 2.5 min with no LED light. Responding in LED ON periods was similar (no group × LED ON period, lever × LED ON period or group × lever × LED ON period interactions; all Fs <1, lowest p=0.623) and so averaged for comparison with the LED OFF period. We found a main effect of devaluation (Figure 4K; F(1,22)=37.57, p<0.001) but no effect of LED and no group × LED × lever interaction (Fs <1, lowest p=0.390), indicating that rats showed intact outcome devaluation, and this was unaffected by optogenetic inhibition of either dSPNs or iSPNs. Additionally, optogenetic inhibition, whether of dSPNs or iSPNs, had no effect on the rats’ ability to discriminate between outcomes or levers during a rewarded devaluation choice test (Figure 4—figure supplement 2H; main effect of lever [valued versus devalued] F(1,22)=29.28, p<0.001; no group × lever, group × LED or group × LED × lever interactions, highest F(1,22)=2.87, p=0.104 [LED × lever interaction]). Therefore, unlike goal-directed learning, bilateral optogenetic inhibition of dSPNs had no effect on goal-directed performance.

To further examine the role of dSPNs and iSPNs in performance we used the same animals and examined the effect of unilateral inhibition of these SPNs on response bias in a choice extinction test conducted without devaluation (i.e. without pre-feeding; Figure 4L). As represented in Figure 4M, a single patch cord was attached to either the left or right cannula (first and second tests, respectively) and rats were again tested for 7.5 min as described previously. There were no differences between the first and last LED ON periods and so these blocks were pooled and compared to the LED OFF block. Test data are presented in Figure 4N. There were no differences in press rates according to lever side or LED, averaged across group, nor between Group eYFP and the two eNpHR groups (averaged) in these measurements (F(1,22)=1.67, p=0.210). There was however a significant group × lever × LED interaction between Group dSPN eNpHR and Group iSPN eNpHR (F(1,22)=5.78, p=0.025), indicating that the effect of the light on lever choice differed between these groups. To follow up, we analysed each of the groups separately, comparing responding on the ipsilateral or contralateral levers during the LED ON period as a ratio of total responding (LED ON plus LED OFF periods) for each lever (Figure 4O). For rats in Group iSPN eNpHR and Group eYFP control, responses on each lever during the LED ON period were ~50% of the total responding on each lever, whereas for rats in Group dSPN eNpHR we observed a significant ipsilateral bias (F(1,22)=6.28, p=0.02); 59% versus 46% for ipsilateral versus contralateral levers, respectively. Therefore, unilateral dSPN inhibition, but not iSPN inhibition, produced an ipsilateral bias in goal-directed performance. To ensure that this effect was not attributable to LED-induced rotational behaviour, we quantified rotations for each animal during this test (Figure 4—figure supplement 2I and J) and found no effect of unilateral LED light delivery on the number of ipsilateral (Fs <2.1, lowest p=0.169) or contralateral (Fs <1, lowest p=0.342) rotations in any group.

Given this finding, we revisited the Zif268 quantification described in our initial experiment, separating expression in each hemisphere based on the location of the lever in the chamber (Figure 4—figure supplement 2K and L); i.e., if a rat had the lever on the right side of the chamber, the left hemisphere was designated as ‘contralateral’, and the right hemisphere as ‘ipsilateral’. For Zif268 expression in dSPNs, there was a pronounced hemispheric difference in Group Instrumental but not in Group Yoked; rats in Group Instrumental had significantly more Zif268 in the hemisphere contralateral to the lever side (group × hemisphere interaction, F(1,20)=4.84, p=0.040). In contrast, analysis of Zif268 expression in iSPNs revealed no significant differences between hemispheres (no group × hemisphere interaction, F(1,20)=0.70, p=0.413). Therefore, acquisition of goal-directed actions appeared to produce a lateralized (contralateral to action space) increase in the activity of dSPNs sufficient to bias instrumental performance.

Our experiments so far have failed to detect any significant role for iSPNs in the acquisition and performance of goal-directed actions. In fact, recent findings suggest that iSPN activity may only be necessary for learning and performance when instrumental contingencies change (Matamales et al., 2020). In order to assess this possibility, we examined the effect of dSPN and iSPN inhibition when action-outcome contingencies were altered using a series of reversals in a two-choice situation. We used an instrumental reward reversal paradigm, wherein the rats were exposed to the same action-outcome contingencies as initial training, but with one lever rewarded and the other lever non-rewarded (i.e., if R1→O1 then R2→Ø; whereas if R2→O2 then R1→Ø). These contingencies were reversed every 2.5 min in semi-random fashion (with no more than two of the same trial type presented sequentially). Each test contained 10 × 2.5 min trials, and rats were tested each day for 3 days. LED light was delivered bilaterally during all trials, but not between trials (Figure 5A and B).

Figure 5 with 1 supplement see all

Download asset Open asset

Press rates on the non-rewarded lever, the rewarded lever, and the difference in press rates between the two levers are presented in Figure 5C–E. There were significant effects of training day in all three metrics, indicating that rats decreased responding on the non-rewarded lever across days (Figure 5C; F(1,22)=9.84, p=0.005) and increased responding on the rewarded lever (Figure 5D; F(1,22)=11.61, p=0.003), and that the magnitude of the difference between the rewarded and non-rewarded levers increased across days (Figure 5E; F(1,22)=19.82, p<0.001). We compared the difference in responding on the rewarded and non-rewarded levers between groups on each day (Figure 5E). On Day 1 there was a significant difference between Group eYFP control and the two eNpHR groups (F(1,22)=7.03, p=0.015), which was driven by reduced discrimination in Group iSPN eNpHR relative to the other groups. This was confirmed by a significant difference between Group iSPN eNpHR and Group dSPN eNpHR (F(1,22)=4.39, p=0.048). The difference between the control group and the two eNpHR groups was maintained on Day 2 (F(1,22)=5.91, p=0.024), but not between the two eNpHR groups (F(1,22)=1.8, p=0.193). There were no significant differences on Day 3 (Fs <1, lowest p=0.548). To confirm that the observed effects were primarily driven by rats in Group iSPN eNpHR, we conducted a follow-up analysis comparing each eNpHR group to the control group. On Days 1 and 2, rats in Group iSPN eNpHR showed a significant deficit in discriminating the rewarded versus the non-rewarded levers as determined by the difference in the press rates on these levers (Bonferroni k=6, Day 1, F(1,22)=12.45, p=0.002; Day 2, F(1,22)=8.4, p=0.008). This deficit reduced over days and was not detectable on Day 3 (Fs <1). There was no difference in performance between Group dSPN eNpHR and the eYFP control on any day (Fs <1.6).

Figure 5F–H and Figure 5—figure supplement 1A–F show the rate of responding on the left and right levers for each group, across the 3 days of training – shaded blocks indicate those in which the left lever was rewarded and unshaded those in which the right lever was rewarded. We analysed each day separately: Rats in Group eYFP control and dSPN eNpHR adjusted their performance according to the positive contingency during each epoch on Day 1 (Figure 5F–H); there was a significant main effect of reversal (lever × trial, Bonferroni, k=9, F(1,22)=56.4, p<0.001) and this did not differ between groups (F(1,22)=1.28, p=0.27). However, rats in Group iSPN eNpHR failed to adjust their performance as the positive contingency shifted across epochs; there was a significant Group × reversal interaction between Groups iSPN eNpHR and eYFP control (Bonferroni, k=9, F(1,22)=12.27, p=0.002) indicating that Group iSPN eNpHR was impaired at flexibly adjusting performance on Day 1. This pattern was partially maintained on Day 2 (Figure 5—figure supplement 1A–C): There was a significant main effect of reversal (Bonferroni, k=9, F(1,22)=87.36, p<0.001); this did not differ between Group dSPN eNpHR and eYFP control (F(1,22)=1.51, p=0.232) and there was a substantial but non-significant Group × reversal interaction between Group iSPN eNpHR and eYFP control (Bonferroni, k=9, F(1,22)=8.39, p=0.008). Finally, on Day 3 (Figure 5—figure supplement 1D–F), groups maintained their sensitivity to contingency (main effect of reversal, Bonferroni, k=9, F(1,22)=77.83, p<0.001), and there were no differences in the magnitude of reversal between Group dSPN eNpHR and eYFP control (F(1,22)=0.37, p=0.548), nor between Group iSPN eNpHR and eYFP control (F(1,22)=0.16, p=0.697). Therefore, optogenetic inhibition of iSPNs impaired rats’ ability to adjust their instrumental performance flexibly in accordance with changes in the action-outcome contingency during early reversal learning, consistent with the claim that iSPNs are necessary to flexibly update instrumental learning (Matamales et al., 2020).

Prior research assessing the involvement of dSPNs and iSPNs in goal-directed learning has generally inferred a greater relative involvement of dSPNs based on the use of cell-type specific GFP expression in either D1-GFP (Maroteaux et al., 2014; Shan et al., 2014) or A2A/D2-GFP (Shan et al., 2014; Matamales et al., 2020) mice. We confirmed this involvement using the circuit-specific approach described here, both in the recruitment of Zif268 signalling, which was specifically increased in dSPNs and not iSPNs following instrumental training, and using chemogenetic inhibition, where we found causal evidence that dSPNs but not iSPNs are required for initial goal-directed learning. Furthermore, this effect of dSPN inhibition on goal-directed learning was specific to dSPNs and was not due to changes in the relative balance between the two populations of pDMS projection neurons; chemogenetic stimulation of iSPNs during the same period of training had no effect on goal-directed learning.

In a functional sense, previous studies conducting online assessments of instrumental conditioning have told us several things regarding the role of SPNs in instrumental performance in relation to reward history (e.g., Tai et al., 2012). However, what such assessments cannot determine is whether the actions being observed are in fact goal directed. In order to make that assertion, tests are required to demonstrate that subjects have encoded the relationship between actions and their consequences and are sensitive to the current value of the outcome (Balleine, 2019). Tests for outcome devaluation under extinction conditions provide prima facie evidence for both criteria; when animals bias responding away from a devalued lever, they demonstrate awareness of the action-outcome relationships, and the capacity to modulate their choice based on the current value of each outcome in the absence of any feedback to inform this choice (Balleine and Dickinson, 1998). Importantly, the current experiments established the necessary functional role of dSPNs (rather than iSPNs) in goal-directed learning for actions verified to be goal directed.

We have previously argued (Peak et al., 2019) that glutamatergic inputs to the dorsal striatum constitute ‘learning pathways’ in the broader basal ganglia circuitry, and striatal output pathways constitute ‘performance pathways’. Central to that argument is that the dorsal striatum constitutes an interface between these two functions and the current results refine this, suggesting that both goal-directed learning and performance rely specifically on dSPN input and output processes in the pDMS.

As has been reported previously, we found an instrumental training-induced increase in activity-related signalling in pDMS dSPNs. However, here we assessed the activity of each hemisphere separately relative to the position of the lever and found that this elevation was response specific; that is, it was lateralized with respect to the position of the manipulanda for the trained response. This is likely related to performance factors. Thus, for example, the activity of dopamine neurons projecting from the SNc to the DMS has been reported to be lateralized according to response type; when rats press a left lever, SNc-to-DMS dopamine neurons in the right hemisphere have been found to increase their activity independently of action value (Parker et al., 2016; Lee et al., 2019). These results predict that the lateralization of activity in dSPNs during goal-directed performance is driven by a lateralized dopamine input converging with the bilateral input from prefrontal cortex (Hart et al., 2018a; Hart et al., 2018b).

Although previous studies have reported an effect of iSPN inhibition on goal-directed performance (Carvalho Poyraz et al., 2016), there have been no studies assessing the role of dSPNs, and indeed, it was our prediction that this performance would be reliant on dSPN activity. Instead, we found that bilateral optogenetic inhibition, whether of dSPNs or iSPNs, had no effect on choice performance on test. Nevertheless, in the same animals, we found that unilateral optogenetic inhibition of dSPNs was effective in inducing an ipsilateral response bias (relative to the hemisphere of inhibition) in goal-directed responding, whereas unilateral iSPN inhibition had no effect. This effect of unilateral, but not bilateral inhibition of dSPNs on goal-directed performance emphasizes the importance of lateralized dSPN activity in inducing this response bias. By bilaterally dampening dSPN output, we did nothing to the relative balance of dSPN activity in each hemisphere and, as such, any existing response bias was unchanged. Lateralized control of motor output by dSPNs has been well demonstrated; unilateral activation or suppression biases contralateral or ipsilateral movements, respectively (Kravitz et al., 2010; Bay Kønig et al., 2019), whereas unilateral stimulation of dSPNs produces a contralateral bias in lever pressing (Tai et al., 2012). Our results indicate that these findings extend to goal-directed actions, consistent with the finding that DA terminals in the DMS respond more strongly during contralateral choices than ipsilateral choices and that this activity, too, precedes action performance (Parker et al., 2016; Lee et al., 2019). What appears to be critical here is the location of the lever relative to the magazine; lever pressing and magazine checking necessitate both ipsilateral and contralateral movements (relative to the hemisphere of recording, stimulation, or inhibition) and this is likely the source of the increased bias (Lee et al., 2019).

Nevertheless, the failure to find an effect of bilateral dSPN inhibition on performance was surprising. Previous studies have found that lesions or pharmacological manipulations of pDMS conducted after training are as effective as lesions made prior to training in abolishing the influence of outcome devaluation on performance (e.g., Yin et al., 2005a; Yin et al., 2005b; Shiflett et al., 2010). The current approach induced a far more specific and subtle reduction in neuronal output to either the SNr or GPe, and it is possible that unlike goal-directed learning, performance can be maintained with dramatically reduced (but not abolished) pathway function. Alternatively, such performance may rely instead on some other, as yet unspecified pathway. Indeed, such findings open up for consideration alternative ways in which the pDMS alters motor performance and the means by which striatal feedback circuits are integrated to that end, a task that awaits future research.

Despite the lack of involvement of iSPNs in learning, we found clear evidence that bilateral inhibition of iSPNs, but not dSPNs, in the pDMS impaired the flexible updating of the contingency controlling goal-directed performance in rats exposed to a series of reversals. A previous study using nose-poke reversals in mice reported effects of inhibition of both dSPNs and iSPNs on reversal, arguing, using a modelling analysis, that these effects reflected differential involvement of dSPNs in learning action values and of iSPNs in updating those values (Kwak and Jung, 2019). That study, however, used D1- and D2-Cre mice with hM4D-DREADDs to inactivate dSPNs and iSPNs infused into a central region of the dorsal striatum encompassing both medial and lateral regions. The dorsal striatum is highly segregated both in terms of its cortical inputs (Hunnicutt et al., 2016) and the functional role of its sub-regions in instrumental actions, with the most robust amongst these distinctions being the specific role of the pDMS, and not the more lateral (or indeed anterior) regions of the dorsal striatum, in goal-directed actions (Balleine, 2019). It is also unclear whether the reported effects were specific to iSPNs or were mediated by other D2-receptor expressing neurons, particularly various interneurons that also express D2 receptors. It is difficult, therefore, to apply the findings of this previous study to formulate predictions for our assessment. Here we found evidence that when iSPNs in the pDMS were inhibited, rats were slower to learn which of two levers was rewarded and to switch their responding appropriately. Importantly, this failure was overcome with continued training, suggesting that, with sufficient additional experience, rats were capable of learning response strategies to compensate for this impairment.

A lack of flexibility in this task could be due to several factors but, primarily, impairments can manifest as an inability to suppress responding on the non-rewarded lever (perseveration), or a failure to increase responding on the rewarded lever. In fact, we found that the impairment induced by iSPN inhibition was related to both of these factors and, indeed, there is recent evidence to suggest that both of these forms of response flexibility require iSPNs. For example, Matamales et al., 2020 demonstrated that ablating iSPNs in the DMS encouraged recurrent responding during the extinction of goal-directed learning (response perseveration), whereas Nonomura et al., 2018 reported that iSPNs in the DMS are sensitive to non-reward and, when optogenetically stimulated following a non-rewarded-response, promote switching to an alternate response.

Although, given prediction error analyses, it might be supposed that outcome omission forms the basis for iSPN involvement in these forms of reversal and extinction learning, Matamales et al., 2020 found similar involvement of iSPNs when animals were exposed to reversal of the identity of the outcomes of two otherwise continuously rewarded actions. It seems reasonable to conclude, therefore, that the causal evidence for the involvement of iSPNs in contingency reversal observed in the current study supports a role for iSPNs in updating the currently (non) rewarded action-outcome contingency. From this perspective iSPNs guide plasticity associated with changes in reward and non-reward so as to interleave new learning with prior encoding in a manner that minimizes interference between them.

Importantly, other evidence suggests that action-outcome updating is not solely dependent on dopamine-related signalling at iSPNs and relies on other sources of local modulation, most notably acetylcholine that is delivered locally via giant aspiny cholinergic interneurons. Cholinergic activity in the striatum is tightly bound to dopamine activity (Threlfell et al., 2012) and is critical for shaping goal-directed learning in the striatum (Bradfield et al., 2013; Matamales et al., 2016). Furthermore, recent evidence suggests that iSPNs and CINs interact to orchestrate striatal circuit function (Tanimura et al., 2019) under the control of inputs from the parafascicular thalamus via the thalamostriatal pathway (Bradfield et al., 2013; Tanimura et al., 2019). It is possible that both the Pf-pDMS pathway and iSPNs contribute to this in a serial manner; thalamic inputs generate burst-pause firing in pDMS CINs which evokes a prolonged postsynaptic excitability in iSPNs but not dSPNs, driven by the activation of M1 muscarinic receptors (Ding et al., 2010). Thus, thalamic activation of pDMS CINs creates a temporal window during which the striatal network is biased towards the activation of neighbouring iSPNs. Given this account, it should be anticipated that a loss in the function of either aspect of this local circuit will be sufficient to abolish the capacity of animals to update goal-directed learning.

The control of instrumental actions according to our current goals requires the differential recruitment of dSPN and iSPN pathways. By assessing each pathway according to projection target, rather than receptor expression, we were able to dissociate their functions in goal-directed actions along several lines. Broadly, dSPNs were found to be important for encoding goal-directed learning whereas goal-directed performance is reflected in, and modulated by, the asymmetrical activity of dSPNs in each hemisphere and the resulting drive of responses in contralateral action space. By contrast, when contingencies change, iSPNs are required to update those contingencies to support response flexibility.

Subjects for all experiments were Long-Evans rats, obtained either from the Animal Resource Centre (Perth, Western Australia) or from the Randwick Rat Breeding Facility (Randwick, New South Wales). All rats were healthy and experimentally naïve at the beginning of each experiment and at least 8 weeks old prior to surgery. Rats were housed in transparent, plastic boxes with two to four rats per box in a climate-controlled colony room and maintained on a 12 hr light/dark cycle (lights on at 07:00 hr). All experiments were conducted during the light phase. Water and standard lab chow were available ad libitum prior to the start of experiments. For all experiments, rats were randomly assigned to experimental groups and sample sizes were based on experiments previously conducted and reported (e.g., Hart et al., 2018b). Each experiment was conducted once and so the same cohort of animals was not tested multiple times (technical replication), nor was the same experiment repeated on multiple occasions (biological replication). All experiments conformed to the guidelines on the ethical use of animals maintained by the Australian code for the care and use of animals for scientific purposes, and all procedures were approved by the Animal Care and Ethics Committee at either the University of New South Wales or the University of Sydney.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

The research reported in this manuscript was supported by a Discovery Grant from the Australian Research Council, #DP150104878, and both a Project Grant, #GNT 1165346, and a Senior Principal Research Fellowship, #GNT1079561, from the National Health and Medical Research Council of Australia to BWB. The authors thank J Bertran-Gonzalez for comments on the manuscript.

Animal experimentation: All experiments conformed to the guidelines on the ethical use of animals maintained by the Australian code for the care and use of animals for scientific purposes, and all procedures were approved by the Animal Care and Ethics Committee at either the University of New South Wales (Protocol number 19/25A) or the University of Sydney (Protocol number 5960/78). All surgery was performed under isofluorane anesthesia, and every effort was made to minimize suffering.

1. Received: May 4, 2020
2. Accepted: November 19, 2020
3. Accepted Manuscript published: November 20, 2020 (version 1)
4. Version of Record published: December 1, 2020 (version 2)

© 2020, Peak et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.