1. Neuroscience

A corticostriatal deficit promotes temporal distortion of automatic action in ageing

  1. Miriam Matamales
  2. Zala Skrbis
  3. Matthew R Bailey
  4. Peter D Balsam
  5. Bernard W Balleine
  6. Jürgen Götz
  7. Jesus Bertran-Gonzalez  Is a corresponding author
  1. The University of Queensland, Australia
  2. University of New South Wales, Australia
  3. Columbia University, United States
https://doi.org/10.7554/eLife.29908
Share this article
Cite this article
  1. Miriam Matamales
  2. Zala Skrbis
  3. Matthew R Bailey
  4. Peter D Balsam
  5. Bernard W Balleine
  6. Jürgen Götz
  7. Jesus Bertran-Gonzalez
(2017)
A corticostriatal deficit promotes temporal distortion of automatic action in ageing
eLife 6:e29908.
https://doi.org/10.7554/eLife.29908
  2. Download BibTeX
  3. Download .RIS

Abstract

The acquisition of motor skills involves implementing action sequences that increase task efficiency while reducing cognitive loads. This learning capacity depends on specific cortico-basal ganglia circuits that are affected by normal ageing. Here, combining a series of novel behavioural tasks with extensive neuronal mapping and targeted cell manipulations in mice, we explored how ageing of cortico-basal ganglia networks alters the microstructure of action throughout sequence learning. We found that, after extended training, aged mice produced shorter actions and displayed squeezed automatic behaviours characterised by ultrafast oligomeric action chunks that correlated with deficient reorganisation of corticostriatal activity. Chemogenetic disruption of a striatal subcircuit in young mice reproduced age-related within-sequence features, and the introduction of an action-related feedback cue temporarily restored normal sequence structure in aged mice. Our results reveal static properties of aged cortico-basal ganglia networks that introduce temporal limits to action automaticity, something that can compromise procedural learning in ageing.

https://doi.org/10.7554/eLife.29908.001

Introduction

Learning of new skills permits optimal interactions with the environment while reducing cognitive costs, a fundamental adaptation contributing to behavioural autonomy and automaticity in many species. Motor skills are generally implemented through sequence learning, which allows elementary action units to be integrated into behavioural streams (Lashley, 1951), reducing latency and increasing speed of action (Sternberg et al., 1978). Similar to memory, movement sequences are thought to be organised into ‘motor chunks’, a cognitive-motor strategy proposed to depend on cortico-basal ganglia circuitry (Graybiel, 1998; Wymbs et al., 2012) that allows expression of whole behavioural programs as a single response (Abrahamse et al., 2013).

Although initial learning capabilities are often preserved (Voelcker-Rehage, 2008; Matamales et al., 2016a), motor skill capacity is reduced in older adults, especially in tasks involving fine motor control. Indeed, some studies report that aged humans have a near zero capacity to form action chunks in newly acquired sequences (Shea et al., 2006; Verwey, 2010), whereas others find evidence of shortened motor chunks (Bo et al., 2009). Sequence learning appears, therefore, to be impaired in ageing, although the acquisition of new skills may still be possible, perhaps through the recruitment of compensatory strategies (Cabeza et al., 2002; Park and Reuter-Lorenz, 2009). Nevertheless, despite increasing knowledge of the neural bases of action automatisation (Jin and Costa, 2015), the precise neuronal mechanism by which advanced ageing affects the acquisition of skills remains unknown.

To address this issue, we first designed behavioural tasks to determine which elements of action sequence learning were impaired by ageing. We then identified neuronal activation changes in the corticostriatal network that correlated with age-related learning defects, and reproduced some of these behavioural features through circuit-specific manipulations in young transgenic mice. Finally, we explored approaches to restore the ability of aged mice to perform action sequences through the introduction of an action-related feedback cue. Our findings suggest that the ageing brain may engage particular neuronal strategies to automatise action during skill learning, and that environmental support in the form of action-related feedback can be used to correct this behaviour.

Results

Aged mice display shorter patterns of action

We first compared the ability of young (2 months old) and aged (20–22 months old) mice to develop action sequences. We focused on homogeneous sequences involving the concatenation of single responses because this procedure is particularly susceptible to chunking (Garcia-Colera and Semjen, 1987; Jin and Costa, 2015). After mild food restriction, we trained animals using an instrumental procedure in which the number of lever presses (LP) required to obtain food outcomes increased as training progressed on a random ratio schedule of reinforcement (from constant reinforcement, CRF, to random ratio 20, RR20; Figure 1A). The formation of action sequences was promoted by introducing a sequence trigger (ST), in which caching of 5 or 7 contiguous LP was required to access the current RR program (see Figure 1—figure supplement 1A). We used a novel approach to monitor the formation of sequences by parsing behavioural data into three elements: initiation (first LP, or LP occurring after a magazine check), execution (LP occurring after a LP) and termination (magazine check occurring after LP) (Figure 1B). Throughout training, both young and aged mice significantly escalated their overall performance (LP rate) as the ratio requirement to obtain an outcome increased (mixed ANOVA using factors of training and age F(3.5,48.9) = 116.852, p<0.001). A non-significant training x age interaction (F(3.5,48.9) = 1.153, p=0.341) suggested that overall instrumental performance was equal in young and aged mice (Figure 1—figure supplement 1B). However, half way through this training, aged mice produced a larger number of action sequences per minute, whereas young mice stabilised, or diminished, their sequence rates (Figure 1C). This was supported by a significant effect of training (F(16,224)= 28.885, p<0.001) and a significant training x age interaction (F(16,224) = 3.716, p<0.001). Further, a comparison of action sequences between days 7 and 17 of training revealed that both young and aged mice were able to extend the number of execution elements in their sequences throughout training (F(1,14) = 158.282, p<0.001) (Figure 1D). This increase was of a different magnitude (as suggested by a significant day x age interaction (F(1,14) = 40.03, p<0.001)), indicating that aged mice extended their sequences to a lesser extent than young mice. Separate analysis confirmed, however, that, although at a different level, both young (F(1,7) = 108.562, p<0.001) and aged (F(1,7) = 106.664, p<0.001) groups showed significant increases in sequence length (Figure 1D). In contrast, analysis of sequence duration showed a significant day x age interaction (F(1,14) = 8.368, p<0.05) with significant increases of sequence duration in young (F(1,7) = 18.376, p<0.01) but not aged (F(1,7) = 1.745, p=0.228) mice (Figure 1E). Importantly, aged mice showed evidence of learning about the ST applied from day 10, as they displayed a marked increase in the number of sequences that broke through the ST and therefore frequently accessed the RR schedule (Figure 1—figure supplement 1C). In contrast, young mice kept these numbers low, suggesting that they comfortably incorporated the ST requirement (F(3.065,42.908) = 7.757, p<0.001; training x age interaction F(3.065,42.908) = 6.027, p<0.01) (Figure 1—figure supplement 1C). Event-time plots recorded in both groups on day seven revealed no sequence structure in their instrumental performance, despite all rewards being obtained by all of the mice (Figure 1—figure supplement 1D). In contrast, analysis on day 17 revealed that both young and aged mice developed clearly defined action sequences over time (with identifiable initiation, execution and termination elements), although these sequences were more frequent and much shorter in the aged group (Figure 1F).

Figure 1 with 2 supplements see all
Download asset Open asset
Evidence of sequence learning in aged mice.

[Figure 1—source data 1] (A) Experimental design of instrumental conditioning under an increasing random ratio (RR) schedule of reinforcement, from constant reinforcement (CRF) to RR20. Sequence triggering (ST) programs were introduced on day 10. (B) Initiation, execution and termination elements were identified in each sequence according to lever press (LP) and magazine entry interspersing. (C) Number of action sequences per minute (sequence rate) displayed throughout instrumental learning [StatsReport1]. (D, E) Average length (D) and duration (E) of LP sequences produced by each young and aged mouse on days 7 and 17 of training. Orange traces represent individual mice. Asterisks denote significant day x age interaction (red) and simple effects (black, see text). N.S., not significant [StatsReport2] [StatsReport3]. (F) Event-time diagrams showing linearly organised LP sequences produced by all young and aged mice on training day 17. Shaded segments (gray) are amplified LP sequences (seconds). See Figure 1—figure supplement 1D for pre-sequence diagrams.

https://doi.org/10.7554/eLife.29908.002
Figure 1—source data 1

Source data for Figures 1 and 3.

https://doi.org/10.7554/eLife.29908.005
Figure 1—source data 2

Source data for Figure 1.

https://doi.org/10.7554/eLife.29908.006
Figure 1—source data 3

Source data for Figure 1.

https://doi.org/10.7554/eLife.29908.007

We next ruled out that this effect was due to the inability of aged mice to hear when pellets were delivered (Figure 1—figure supplement 2). First, we confirmed that the delivery of pellets in the operant conditioning boxes generated a clear vibration signal, which was more salient than the sound of the dispenser engagement and/or pellet dropping (Figure 1—figure supplement 2A). We then exposed a group of young and aged mice to six days of magazine training, where 20 pellets were randomly delivered in ~30 min sessions (Figure 1—figure supplement 2B–C). We found that all mice reduced their reaction time between pellet delivery and first magazine check (first check intervals) as training progressed (mixed ANOVA with factors training and age F(5,70) = 25.822, p<0.001), indicating that they learned to detect when pellets were delivered. A non-significant training x age interaction (F(5,70) = 0.360, p=0.874) confirmed that this reduction was similar in both groups (Figure 1—figure supplement 2B and D). This similar performance was not due to a higher overall number of magazine checks in the aged group, since both young and aged mice showed a similar increase of magazine approach responses throughout this training (F(5,70) = 25.822, p<0.001; training x age interaction F(5,70) = 0.360, p=0.874) (Figure 1—figure supplement 2C). Overall, our results showed that aged mice produced a higher number of action sequences, and in contrast to previous studies in humans (Shea et al., 2006; Verwey, 2010), these sequences could be extended (in terms of action number) but not temporally sustained.

We thus explored whether sustained actions in aged mice were subject to temporal constraints, something that could explain the formation of shorter sequences of action. To address this, we used a Lever Hold (LH) instrumental procedure (Figure 2), a variant of the variable interval hold task (Bailey et al., 2015) in which the delivery of the reward depended on pseudorandom increments of LH time during training (Figure 2A and B). In this task, both young and aged groups displayed parallel escalations of LH times up to ~0.5 s. However, young mice increased their LH times throughout subsequent stages of training, whereas aged mice maintained their maximum performance around 0.5 s holds (Figure 2C). Statistical analysis supported this observation: mixed ANOVA (factors: training and age) showed that the time of lever hold significantly varied as instrumental training progressed in both age groups (F(16,176) = 73.471, p<0.001), although there was a significant training x age interaction, indicating that the age of the animals significantly affected their performance (F(16,176) = 16.796, p<0.001). Importantly, simple effects analyses confirmed that despite these differences, both groups independently elevated their performance throughout training (Young: F(16,112) = 69.387, p<0.001; Aged: F(16,64) = 51.33, p<0.001). Event scatter plots at different LH requirements revealed that, although both groups displayed a large number of very short presses, only young mice managed to produce sustained presses that approached, or exceeded, each LH requirement (Figure 2D). Event-time diagrams showed how all aged mice displayed much shorter holds than younger controls from the start of the session (Figure 2—figure supplement 1).

Figure 2 with 1 supplement see all
Download asset Open asset
Aged mice are unable to temporally sustain their actions.

[Figure 2—source data 1] [StatsReport4] (A) Mice were trained to press and hold the lever down for increasing periods to obtain rewards. (B) Experimental design of instrumental conditioning under an increasing pseudorandom lever hold (LH) schedule of reinforcement. Average LH times are indicated in seconds. (C) Acquisition of LH instrumental behaviour across training. Data are mean ±SEM (7 and 8 mice per group). Asterisk denotes significant training x age interaction. (D) Representation of each individual LH performed by all young and all aged mice on the last day of LH0.3, LH0.8, LH2, LH3.2 and LH5 training. LH times are presented in a Log10 scale (x axis), and sequences are distributed randomly in the y axis.

https://doi.org/10.7554/eLife.29908.008
Figure 2—source data 1

Source data for Figure 2.

https://doi.org/10.7554/eLife.29908.010

As such, these results provide evidence of action sequence formation in aged mice while also suggesting that these sequences could be influenced by temporal constraints, in line with previous results in humans (Bo et al., 2009).

Sequence learning leads to rapid response bursts and aberrant chunking in aged mice

We next investigated the forms of automaticity generated in both age groups during LP-based sequence learning. First, we analysed in our sequence timestamp dataset whether aged mice showed higher overall sequence speed. We found that average speed on the last day of training was higher in aged mice, although the effect fell below significance (t(14) = −2.104, p=0.054) (Figure 3A). However, subsequent analysis of the serial response times based on inter-press-intervals (IPIs) recorded on the same day revealed that aged mice produced a much higher proportion of very short IPIs (Figure 3B, Figure 3—figure supplement 1A), suggesting that they were reaching a higher sequence speed than young mice. Due to a resolution limit in the timestamp data (Figure 3—figure supplement 1B), we sought to establish the true maximum sequence speed by means of acoustic data, where the fastest bursts of LP in each mouse were identified by extracting the acoustic fingerprint of lever depression (Figure 3C, see Materials and methods). We found a strikingly high speed of responding in aged mice, in some instances displaying spates of LP with over 13 presses per second, much faster than the fastest bursts recorded in young mice (Figure 3C, Video 1 and Video 2). These speeds are similar to fully automatised scratch reflex movements in mice (Inagaki et al., 2003). Quantification analysis of fast sequences confirmed that average speed within fast bursts (in number of presses/sec) was significantly higher in aged than young mice (t(7.417) = −4.201, p<0.01, Figure 3D), whereas within-burst IPIs were significantly shorter (t(14) = 2.724, p<0.05, Figure 3E). Interestingly, despite their speed of responding, aged mice were less efficient at earning rewards (t(14) = 3.101, p<0.01), indicating that although their action bursts often broke through the sequence trigger (see Figure 1—figure supplement 1C), they rarely completed the RR requirements (Figure 3F).

Figure 3 with 2 supplements see all
Download asset Open asset
Aged mice produce ultrafast sequences and aberrant chunking.

[Figure 1—source data 1] [Figure 3—source data 3] (A) Average sequence speed calculated from timestamp data after 20 min of continuous instrumental performance on training day 18 [StatsReport33]. (B) Frequency distribution of the number of inter-press-intervals (IPIs) recorded in all young and aged mice at increasing intervals on training day 18. (C) Real IPIs from the fastest sequence recorded in a young and an aged mouse on day 18 (Fast sequence one in Video 1 and Video 2). Real lever press times were identified by aligning the sound waveform (top) and acoustic spectrogram (bottom) generated by lever press activity during the session. Identified presses are marked with a blue dashed line. (D, E) Average speed (B) and IPIs (C) of the five fastest LP sequences displayed by young and aged mice during the 20 min session on day 18 extracted from acoustic data [StatsReport5] [StatsReport6]. (F) Efficiency of LP instrumental behaviour (rewards earned per minute) in young and aged mice on day 18 [StatsReport7]. Asterisks (B–D) denote significant effects (see text). (G) Scatter plots of each within-sequence inter-press-interval (IPIs, blue) and sequence boundary interval (orange) produced by one young and one aged mouse (Young #7 and Aged #5) at different stages of training. Kernel density curves of each interval type are plotted at the bottom. Grey shades delimitate the within-sequence chunking space. (H, I) Average within-sequence IPIs (G) and sequence boundary (H) intervals. Insets are corresponding standard deviations. Data are mean ±SEM. Asterisk denotes significant training x age interaction [StatsReport27] [StatsReport28] [StatsReport31] [StatsReport32]. (J, K) Proportional scatter plots and kernel density curves showing intervals of the within-sequence (I) and sequence boundary (J) elements produced by all young and all aged mice during the first 600 s of training day 18. Insets are the total element counts for this period [StatsReport29] [StatsReport30]. Data in A, D-F, J and K are mean +SEM. p, p-value; asterisk, significant effect; N.S., not significant.

https://doi.org/10.7554/eLife.29908.011
Figure 3—source data 1

Source data for Figure 3.

https://doi.org/10.7554/eLife.29908.014
Video 1
Download asset
Representative performance of a young mouse during the last day of sequence training.

Two fast action sequences are captured (indicated in red font, top left corner). Fast action sequence one is represented in Figure 3A. Subject Young#5; active lever: right; training day 18.

https://doi.org/10.7554/eLife.29908.015
Video 2
Download asset
Representative performance of an aged mouse during the last day of sequence training.

Two fast action sequences are captured (indicated in red font, top right corner). Fast action sequence one is represented in Figure 3A. Subject Aged#4; active lever: left; training day 18.

https://doi.org/10.7554/eLife.29908.016

We next sought to analyse whether different chunking patterns emerged in young and aged mice during sequence learning by studying the temporal relationships between consecutive action elements throughout training (Figure 3G–K). We categorised the action repertoire into two classes of interval: the within-sequence interval (IPIs that reside within each sequence) and the sequence boundary interval (a combination of press-check and check-press intervals that lay in-between sequences) (see methods). The analysis of the distribution of these intervals at different stages of training provided a visual readout of the chunking that emerged during sequence development (Figure 3G). In both young and aged mice, within-sequence IPIs (Figure 3G, blue) clustered around a similar interval space (from ~0.1 to ~1 s intervals) by day 13, suggesting that both groups developed chunking of within-sequence (i.e. execution) elements. In support of this, we found that both young and aged mice similarly reduced the within-sequence IPIs throughout training (Figure 3H), as reflected by a significant effect of training (two-way mixed ANOVA with factors training and age: F(1.955,27.377) = 18.606, p<0.001) but no training x age interaction (F(1.955,27.377) = 0.562, p=0.573). Similarly, the variability of these IPIs was equally reduced in both groups (Figure 3H, inset), again showing a significant effect of training (F(2.352,32.904) = 6.545, p=0.003) with no training x age interaction (F(2.352,32.904) = 1.785, p=0.179). On the other hand, the development of sequence boundary intervals throughout training (Figure 3G, orange) was very different in young and aged mice. Whereas young mice displayed longer boundary intervals that were distant from the chunking space, aged mice tended to accumulate them in a narrow band near the chunking space (Figure 3G). Importantly, this band appeared earlier in training (day 10) in all aged animals, before within-sequence intervals chunked (Figure 3G, Figure 3—figure supplement 2A and B). This was evidenced in the quantitative analysis, where the young group showed an increase in sequence boundary intervals from days 10 to 17, whereas the aged group kept these intervals consistently low (Figure 3I), as indicated by a significant effect of training (F(3.884,54.375) = 8.005, p<0.001) and a significant training x age interaction (F(3.884,54.375) = 4.729, p=0.003). Although no drop in variability for this interval was found from day 10 to 17 (F(7,98) = 1.428, p=0.203), aged mice displayed lower variability than young mice in this period (F(1,14) = 8.855, p=0.01) (Figure 3I, inset), suggesting that the temporal relationships of boundary elements in aged mice had already been established in day 10 (see Figure 3—figure supplement 2). In order to obtain proportional measures of chunking in both groups, we compared the within-sequence and sequence boundary intervals produced by all young and all aged mice during the first 600 s of continuous action sequence behaviour (Figure 3J–K). Again, we found in aged mice a larger proportion of very short within-sequence IPIs (Figure 3J), although the overall number of within-sequence presses was equal across groups (t(14) = 1.692, p=0.113) (Figure 3J, inset). The sequence boundary intervals produced during this period were also shorter in aged mice (Figure 3K), and the larger number of boundary elements in this group (t(7.341) = −4.248, p=0.003) (Figure 3K, inset) reflected the higher number of action sequences produced by aged mice per minute (see Figure 1C). Therefore, we found clearly different action automatisation processes shaping within-sequence and sequence boundary elements in young and aged mice. While young mice appeared to exclusively chunk the elements within the sequences, aged mice displayed ultrafast chunking of within-sequence elements, and showed evidence of early chunking in sequence boundary elements. Together, these results suggest that early chunking of the extra sequence elements in aged mice could cleave behaviour into ultrafast microchunks, revealing previously unrecognised deficits in automatic action in ageing.

Sequence features correlate with defective development of corticostriatal activity in aged mice

The development of action sequences and stimulus-response habits is attributed to mediolateral shifts of activity in cortico-basal ganglia circuits (Balleine, 2005; Yin et al., 2009; Balleine and O'Doherty, 2010), likely following a medial-to-lateral disposition of functionally relevant corticostriatal projections in rodents (Hunnicutt et al., 2016; Voorn et al., 2004). Here, we studied the state of activation of the corticostriatal network at different rostrocaudal levels in young and aged mice expressing uninterrupted action sequences for 20 min (Figure 4). We identified over 60,000 activated neurons across cortical and striatal regions in sixteen mice using a confocal imaging-computational approach based on the cyto-nuclear detection and mapping of phosphorylated MAPK ERK1/2 (p-MAPK) (Figure 4A and B), a marker that is widely used to study neuronal activity throughout the brain (Valjent et al., 2004). We first contrasted the extent of activation detected in cortical and striatal regions in young and aged mice (Figure 4C). Two-way ANOVA (factors: age; region) revealed no overall effect of age (F(1, 111)=1.868, p=0.174), although there was a significant age x region interaction (F(7,111) = 2.136, p<0.05). Simple effect analyses (factor: age) showed that this interaction was primarily driven by differences in the anterior cingulate/motor area 2 (aCg/M2; F(1,14) = 4.912, p<0.05) and the frontoparietal regions of the cortex (FPx: F(1,14) = 15.364, p<0.01), as well as in their target territories of the dorsolateral striatum (DLS: F(1,13) = 4.753, p<0.05) (Ebrahimi et al., 1992; McGeorge and Faull, 1989). We next used this extensive dataset to identify the corticostriatal regions that were primarily related to action sequence execution. We conducted a correlation study by contrasting the corticostriatal activity profile based on p-MAPK activation in different cortical and striatal areas in each mouse with the different features of sequence performance. We found that sequence length, duration and speed of action displayed on the last day of training could only be predicted by activation in DLS and FPx, whereas activation in all other areas did not significantly correlate with any feature of sequence execution (Figure 4D). We then investigated the relative functional engagement of different territories across the corticostriatal network in each age group by performing a regional cross-correlation analysis in which the mean level of p-MAPK activation in each region was correlated with all other regions across animals (Figure 4E). Young mice showed very low levels of correlation overall, with no significant relationships between cortical and striatal territories throughout the rostrocaudal extent of the network, perhaps due to automaticity-related reductions in neuronal variability and/or processing loads (Santos et al., 2015; Dayan and Daw, 2008) (Figure 4E, left panel). In contrast, aged mice showed widespread correlations that were significant in different regions of the corticostriatal network, especially the rostromedial, but not caudolateral, territories (Figure 4E, right panel). In order to obtain further regional information on this effect, we compared the spatial distribution of activated neurons in the corticostriatal network by mapping the position of individual neurons in each territory of sections from young and aged mice (Matamales et al., 2016b) (Figure 4F). Our data in aged mice revealed large regions within the DLS and deep layers of the FPx that were devoid of neuronal activation (Figure 4F). Overall, these results suggested an age-related deficit in the posterior corticostriatal network during action sequence execution that principally involved functional impairments in dorsolateral striatum and frontoparietal cortex, the two regions primarily implicated in motor chunking (Wymbs et al., 2012).

Figure 4
Download asset Open asset
Action sequence features correlate with defective activation of caudolateral corticostriatal networks in aged mice.

[Figure 4—source data 1] (A) Extensive analysis of neuronal activation in different rostrocaudal cortical (prelimbic [PrL], anterior cingulate/motor area 2 [aCg/M2], frontoparietal [FPx]) and striatal (anterodorsal [aDS], core, shell, dorsomedial [DMS], dorsolateral [DLS]) areas after 20 min of uninterrupted instrumental performance (day 18). (B) Position and intensity of each activated neuron (p-Thr202-Tyr204-MAPK-immunoreactive [p-MAPK]) were detected through semi-automated quantitative fluorescence (~60,000 neurons analysed). (C) Horizontal floating bar chart displaying the quantification of p-MAPK counts in different cortical and striatal areas of young and aged mice. Minimum, maximum and mean of the data are represented (vertical lines). Circles indicate individual counts for each animal (eight mice per group). Asterisks denote significant age x region interaction (red) and simple effects (black, see text) [StatsReport9]. (D) Activity – performance correlation study comparing the counts of p-MAPK detected in the different cortical and striatal areas (arranged from rostro-medial to caudolateral) and the action sequence features (length, duration and speed) as well as overall instrumental performance (total number of lever presses) displayed by each young and aged mouse during the last day of training. The corresponding p value matrix (bottom inset) indicates the significance of each correlation. Pearson r (multicolor scale) and p (green scale) value diagrams are shown for each correlation [StatsReport10]. (E) Cross-correlation analysis of the p-MAPK activity recorded in the different areas (arranged from rostromedial to caudolateral) in all young (left) and aged (right) mice. Multi-colored diagrams show the extent of correlation between each pair of regions (Pearson r values), and green diagrams (right) indicate their significance at each position (p values) [StatsReport11]. Correlations in D and E involve within-subjects measures. (F) Digitised reconstructions of the distribution of activated neurons in corticostriatal regions at rostral (left), medial (middle) and caudal (right) levels in one young and one aged mouse. Each dot represents the position of one neuron expressing p-MAPK.

https://doi.org/10.7554/eLife.29908.017
Figure 4—source data 1

Source data for Figure 4.

https://doi.org/10.7554/eLife.29908.018

Chemogenetic inhibition of direct pathway projection neurons in dorsolateral striatum increases speed of action sequences

Striatal output circuits are organised into direct and indirect pathway neurons (dSPNs and iSPNs, respectively), two parallel projection systems that orchestrate functions in the basal ganglia (Gerfen and Surmeier, 2011). Recent studies have suggested that action duration could be influenced by dSPNs in the DLS based on their sustained firing during the time of sequence execution (Jin et al., 2014), and the prolongation of an ongoing action by optogenetic stimulation (Tecuapetla et al., 2016). Capitalising on our p-MAPK mapping results, and based on findings that this intracellular activity marker is more sensitive in dSPNs than iSPNs (Bertran-Gonzalez et al., 2009), we hypothesised that age-related hypoactivation of dSPNs in the DLS could be contributing, at least in part, to deficits in action sequence learning observed in aged mice. With the aim of reproducing the molecular deficit observed in DLS striatal neurons of aged mice throughout sequence learning, we used M4-based chemogenetics (Gαi) to prevent intracellular signalling in dSPNs of the DLS in young transgenic mice at the time of sequence implementation (i.e. day 10 onwards). Drd1a-Cre:Drd2-eGFP double transgenic mice were given a unilateral intra-DLS infusion of an AAV vector containing the Gαi-hM4Di DREADD with Cre specificity coupled to the reporter mCherry, a procedure that allowed manipulation and visualisation of dSPNs and iSPNs, respectively (Figure 5—figure supplement 1). In order to test the validity of this system, we combined the injection of CNO (3 mg/kg, i.p.) with a delayed injection of GBR12783 (15 mg/kg, i.p.), a specific dopamine reuptake blocker known to induce p-MAPK in dSPNs (Valjent et al., 2010) (Figure 5—figure supplement 1B). We found that hM4Di-mCherry expression was contained, within the DLS, to eGFP-negative neurons, confirming specific infection of dSPNs in the DLS (Figure 5—figure supplement 1C,D). Importantly, dSPN-hM4Di expression and subsequent activation by CNO prevented GBR-induced MAPK phosphorylation when compared to the non-injected hemisphere of the same animals, ensuring that this system prevents intracellular activation (Figure 5—Figure 1D). Quantification of p-MAPK +neurons confirmed this effect (Figure 5—figure supplement E): paired comparisons revealed a pronounced decrease of p-MAPK +neuron density (t(5) = −15.565, p<0.001) in equivalent areas of infected and noninfected hemispheres (t(5) = −1.222, p=0.276).

Figure 5 with 2 supplements see all
Download asset Open asset
Chemogenetic inhibition of dorsolateral direct pathway neurons increases within-sequence speed.

[Figure 5—source data 1] A cohort of 7 Drd1a-Cre(−):Drd2-eGFP and 7 Drd1a-Cre(+):Drd2-eGFP young mice (dSPN-hM4Di(−) and dSPN-hM4Di(+)) were bilaterally injected with AAV2-hSyn-DIO-hM4Di-mCherry in the DLS. (A) Diagram shows the overlapping spread of viral infection in all dSPN-hM4Di(+) mice throughout the DLS as visualised by mCherry-expressing territories. Blue dots represent mapped p-MAPK-positive neurons in the same sections. (B) Upon recovery, all mice were submitted to LP sequence training with daily CNO injections from day 10 (3 mg/kg, 45 min before each session). (C) Number of action sequences per minute (sequence rate) displayed throughout instrumental learning. Data are mean ±SEM. Asterisk denotes significant between subjects effect across CNO period (see text) [StatsReport37]. (D, E) Average length (D) and duration (E) of LP sequences produced by each mouse on days 10 and 17 of training. Orange traces represent individual mice. Asterisks denote significant day x group interaction (red) and simple effects (black, see text) [StatsReport13] [StatsReport14]. (F) Average speed of the LP sequences displayed by dSPN-hM4Di(−) and dSPN-hM4Di(+) on day 18 (timestamp data) [StatsReport15]. (G) Frequency distribution of the number of inter-press-intervals (IPIs) recorded in all dSPN-hM4Di(−) and dSPN-hM4Di(+) mice at increasing intervals on training day 18. (H, I) Scatter plots and kernel density curves showing intervals of the within-sequence (H) and sequence boundary (I) elements produced by all dSPN-hM4Di(−) and dSPN-hM4Di(+) on training day 18. Insets are the total element counts for this period [StatsReport35]. (I, J) Average within-sequence IPIs (I) and sequence boundary (J) intervals on day 18. Data are mean +SEM [StatsReport16] [StatsReport36]. (K) Efficiency of LP instrumental behaviour (rewards earned per minute) in by dSPN-hM4Di(−) and dSPN-hM4Di(+) mice on day 18 [StatsReport17]. Asterisks denote significant effects (see text). N.S., not significant.

https://doi.org/10.7554/eLife.29908.019
Figure 5—source data 1

Source data for Figure 5.

https://doi.org/10.7554/eLife.29908.022
Figure 5—source data 2

Source data for Figure 5.

https://doi.org/10.7554/eLife.29908.023

Based on the territories found to be devoid of p-MAPK +neurons in aged mice (see Figure 4F), we induced bilateral transduction of hM4Di DREADD into the DLS in a new cohort of young Drd1a-Cre+:Drd2-eGFP+ (dSPN-hM4Di(+)) mice and their littermate controls (Drd1a-Cre−:Drd2-eGFP+; dSPN-hM4Di(−)) before LP sequence training (Figure 5A). From day 10 onwards, all mice received a 3 mg/kg CNO injection (i.p.) 45 min prior to the start of each session (Figure 5B). A two-way mixed ANOVA (factors: training; genotype) showed that the CNO treatment did not affect the ability of mice to acquire instrumental responses, as the rate of LP significantly increased as training progressed in both groups (F(16,192) = 6.389, p<0.001) (Figure 5—figure supplement 1F). Importantly, this treatment did not cause a drop in overall performance (Figure 5—figure supplement 1F, compare days 9 and 10), which equivalently increased in both groups across training (training x genotype interaction: F(16,192) = 0.601, p=0.881). This suggested that 3 mg/kg of systemic CNO did not cause detectable behavioural side effects.

We studied in these mice whether differences in sequence learning emerged throughout training, similar to the effects observed in aged mice (see Figure 1). We found that dSPN-hM4Di(−) and dSPN-hM4Di(+) mice produced a similar number of sequences per minute up to day 9, but hM4Di(+) littermates tended to display higher sequence rates once the CNO treatment started (Figure 5C), as supported by a significant between subjects effect in a two-way mixed ANOVA applied only on CNO days (F(1,12) = 6.380, p=0.27). Importantly, both groups showed an equal increase in the number of presses per sequence from day 10 (first day of CNO) to day 17 (Figure 5D). Accordingly, a two-way mixed ANOVA showed a significant increase in sequence length in all mice (F(1,12) = 91.036, p<0.001), without a significant day x group interaction (F(1,14) = 19.724, p=0.218). This was further supported by simple effects (factor: day): both dSPN-hM4Di(−) and dSPN-hM4D(+) mice significantly increased their sequence lengths (F(1,6) = 59.546, p<0.001; and F(1,6) = 33.521, p<0.01; respectively), indicating that both groups were equally capable of extending (in terms of number of elements) their action sequences. In contrast, we found that only dSPN-hM4Di(−) controls increased the duration of their sequences from day 10 to 17, whereas dSPN-hM4Di(+) littermates displayed the same short latencies throughout (Figure 5E), as supported by a significant day x genotype interaction (F(1,12) = 5.087, p<0.05). Simple effects confirmed a significant extension of action latency in dSPN-hM4Di(−) (F(1,6) = 30.545, p<0.01) but not dSPN-hM4Di(+) mice (F(1,7) = 1.624, p=0.250). Event-time plots recorded in both groups on day 17 revealed that both groups developed clearly defined action sequences (with identifiable initiation, execution and termination elements), although these sequences tended to be more frequent and shorter in dSPN-hM4Di(+) littermates (Figure 5—figure supplement 2A–B).

We next sought to compare the patterns of action chunking displayed by both groups on training day 18, similar to our previous study in young and aged mice (see Figure 3). We found that dSPN-hM4Di(+) mice produced, overall, faster action sequences (t(12) = −2.3841, p=0.035) (Figure 5F), and also displayed a higher proportion of short IPIs (Figure 5G). This was similar to the effect observed in aged mice (see Figure 3A–B), although peak speeds within fast sequence bursts in dSPN-hM4Di(+) were not as high as the ones recorded in aged mice. Temporal analysis of sequence elements on day 18 revealed that dSPN-hM4Di(+) mice presented a shifted distribution of within-sequence elements in the interval space (Figure 5H) and, consequently, displayed significantly shorter average IPIs (t(12) = 2.964, p=0.012) (Figure 5J), which was a similar trend to that observed in aged mice (see Figure 3J). However, we found no group differences in the distribution (Figure 5I) and average value (Figure 3K) of sequence boundary intervals (t(12) = 1.435, p=0.177), which was in contrast to the pattern observed in aged mice (see Figure 3K). Altogether, these findings suggest that interrupting intracellular signalling in dSPNs of the DLS during periods of sequence development can increase the speed of individual action sequences, although the number of elements fitted within each sequence as well as the extra sequence intervals remained unaltered. Therefore, this manipulation reproduced within-sequence but not extra-sequence features of action chunking in ageing, a pattern that could explain the increased—rather than reduced—ability of dSPN-hM4Di(+) mice to obtain rewards during ST training (t(12) = −2.4, p=0.033) (Figure 5L).

Action-related feedback transitorily restores sequence structure in aged mice

A possible source of the behavioural deficit we observed in aged mice is a loss of action-related feedback; the use of environmental feedback in the form of action-associated cues has been reported to improve self-initiated performance in older adults (Lindenberger and Mayr, 2014). We, therefore, attempted to normalise the behavioural patterns of aged mice by providing action-related feedback in the form of an external cue. We submitted two groups of aged mice to LP sequence training, both of them receiving a sound cue (0.75 s white noise) that signalled the last LP of each sequence preceding the delivery of reward. For one of the groups the feedback cue was given for the duration of the experiment (group Cue Maintained, days 1–36), whereas for the other group cue presentation stopped on day 19 (group Cue Lost) (Figure 6A). This treatment had no apparent effects on the overall performance of either group: two-way mixed ANOVA (factors: training; cue) showed that all mice escalated their instrumental responses throughout the duration of the experiment (F(35,420) = 72.869, p<0.001), and there was no significant training x cue interaction (F(35,420) = 1.378, p=0.078) (Figure 6B). We then compared the sequence features displayed on days 7, 18 and 19, the latter being the first day of cue withdrawal for the Cue Lost group (Figure 6C,D). Two-way mixed ANOVA (factors: day; cue) showed significant differences in sequence length on days 7, 18 and 19 of training (F(2,26) = 199.993, p<0.001). However, there was a significant day x cue interaction (F(2,26) = 6.567, p<0.01), suggesting that both groups performed differently across these days. Simple effects (factor: day) confirmed that the sequence length dropped significantly from day 18 to day 19 in the Cue Lost group (F(1,8) = 15.696, p<0.01), whereas it remained constant in the Cue Maintained group (F(1,5) = 0.851, p=0.399) (Figure 6C). This effect was more pronounced on sequence duration: we found significant differences in the duration of the sequences between days 7, 18 and 19 (F(2,26) = 20.643, p<0.001), and again a significant day x cue interaction (F(2,26) = 6.256, p<0.01) with simple effects confirming that, upon cue removal, the duration of action sequences dropped to pre-sequence levels in group Cue Lost (F(1,8) = 21.432, p<0.01) but not in group Cue Maintained (F(1,5) = 0.996, p=0.364) (Figure 6D). These results suggest that the execution of extended action sequences in aged mice depends on the provision of action-related feedback instructing the duration of action.

Figure 6 with 1 supplement see all
Download asset Open asset
Cues signalling sequence termination temporarily normalise action structure in aged mice.

[Figure 6—source data 1] (A) Aged mice were submitted to extended LP sequence training where the termination element in each sequence was cued with a 0.75 s white noise. One group maintained the termination cue until training day 36 (Cue Maintained), whereas in the other group (Cue Lost) the cue was removed from day 19. (B) Resulting acquisition of LP instrumental behaviour. Data are mean ±SEM (6–8 mice per group). Asterisk denotes significant overall effect of training (see text) [StatsReport18]. (C, D) Average length (C) and duration (D) of the LP sequences produced by both groups on days 7, 18 and 19 of training. Asterisks denote significant day x group interaction (red) and simple effects (black, see text). N.S., not significant [StatsReport19] [StatsReport20]. (E) Structure of LP sequences produced by a ‘Cue Maintained’ and a ‘Cue Lost’ mouse at the start and at the end of the session on training day 19. See also Figure 6—figure supplement 1. (F) Average length of the 10 first sequences and the 10 last sequences produced by ‘Cue Lost’ mice on training day 19. Asterisks denote significant sequence x start/end period interaction (red) and simple effects (black, see text). N.S., not significant [StatsReport21]. (G) Total number of action sequences produced in each day of training before and after the loss of the termination cue. Asterisks denote significant overall effect of training (black) and training x group interaction (red, see text) [StatsReport22]. (H) Scatter plots of each within-sequence inter-press-interval (IPIs, blue) and sequence boundary interval (orange) produced by one ‘Cue Maintained’ (subject #R8_M) and one ‘Cue Lost’ (subject #B4_L) mouse across different days of training. Kernel density curves of each interval type are plotted at the bottom. Grey shades delimitate the within-sequence chunking space.

https://doi.org/10.7554/eLife.29908.024
Figure 6—source data 1

Source data for Figure 6.

https://doi.org/10.7554/eLife.29908.026

We next investigated in these mice the transition from long to short sequences on cue withdrawal (day 19). Event-time diagrams representing the sequences executed at the beginning and at the end of the training session showed that all mice in group Cue Lost started by executing long action sequences even in the absence of the auditory cue, but that these sequences rapidly shortened throughout the session (Figure 6E,F; Figure 6—figure supplement 1A,B). Accordingly, two-way mixed ANOVA (factors: sequence number; period) showed overall significant differences in the length of the sequences as the session progressed (F(9,144) = 3.635, p<0.001). However, there was a significant sequence number x period interaction (F(9,144) = 218,993, p<0.01), indicating that the length was significantly different in the Start and End periods of the session. Simple effects analysis (factor: sequence number) revealed that the length of the sequences progressively decreased at the start of the session (F(9,72) = 5.089, p<0.001), but remained constant towards the end (F(9,72) = 0.921, p=0.512) (Figure 6F). Importantly, the shortening of action sequences precipitated by cue removal was translated into a dramatic increase in the total number of sequences expressed by the Cue Lost mice that persisted throughout training (Figure 6G), as indicated by a significant day x cue interaction (F(35,420) = 2.339, p<0.001).

Finally, we assessed the action chunking patterns across training in both groups of aged mice and how these patterns changed with cue removal (Figure 6H). We found that, despite the presence of the feedback cue, both groups of mice displayed similarly aberrant automaticity on day 10, with boundary intervals accumulating in a narrow band nearby the chunking space (Figure 6H, day 10). However, mice seemed to learn to categorise their sequence elements with further cued training as the boundary interval band disappeared in both groups (Figure 6H, Day 18). Strikingly, group Cue Lost precipitated microchunks as soon as the instructive cue was removed (Figure 6H, compare days 18 and 22). Taken together, these results show that aged mice can execute longer action sequences as long as action-related feedback is provided regarding sequence duration, suggesting that this deficit is cognitive-motor rather than purely executional in origin.

Discussion

In the present study, we combined a series behavioural designs with comprehensive neuronal mapping and targeted circuit manipulation to reveal a fundamental deficit in the development of action sequences in mice of advanced age (20–22 m.o., average life expectancy is 24 months). It is well known that certain forms of motor skill learning are impaired in older adults (Voelcker-Rehage, 2008). In procedural learning, studies in humans demonstrate that both early (65-68) and late elderly (75-88) show very limited use of motor chunking during multi-element sequence tasks (Shea et al., 2006; Verwey, 2010). Although the precise patterns of action chunking differ between homogenous and heterogeneous sequence learning (i.e. sequences involving one vs. more than one action element) (Garcia-Colera and Semjen, 1987; Sternberg et al., 1978), our results in mice pressing a single lever revealed that older subjects consistently developed a form of chunking that involved shorter and faster sequences (microchunks, see Video 2). Similarly, using a serial response time task in humans, in which the formation of chunks was encouraged using visuospatial cues, Bo et al. (2009) found that older adults developed motor chunks overall, although the length of those chunks was reduced. Taken together, these results suggest that the automation of actions is possible in the aged, but is temporally distorted compared to that expressed by younger individuals.

Over and above the task-efficiency benefits of well-developed skills, automated or model-free behaviour has advantages for computational and space complexity loads; operationally, model-free reinforcement learning systems require less memory than model-based systems (Schneider and Shiffrin, 1977; Dayan and Daw, 2008). One important implication of this is that implementing model-free operations is more appropriate when computational resources such as working memory are limited (Otto et al., 2013). On this account, the computational resources of the aged brain can be limited by its clearly reduced working memory capacity (Bo et al., 2009; Wang et al., 2011; Wingfield et al., 1988). Therefore, in light of these memory caching limits, implementing model-free or automatic processes early in training can be seen as a successful strategy. Although this perspective can tentatively explain the formation of premature, unbreakable microchunks during early stages of training in aged mice, there are two features of our data that could argue against it. First, model-based to model-free transition (i.e., goal-directed to habitual modes of action) is thought to rely on shifts from dorsomedial to dorsolateral corticostriatal networks (Balleine, 2005; Yin et al., 2009). In the particular case of sequence learning and action chunking, engagements of posterolateral corticostriatal circuits have been demonstrated in humans and animals that are producing clearly automatised instrumental behaviours. Wymbs et al. (2012) found in humans that activity in the frontoparietal cortex and sensorimotor putamen (equivalent to the posterior dorsolateral striatum in rodents) strongly predicts critical features of action sequence execution, and studies in mice have provided information on the specific circuitry within posterior dorsolateral areas involved in action sequence initiation, execution and termination (Jin and Costa, 2010; Jin et al., 2014; Tecuapetla et al., 2016). Accordingly, a premature implementation of model-free processing (such as observed in microchunking) should predict an early medio-lateral shift of activity in corticostriatal networks. However, our neuronal activity assessments in aged mice suggest the contrary: aged brains seem to fail to disengage rostro-medial corticostriatal circuits, and shorter and faster sequences predict lower activity in dorsolateral regions of the striatum (Figure 4). Similarly, preventing such a shift in young mice by disrupting the activity of a subset of neurons in the dorsolateral striatum early in training reproduced some aspects of the aged behaviour, including the shorter latencies and higher speeds (see Figure 5).

Secondly, the formation of premature microchunks is not consistent with the results of our last experiment, where similar oligomeric sequences were instantly precipitated by the loss of an action-related feedback cue (Figure 6). In this case, aged mice expressed long, uninterrupted sequences from the beginning of training, and never displayed the constricted sequences prior to the loss of the cue. These results suggest that the execution of microchunks is not a consequence of deficient learning, but rather the result of different automaticity processes put in place by the aged brain.

Rather, our study supports the notion that young and aged brains start from a different neuronal environment or scaffolding (Park and Reuter-Lorenz, 2009), and attempt to optimise instrumental task performance by implementing entirely different strategies for automaticity. The critical question is: what are the features of the aged brain that produce these shorter patterns of action? Our data suggest that the dynamic medial-to-lateral shifts that are predicted to occur in the corticostriatal network during the development of automatic behaviour (Yin et al., 2009) may not happen, or may only occur partially, and this neuronal staticity might be key to explaining the recruitment of unusual automaticity processes in aged mice. Indeed, several functional imaging studies in humans suggest a similar shift in activity to anterior regions of the brain in elderly subjects (see Park and Reuter-Lorenz, 2009) for an extensive review): a compensatory overactivation in declining prefrontal cortical regions drives the use of different behavioural strategies to those of younger adults in certain tasks (Cabeza et al., 2002; Gutchess et al., 2005; Raz et al., 2005). This can be regarded as one possible mechanism for the marked differences in motor chunk development in aged humans during sequence learning (Bo et al., 2009; Verwey, 2010; Wymbs et al., 2012).

Nevertheless, why an anterior shift in corticostriatal activity with ageing generates such constricted spurts of action during sequence execution remains unresolved. Again capitalising on age-related working memory deficits and prefrontal cortex dysfunction (Wang et al., 2011; Wingfield et al., 1988), one could speculate that the overactivity observed in prefrontal areas in aged individuals reflects failed attempts to boost working memory capacity in distal parietal cortical regions, following a similar process to that proposed in computational models of visuospatial working memory retention (Edin et al., 2009). Interestingly, our data reveals specific activation reductions in dorsomedial prefrontal cortex as well as frontoparietal cortex, regions likely to be part of the prefronto-parietal network for top-down control of working memory capacity (Edin et al., 2009; Gruber and Goschke, 2004). Certainly, these prefrontal deficits could contribute to part of the problem, although the question of how working memory contributes to action organisation remains largely unresolved and is complex, likely involving subcortical basal ganglia circuitry (O'Reilly and Frank, 2006).

Another, perhaps related, neuronal process that could in part explain the microchunk behaviour in our study is malfunction in the prefronto-basal ganglia network and related deficits in temporal control with ageing (Gunstad et al., 2006). One interesting possibility is that the aged brain is subject to constraints during the temporal scaling of sequence processing in corticostriatal networks (Fukai, 1999; Igarashi et al., 2013). Although the precise mechanism by which time is encoded in the brain is unknown, recent evidence clearly implicates subcortical networks (dorsal striatum) as a key structure in scaling time for action (Mello et al., 2015; Soares et al., 2016). In our study, we found that selectively reducing the activity of direct pathway spiny projection neurons (dSPNs) in dorsolateral regions of the striatum reduced action duration while increasing action speed. These results directly support and extend recent findings on the involvement of dSPNs in encoding of the latency of action. Jin et al. (2014) found that a large proportion of dSPNs recorded in the dorsolateral striatum showed sustained patterns of firing that spanned the duration of an action sequence. More recently, the same group found that optogenetic stimulation of these neurons in the same striatal territory extended the length and duration of ongoing action sequences only during periods of stimulation (Tecuapetla et al., 2016). It should be emphasised, however, that although our striatal manipulations may have recapitulated some of the microchunking features observed in aged mice, we did not attempt to recreate accompanying prefronto-parietal dysfunction (and the resulting working memory deficit). This might explain why our young mice with the striatal manipulation were still capable of building up a reasonable action structure, something that unexpectedly rendered the animals more efficient at obtaining outcomes (Figure 5). Thus, our results add to studies elucidating the microcircuitry responsible for the temporal scaling of action, and suggest the intriguing possibility that the static properties of aged cortico-basal ganglia networks have direct effects on the duration of action sequences, compromising behavioural automaticity.

In conclusion, we provide evidence for a severe deficit in corticostriatal function during normal ageing in mice that engenders a distortion of action sequences after extended training. Our findings have implications for skill learning in ageing and point to the stationarity of cortico-basal ganglia transmission as a major feature of the aged brain hampering its ability to acquire new skills. Importantly, we found that action-related feedback cues given throughout training transitorily solves this problem, a solution that could be readily implemented in current rehabilitation strategies and behavioural therapies for cognitive ageing (Lindenberger and Mayr, 2014).

Materials and methods

Mice

All procedures were approved by the University of Queensland Animal Ethics Committee (QBI/412/14/NHMRC and QBI/027/12/NHMRC) in accordance with the Animal Care and Protection Regulation (Queensland Government, 2012)and the Australian Code for the Care and Use of Animals for Scientific Purposes (National Health and Medical Research Council, 2013National Health and Medical Research Council, 2013).

C57BL6/J mice (RRID:IMSR_JAX:000664) were purchased from the Animal Resources Centre (Perth, Australia), and an ageing colony was established in the Queensland Brain Institute Animal Facility. Young mice used for the experiments were 2 months old at the beginning of the experiments, whereas aged mice were 20–22 months old. Transgenic mice were purchased from the Mutant Mouse Resource and Research Centers (MMRRC, NIH) and outbred to C57BL6/J (RRID:IMSR_JAX:000664) for more than six generations in the Queensland Brain Institute Animal Facility. Heterozygous Drd1a-Cre+/− mice (strain B6(Cg)-Tg(Drd1a-cre)FK150Gsat/Mmucd [RRID:MMRRC_029178-UCD]: expresses Cre in dSPNs) were bred with homozygous Drd2-EGFP mice (strain B6(Cg)-Tg(Drd2-EGFP)S118Gsat/KreMmucd [RRID:MMRRC_036931-UCD]: expresses EGFP in iSPNs). This crossing produced 50% of Drd1a-Cre+/− Drd2-eGFP+/− mice and 50% of Drd1a-Cre−/− Drd2-eGFP+/− mice, which were used in the experiments at 2 months of age. All mice weighted 23–26 g (young) and 28–34 g (aged) at the beginning of each experiment, although food intake was restricted throughout most of the experiments (mice were maintained at approximately 85% of their free-feeding weight). All mice were housed in plastic home cages (Optimice, Animal Care Systems, CO) in a climate-controlled colony room on a 12 hr light/dark cycle and handled daily, and were allowed ad libitum access to water. Littermates in each home cage were maintained throughout the experiment, and were randomly allocated to each experimental group.

Behavioural procedures

Apparatus

All training procedures were carried out in sound- and light-resistant operant chambers (MED Associates, PO). The chambers were illuminated with a 3 W, 24 V house light. Each chamber contained a recessed feeding magazine in the center of the right side wall connected to two individual food dispensers that could deliver 20 mg dustless precision pellets (Bio Serv, NJ, #F0163) into the magazine when activated. Either side of the magazine contained retractable levers. Med-PC software was used to direct the insertion and retraction of the levers, illumination of the light and delivery of the pellets. The software allowed the recording of timestamp events each 10 msec, including the number of lever presses, magazine entries and food pellets delivered throughout the duration of the experiment. Activity was also monitored using D-Link DCS-932L IP LED infrared cameras (VGA 1/5 inch CMOS sensor with built-in microphone) through D-ViewCam Software (D-link Corporation, Taiwan).

Magazine training

Request a detailed protocol

Prior to all instrumental procedures, mice were exposed to four or six days of magazine training in the operant chambers (one session/day). Each mouse was assigned one operant chamber, which was maintained throughout the duration of the experiment. The chamber light was illuminated to signal the beginning of the session and extinguished at its termination. Both levers were retracted and mice explored the chamber freely. Twenty grain food pellets (20 mg, 3.35 kcal/g each) were delivered into the magazine at random time intervals (RT 60 s) over ~30 min. These pellets differed in taste but were calorically similar (grain: 3.35 kcal/g, purified: 3.6 kcal/g). Consumption of pellets was monitored at the end of each session.

Standard instrumental training

Request a detailed protocol

All behavioural procedures were based on single-lever operant conditioning and started with a standard instrumental training module. Each training session began with the illumination of the chamber light and insertion of the two levers into the operant box, and ended with the extinguishing of the chamber light and lever retraction. The mice were exposed to one training session each day where only one of the levers was contingent to the delivery of the pellets (active lever). Assignment of right or left active lever was counterbalanced across mice. Each training session continued until mice had received a total of 20 pellets, or the session timed out at 30 min. For the first three days of instrumental training (days 1–3), mice were trained on a constant reinforcement schedule (CRF); every lever pressing action was rewarded. The probability of the outcome given a response (P) was gradually shifted over the following days of training using increasing random ratio (RR) schedules of reinforcement: a RR5 schedule (p=0.2) was always used on days 4–6, and RR10 (p=0.1) and RR20 (p=0.05) were applied from day seven as indicated in each particular experiment (see below).

Sequence training

Request a detailed protocol

Sequence training procedures started with standard instrumental training (days 1–6). RR10 and RR20 were then applied as indicated in each experiment (see Figures 1A, 5E and 6A). In parallel to this schedule of reinforcement, an additional rule to promote sequence learning was applied from day 10 onwards (sequence trigger, ST, Figure 1—figure supplement 1), which limited the access to RR reinforcement to action sequences that were initiated with at least 5 (applied from day 10) or 7 (applied from day 13) consecutive lever presses. Sequences that were truncated with a termination element (lever press → magazine entry) before reaching the fifth or seventh consecutive press blocked access to the RR program. Unlocked sequences (≥5 initial presses) that were truncated with a termination element before earning reward also reset the RR trial. This was achieved in MED PC-IV by caching and verifying, for each lever press, the identity of the behavioural elements stamped 5 or 7 positions back in the timestamp array along each round of reinforcement. On training day 18, the mice underwent an additional training session that lasted for 20 min and had no limit of rewards prior to perfusion (see section ‘Transcardial fixation and tissue sectioning’ below). In the feedback cue experiment (Figure 6), the termination element in each sequence (i.e., the lever press that triggered the delivery of the reward) was cued with a 0.75 s white noise throughout training (see Figure 6A). On day 19, the cohort of mice was divided in two groups, according to matching performances. Group Maintained kept the same procedure with cued sequences until day 36, whereas Group Lost stopped receiving such cue but kept all the other parameters of the task unchanged. Mice in Group Lost were run in a separate session to prevent sound propagation from cued animals.

LH training

Request a detailed protocol

Lever Hold (LH) operant conditioning procedures (Figure 2) were a variation of our previously reported variable interval hold task (Bailey et al., 2015), and started with standard instrumental training (days 1–3), where each individual press was reinforced (CRF or LH0). The probability of outcome delivery was then rendered dependent on the time of lever hold, and was gradually shifted over the following days of training using increasing pseudorandom lever hold (LH0.3-LH5) intervals. During LH0.3 (days 4–6), the distributions of required hold durations had a mean of ~0.3 s (LH0.3: min = 0.05 s; max = 0.88 s). The required hold durations for the subsequent sessions were drawn from an exponential distribution with a higher mean (LH0.8 [days 7–9], LH2 [days 10–12], LH3.2 [days 13–15] and LH5 [days 16–17). Thus, during the final session of LH training, subjects were required to hold down the lever for intervals that averaged 5 s but could be as long as 11.7 s.

Chemogenetics

Request a detailed protocol

Specific inhibition of dSPNs regionally contained in the DLS was achieved by pulse-based microinjection of Cre-dependent hM4Di DREADDs virus (AAV2-hSyn-DIO-hM4D(Gi)-mCherry, UNC Vector Core, NC) into the DLS at two different rostro-caudal levels of Drd1a-Cre−/+ Drd2-EGFP+/− and Drd1a-Cre−/− Drd2-EGFP+/− double transgenic mice through stereotaxic surgery (coordinates: R-C: +1.1/+0 mm [bregma]; M-L: ±2/±2.75 mm [bregma]; D-V: −2.625/–2.75 mm [skull surface]) combined with an intraperitoneal injection of clozapine N-oxyde (CNO; dissolved in saline to 3 mg/kg; NIMH Chemical Synthesis and Drug Supply Program, Bethesda, MD). In order to validate the effects of this manipulation, a first cohort of these mice (2 Drd1a-Cre−/+ Drd2-EGFP+/− and 2 Drd1a-Cre−/− Drd2-EGFP+/−) was used where the virus was injected unilaterally (see Figure 5A). After recovery from surgery, these mice received a first injection of CNO and 30 min later another injection of GBR12783 (15 mg/kg, i.p. dissolved in sterile water) prior to rapid intracardial perfusion (see Figure 5A–C). Animals used for sequence training received a bilateral injection of the virus and were injected with CNO on days 10–17 45 min prior to the start of the training session (see Figure 5D–L).

Stereotaxic surgery

Request a detailed protocol

Mice were pre-anesthetised in an induction chamber with oxygen/isoflurane mixture (Laser Animal Health, Pharmachem, Australia). Pedal reflex was used to monitor anesthesia before placing the mouse in the stereotaxic frame (Kopf Instruments) fitted with a mask supplying a continuous flow of oxygen/isoflurane mixture (1 L/min oxygen, 1.5 ml/min isoflurane). Once deeply anesthetised, a 1.5 cm incision was made to expose the skull. For each microinjection, a small hole was pierced (~0.2 mm width) at the appropriate A-P and M-L coordinates (see above). A pulled glass capillary (GC100TF-15, Harvard Apparatus) was pulled using a micropipette puller (P-97, Sutter Instrument) and pre-filled with the solution and fitted into a Nanoject II (Drummond Scientific) before being slowly inserted vertically through each hole until reaching target (D-V coordinate). Once in target, injections were preceded by a 2 min waiting interval. Two sets of pulse injections of hM4Di-DREADDs AAV were delivered in one (unilateral experiments) or both (bilateral experiments) sides of the brain. Each injection consisted of 14 pulses of 69 nL spaced by 30 s. Two min after the last pulse, the pipette was gently pulled out, and the incision was closed with surgery suture silk and sealed with tissue adhesive (3M Vetbond). Animals were placed in cages separately on a warming plate (37°C) during recovery. A pre-operative s.c. administration of butorphanol (torbugesic, 3 mg/kg) ensured analgesia during and after the intervention. The antibiotic Baytril (5 mg/Kg) was administered at the end of the procedure. Both torbugesic and baytril were administered once a day for 2 days after surgery. Behavioural procedures started after a 3 week recovery period. None of the mice displayed perceptible wounds by the start of the behavioural experiments.

Tissue processing and immunofluorescence

Transcardial fixation and tissue sectioning

Request a detailed protocol

Twenty min after the start of the session (day 18), the mice were rapidly anesthetised by exposure to 10 s of isoflurane gas (4% in air; Laser Animal Health, Pharmachem, Australia) in a sealed chamber, followed by a lethal intraperitoneal injection of sodium pentobarbital (500 mg/kg; Virbac Pty. Ltd., Australia). After ensuring deep anesthesia by testing paw and tail reflexes, mice were perfused transcardially using an air pressure system (constant flow of 15 ml/min) with 4% paraformaldehyde (PFA) in a solution of 0.1 M sodium phosphate buffer (pH 7.4). Brains were dissected and post-fixed overnight in PFA solution at 4°C. Consecutive 30 μm coronal sections spanning the rostro-caudal extent of the striatum were obtained for each animal using a vibratome (VT1000s, Leica Microsystems, Germany). Free-floating slices were stored at −20°C in a cryoprotectant solution (30% ethylene glycol, 30% glycerol, 0.1 M sodium phosphate buffer) until processed for immunofluorescence.

Immunofluorescence

Request a detailed protocol

Free-floating slices were rinsed three times in Tris-buffered saline (with sodium fluoride id phosphorylations were detected: TBS NaF; 0.25 M Tris, 0.5 M NaCl, 0.1 mM NaF, pH 7.5) for 10 min on an orbital shaker at room temperature prior to membrane permeabilisation treatment (0.3% triton X-100 in TBS [NaF]), which was applied for 1 hr. After three 10 min washes with TBS (NaF), slices were incubated at 4°C on an orbital shaker with a rabbit polyclonal anti- threonine 202 and tyrosine 204-phosphorylated p44/42 MAPK-ERK1/2 primary antibody (p-MAPK; diluted 1:500; Cell Signaling Technology, MA; Cat# 9101L RRID:AB_331646). Following a 24 hr incubation period, unbound primary antibodies were washed off with 3 washes of TBS (NaF), and bound primary antibodies were detected through incubation for 1 hr at room temperature with donkey anti-rabbit fluorescent Cy3-conjugated secondary antibodies (diluted 1:800, Jackson ImmunoResearch Laboratories, PA). In the viral infection studies, simultaneous detection of iSPN fluorescence (Drd2-eGFP), dSPN virus expression (Drd1a-hM4Di-mCherry) and neuronal activation (p-MAPK, Cy5 donkey anti-rabbit antibody, 1:800 dilution) was conducted. Unbound secondary antibodies were washed off through four final rinses in TBS. Samples were then placed on Superfrost Plus coated slides (Thermo Fisher Scientific, MA) and mounted with a coverslip on Vectashield fluorescence medium with or without DAPI nuclear staining (Vector Laboratories, CA). Slides were stored at 4°C in the dark until image acquisition.

Image acquisition

All image datasets in this study were obtained in the Queensland Brain Institute Advanced Microimaging and Analysis Facility.

Spinning disk microscopy

Request a detailed protocol

Neuronal activity mapping experiments were performed through wide-field spinning disk confocal microscopy. A spinning disk confocal microscope (W1 Yokogawa spinning disk module; Zeiss Axio Observer Z1) equipped with sCMOS camera (Hamamatsu Flash 4.0 with 2048 × 2048 chip) was used to capture p-MAPK-labeled individual tiles (optical magnification: 20X; pixel depth: 16 bit; image resolution: 0.625 pixels/µm; image size: 1024 × 1024 pixels, 561 laser). Tiles were stitched together using Slidebook 6.0 software to reconstruct a high-resolution image of bilateral striata. In each mouse, three sections at the following approximate coordinates were reconstructed (A-P distance from bregma): 1.7 mm (rostral, 1.1 mm (medial) and 0.15 mm (caudal). Three-channel tiles (488 nm, 561 nm and 637 nm lasers) were obtained in viral infection experiments.

Scanning confocal microscopy

Request a detailed protocol

Simultaneous analysis of iSPN eGFP fluorescence, hM4Di-mCherry virus expression and p-MAPK distribution was performed through point scanning confocal microscopy. Single plane confocal images (optical magnification: 40X; averaging scans: 4; pixel depth: 12-bit) were captured using a Zeiss LSM 710 confocal laser-scanning microscope (Carl Zeiss AG, Oberkochen, Germany). Three-channel images (488 nm, 561 nm and 633 nm lasers) of the dorsolateral striatum were taken for each hemisphere in 2 Drd1a-Cre−/+ Drd2-EGFP+/− and 2 Drd1a-Cre−/− Drd2-EGFP+/− mice.

Data acquisition and analysis

Action data (timestamp)

Request a detailed protocol

[Figure 1—source data 1, Figure 2—source data 1, Figure 5—source data 1, Figure 6—source data 1, Figure 1—source data 3] A combination of MEDState Notation (MSN, MED Associates) language and a series of customised MATLAB (MathWorks, Natick, MA) scripts were used to retrieve, sort and plot timestamp datasets. All individual events (left and right lever press, magazine entries, and pellet deliveries) that occurred during each training session were recorded. The time at which each event occurred (y matrix) was stamped with a 10 ms resolution (z matrix, centiseconds). On each subject, streams of timestamp data of each category (i.e. 1–4) were combined and chronologically organised. Discrete sequences and their initiation, execution and termination elements were then identified based on the interspacing of lever presses and magazine checks. Consecutive magazine checks produced after the termination of a sequence were ignored. Parameters such as within-sequence inter-press-intervals (IPIs), sequence length (number of actions per sequence) and sequence latency (time elapsed from initiation to termination) were calculated at this point for each sequence, and average values were obtained for each mouse. Chunking during action sequence learning was represented with scatter plots of identified sequence elements (see Figure 3G,J,K; Figure 3—figure supplement 2; Figure 5H; Figure 6H). Action intervals were categorised into within-sequence IPIs and sequence boundary intervals. Within-sequence IPIs (execution) involved the intervals formed between presses contained within each sequence. Sequence boundary intervals were calculated by adding up press-check (termination) and check-press (initiation) intervals. Boundary intervals that went beyond 20 s involved engagement in other behaviours and were disregarded. Data in the scatter plots are random (y axis) and Log10 transformation of element intervals (x axis). Kernel density curves indicating clustering of each interval type within a particular session are also plotted.

Action data (acoustics)

Request a detailed protocol

[Figure 3—source data 3] Due to the temporal bias induced by timestamp-based data retrieval at short time intervals (see Figure 3—figure supplement 1C), we calculated real inter press interval times on selected fast sequences on day 18 (see Figure 3A–C). The five fastest sequences (from the timestamp data) were identified in the video recordings (.asf files) in each mouse, and the corresponding audio file (.wav) was stripped using VideoLAN software (VLC v2.1.3). Bursts of lever presses were isolated by identifying the acoustic fingerprint of single lever depressions based on superimposing the waveform frequency trace (MATLAB) with the matching spectrogram (WaveSurfer software) (see Figure 3C). This method allowed to confidently distinguishing discrete lever press sounds over the background noise as well as to rule out lever-bouncing artefacts and other unrelated sounds. Isolated sequences were confirmed by video inspection. Real IPI and speed data were then calculated in each burst by measuring the time between and frequency of peaks (see Figure 3C).

Pellet drop detection

Request a detailed protocol

[SourceData 7]. Recordings were performed inside the same mouse operant conditioning chambers used in all experiments (MED Associates, PO), with all doors closed and the fans activated. Ten food pellets were randomly delivered to the food receptacle by activating the pellet dispenser output (electric rotor moves 1/12th of a turn) with a p=0.1 each second. Sessions ended when 10 pellets had been delivered (average session time = 1.66 min). Pellet deliveries were timestamped with 1 ms resolution. During this time, parallel sound level and vibration recordings were conducted in different devices. Sound level data was recorded using Sound Analyzer v2.2 at variable time weightings (Fast [F]: 125 ms up and down; Slow [S]: 1 s up and down; and Impulse [I]: 35 ms up and 1.5 s down) and different frequency filters (Z [zero]-filter: audio range 10 Hz to 20 kHz ±1.5 dB; A-filter: audio range −20 Hz to 20 kHz; C-filter: loud sound detection). Sound exposure level (E) and equivalent continuous sound level (eq) were also recorded for each filter. Only recordings best detecting the pellet drop sound are shown in Figure 1—figure supplement 1A. Vibration was recorded with VibSensor v2.1.1 (Now Instruments and Software) as high-pass filtered accelerometer data (99.3 Hz) on three different movement axes (x, y, z).

Phospho-MAPK mapping

Request a detailed protocol

[Figure 4—source data 1] High-resolution mosaics obtained in Slidebook 6.0 (3 per animal from rostral, medial and caudal sections; see ‘Image acquisition’ section above) were processed using the non-commercial open-source ImageJ2/Fiji software (v.1.49j10; Wayne Rasband, National Institutes of Health, USA). The following cortical and striatal regions were analysed: Prelimbic cortex [PrL], anterior cingulate/Motor region 2 [aCg/M2], anterior dorsal striatum [aDS], core [Co], shell [Sh], dorsomedial striatum [DMS], dorsolateral striatum [DLS], frontoparietal cortex [FPx]. The outer boundary of the different cortical and striatal regions was manually drawn in each hemisection, the area of the region was measured (mm2), and Cartesian coordinates (x, y) of the points defining the edge were obtained. To quantify the number of activated neurons and retrieve their location across the striatum, images were thresholded based on p-MAPK immunolabelling and individual somata were automatically detected using the ‘Analyze Particle’ command in ImageJ2/Fiji, which finds the edge of an object, determines its position and outlines its contour creating a region of interest (ROI). Resulting data, including neuron count, position (x, y coordinates) and area (px2) of the centroid of each labeled neuron, was imported into MATLAB (MathWorks, Natick, MA). We then used a customised script that performs the in-built ‘inpolygon’ function, which returns a list of the points lying inside the edge of the polygon area (cortical and striatal regions). The spatial distribution of neurons was digitally reconstructed by combining a line plot defined by the edge points of the regions and a scatter plot with circles at the locations specified by the neurons’ centroid coordinates.

Statistics

Data were analysed using SPSS Statistics software (version 22; IBM Corporation, Somers, NY). The a priori alpha level was set at p<0.05. All statistical details of each experiment, including the n, the statistical test applied and significance, can be found in the Results and in the figure and figure legends. The factors involved are indicated in the text (in brackets). To analyse the effect of one or more independent variables, Levene’s test was first used to test the null hypothesis that the error variance of each dependent variable was equal across groups, and uni or multi-factorial ANOVA was then conducted. If one or more repeatedly measured variables were considered, uni- or multifactorial repeated measures ANOVA was conducted. Homogeneity of variance was tested through Mauchly’s test of sphericity, and Greenhouse-Geisser corrections were considered when sphericity was not met. In mixed ANOVA analyses involving within-subject and between subjects comparisons, factors in brackets corresponded to within-subject (first) and between subjects (second). In instances where further statistical detail was appropriate, we conducted additional simple effects comparisons based on individual one-way ANOVAs. Statistical comparisons between two means were conducted through independent t-tests, and equal variances were assumed or not according to the Levene’s test for equality of variances. To measure the relationship between two variables, Pearson’s correlation coefficient (r) was calculated, where the value can lie between −1 (as one variable changes, the other changes proportionally in the opposite direction), to +1 (as one variable changes, the other changes in the same direction by the same amount). Correlations involved within-subjects measures.

References

    1. National Health and Medical Research Council
    (2013)
    Australian Code for the Care and Use of Animals for Scientific Purposes
    Australian Code for the Care and Use of Animals for Scientific Purposes, 8th edn, Canberra, 1864965975.
    1. Lashley KS
    (1951)
    Cerebral Mechanisms in Behaviour
    112–131, The problem of serial order in behaviour, Cerebral Mechanisms in Behaviour, New York, Wiley.
    1. Sternberg S
    2. Monsell S
    3. Knoll RL
    4. Wright CE
    (1978)
    Information Processing in Motor Control and Learning
    117–152, The latency and duration of rapid movement sequences: Comparisons of speech and typewriting, Information Processing in Motor Control and Learning, Academic Press.

Decision letter

  1. Geoffrey Schoenbaum
    Reviewing Editor; NIDA Intramural Research Program, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Corticostriatal deficit introduces temporal limits to automatic action in aging" for consideration by eLife. Your article has been reviewed by three peer reviewers, one of whom, Geoffrey Schoenbaum (Reviewer#1), is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Richard Ivry as the Senior Editor.

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

In this study the authors tested the ability of young and aged mice to organize actions in an RR20 lever pressing task into discrete sequences or chunks. Young controls developed lever press sequences that were of the approximate correct length to obtain reward efficiently, with short periods between presses, and longer periods preceding the first press after a magazine entry or following the last press before entry. Aged mice also increased their pressing, however their behavior was more sporadic. Sequences were shorter, executed much more quickly, and gaps between presses and other presses and between presses and food magazine behavior were not distinguishable. As a result, aged mice were much less efficient at harvesting reward. The authors use MAPK expression to identify a circuit including mPFC and dorsal striatum normally involved in habit learning in which activity seemed to be reduced in aged subjects. Cuing aged rats with a tone at the end of the sequences allowed for temporary rescue of their behavior, indicating changes were not low level motor problems. But improved performance was not lasting if the cue was removed. Chemogenetic inactivation of direct pathway neurons in young controls in dorsolateral striatum reproduced some of the speeding and microchunking behavior but did not affect overall efficiency the same way.

Essential revisions:

The reviewers identified 3 areas that required revisions that were deemed essential for publication. The first and most important is to clarify that the behavior shown by young rats is in fact chunked and that this is lost in aged rats. While the current detailed analysis was appreciated by all 3 reviewers, it lacks some of the traditional measures to show this. This is captured most clearly by the comment by reviewer 2 on sequence variability in Figure 1 and also analysis of Figure 3. The second critical area was to address whether the deficit shown is primarily in sequence generation or whether this is secondary to another problem, such as inability to hear the pellet dispenser or differences in the use of the trigger in the sequence in aged rats or perhaps simply locomotor problems. Finally there were concerns about the dose of CNO used and also generally the differences between the pattern of effect of the deficit in this experiment versus what is shown by the aged rats. These must all be addressed to satisfy the reviewers.

Reviewer #1:

In this study the authors tested the ability of young and aged mice to organize actions in a simple lever pressing task in to discrete sequences or chunks. Mice were trained on a RR20 task in which chunking was encouraged by a requirement that a series of presses be executed without checking the magazine in order to "access" the RR20 sequence. In young controls, this resulted in the development of lever press sequences that were of the approximate correct length, with short periods between presses, and longer periods preceding the first press after a magazine entry or following the last press before entry. Aged mice also increased their pressing, however their behavior was more sporadic. Sequences were shorter, executed much more quickly, and gaps between presses and other presses and between presses and food magazine behavior were not distinguishable. As a result, aged mice were much less efficient at harvesting reward. The authors use MAPK expression to identify a circuit including mPFC and dorsal striatum normally involved in habit learning in which activity seemed to be reduced in aged subjects. Cuing aged rats with a tone at the end of the sequences allowed for temporary rescue of their behavior, indicating changes were not low level motor problems. But improved performance was not lasting if the cue was removed. Chemogenetic inactivation of direct pathway neurons in young controls in dorsolateral striatum reproduced some of the speeding and microchunking behavior but did not affect overall efficiency the same way.

Overall the study is excellent. It addresses a very interesting question of how action chunking is affected by aging with an elegant and carefully designed and analyzed task. The authors use gene expression to link changes in aged mice with alterations in a neural circuit implicated by a number of labs in the motor sequence learning, and they go the extra mile to manipulate an important part of this circuit to reproduce part of the deficit which gives confidence that the deficits are related to the genetic markers of function. I only have a few suggestions.

One question I had is whether there is any relationship between the aged rats behavior and the unusual requirement for a short burst of presses to access the RR20 routine. I am not sure I understand fully what this means and its implications, but it seemed to me initially that the microchunks could be influenced by this unusual requirement. Is there any possibility of this?

Second I think the authors need to do a better job clarifying the relationship they see between the aging deficit and the effects of the chemogenetic manipulation. It reproduced some features of the aged mice, but it did not cause the loss of efficiency or (oddly) the pattern of IPI's between lever presses versus magazine-lever interactions (panels K). This seems like it deserves to be noted more clearly in the results (assuming I am not misunderstanding) and hit more head on in the discussion. Obviously there is no reason to require one specific pathway to account for all the changes in aging. Just saying this more clearly would be helpful, unless there is a more specific or different explanation.

Lastly I think the authors should make clearer in the discussion why one would think the microchunking in the aged mice is a form of habit or model-free learning. They discuss nicely why their data suggests it is not. But there was not much of a lead in to this section to convince me this was an important question.

Reviewer #2:

This is an interesting body of work examining the capacity for sequence generation in aged mice. A self-initiated paradigm is employed (minus Figure 6) to examine the generation of sequences in lever press behavior devoid of any obvious predictive cue. They report decreased capacity of aged mice to develop automatic sequences, and proceed to investigate circuit activation, potential contribution, and rescue. While there are some nice behavioral analyses, it is not clear that mice are producing automatic sequences; there is a noted absence of analyses on the variability in any aspect of sequence generation. There are major concerns about the circuit activity analyses, and the targeted circuit manipulation was a blunt measure of direct pathway involvement and does not provide any insight into how D1 SPNs may differentially encode or represent sequences. In addition, there are interpretational concerns that arise with the data from the rescue experiment. Overall, while the data approach was interesting, it largely appears that older mice might just be checking more, or at least aging predisposes an increase in checking behaviors. At this point, the conclusion that aging results in impairments specifically to sequence generation in aged mice and related deficits in associated neurobiological mechanisms is not all together convincing.

Concerns

Figure 1: In relation to Figure 1, necessary data and analyses for interpretation are missing. Do young and old mice differ on the amount of sequences performed? Data should be included on how well mice were able to meet the sequence trigger (the efficiency graph on Figure 3D addresses overall performance, but not performance of the ST). It is possible that the aged mice never learned the contingency between the ST and access to the RR, particularly as age is known to engender behavioral inflexibility, and pre-training without the ST requirement may have prevented its acquisition. It would similarly be of interest to examine sequence behavior in these two groups prior to the introduction of the ST requirement.

In addition, the authors should include analysis and discussion of how variable their sequence behavior is. In Figure 1F it appears as though, even by day 17 of training, there is substantial variability associated with the sequences length and duration, both within and across subjects. To what extent are the mice actually performing stereotyped sequences vs. variable ones? This is an important concern, since automaticity would be expected to lead to relatively invariable sequences, yet it appears as though the sequences are highly variable, and therefore, it is unclear how mice are understanding and solving the task.

Also missing are appropriate post hoc analyses that would arise from the significant interactions reported for D and E, which may affect the conclusions proposed. For example, a significant interaction is reported for Figure 1D, and indeed the magnitude of sequence length is visually different between young and aged. However, all additional follow up analyses were not performed. They report a main factor of day for both young and old, but no comparison is made between young and aged for the different days. It seems likely that a difference would arise between ages on Day 17. This could then raise an additional hypothesis that aged mice do not extend sequences in the same manner, and importantly, and such, changes in the variability of such sequences needs to be examined.

Figure 3: The chunking analyses are quite interesting, however, they rely on visual interpretation of differences. The authors should apply some quantitative methods to these data to assess group differences. For instance, visually (figure supplement) it appears as though aged mice simply perform many more sequences than young mice (see concern above about amount of sequences), which might explain some of the chunking results. Proportional analyses may be appropriate here. As it stands, this data does not provide evidence that aged and young animals chunk sequences differently (and may suggest different amounts of overall lever pressing). Average inter-event intervals of various types could also be compared and would be informative.

Figure 4: The investigation into how recruitment of activity in corticostriatal networks represented in Figure 4 is somewhat troubling, in that it may be inherently confounded by general differences in lever pressing and have nothing to do with sequence generation. Are the groups matched for total lever presses? Is it that one group (aged) is just making more lever presses and hence, that could account for the differential activation in the circuit (based on action execution and nothing to do with sequence generation)? In Figure 4C, the large variability between subjects is apparent, and while various sequence factors were correlated with counts observed, whether overall lever presses correlated was not presented. Given this variability within a given group, do similar relationships arise for within group correlations? Further, for the analyses in D and E, how is the variability in each animal factor into the variability in the overall analyses? This is not clear in the Materials and methods. Are total activity levels normalized is some manner?

Figure 5: The CNO dosage used is quite high (3mg/kg), and this coupled with repeated dosing across days raises concerns about the potential changes in plasticity that may have occurred. This would raise alternative hypotheses other than an effect of altered D1 SPNs underlying sequence deficits in older animals. If specifically, D1 SPNs control over sequence execution is altered in aged mice, then acute attenuation should result in some alteration in performance. Also, it is unclear from the present data if dMSNs in the DLS are important in learning/acquiring these sequences, in their performance, or both. Although not unconvincing, for Figure 5I and J, the authors should have consistency between the data presented here and in Figure 3B and C. In Figure 3, the authors show IPI and speed data from only the fastest sequences, while in Figure 5 the speed and IPI data appears to be collapsed across all sequences. Does CNO replicate the effect on the fastest sequences, and additionally, does age have the same overall effect as CNO appears to? Is there any effect on the overall number of sequences? Why was the comparison made at day 10 instead of 7 (different schedule requirements were in place)? Perhaps the authors should temper their comparisons between age and dMSN inhibition. Particularly since CNO did not recapitulate the effects of age on either action chunking or task performance (where indeed, CNO and age had opposite effects).

Figure 6: This is interesting data, but also seems open to additional interpretations. While the introduction of the auditory cue could be perceived as action feedback, it may also act as a conditioned reinforcer and able to support actions independently. The authors raise the spectrum of cognitive as well as motor deficits in aging, but initiating actions under the control of conditioned reinforcement is a different hypothesis than self-initiated automatic action sequences. Additionally, the improved performance in aged mice with the addition of a loud salient auditory noise also raises the question of what action feedback is missing in Figure 1? Old mice in these age ranges, especially C57s loose large frequencies ranges in their hearing capacity. They appear to hear the white noise, but it does raise the potential that they are missing some other auditory cue that is there during sequence learning. Are they just hearing impaired and can't rely on any auditory feedback that normally arises during training (pellet dropping, lever, etc.)? This may not reflect anything on the ability to form sequences exactly, but to use auditory information?

Reviewer #3:

These data show action sequence chunking in young mice that is disrupted in aged mice. Aged mice show shorter action sequences that are, somewhat unexpectedly, sped up. This is associated with disrupted activation of caudolateral corticostriatal networks and is restored with the addition of a sequence termination feedback stimulus. Selective inactivation of DLS D1-expressing MSNs, like aging, also caused increased sequence speed. The behavioral analysis is novel and strong and is itself a significant contribution to the field. I particularly appreciate the inclusion of individual data. The ability to remedy the behavioral deficit with the feedback cue in aged mice is also a major strength of the report. The report is well-written and likely to make a significant contribution to several fields.

1) It is not clear how the DLS D1-MSN inactivation data relate to the aging data. Both findings are very interesting in their own right. But the manuscript (e.g., these statements from the Abstract: "Pharmacogenetic disruption of a striatal subcircuit in young mice reproduced most age-related action features" and Results "we hypothesized that age-related changes in corticostriatal function and resulting hypoactivity of dSPNs in the DLS could be generating deficits in action sequence latencies. We directly tested this possibility […]") implies that disrupted DLS D1 MSN activity underlies the deficits in aged mice. There are no data to support this and the DLS D1 MSN inactivation does not recapitulate the full or even most of the phenotype of aged mice. Gathering data to link these two data sets (e.g., evaluation of DLS D1 MSN activity in aged mice), as far as I can tell, is not feasible on any sort of reasonable timeframe. One option is to remove these data from the report. If they are included, then their presentation should be changed throughout to make clear this is not an evaluation of the mechanism of the aging effect and the limitations linking this result to the aging result should be thoroughly discussed.

2) Is it possible that differences in the ability to hear explain the differences in sequence performance in young vs. aged mice? The click of the pellet dispense provides a subtle cue signaling reward delivery. If the aged mice cannot hear this, they cannot use it as a cue to signal they should terminate pressing and check the magazine, as a result they have shorted sequences because they have to check the magazine more often. Data to support or refute this possibility are needed. Perhaps there are data from the literature to draw on. This possibility might also explain why the explicit auditory feedback cue that the aged mice can hear facilitates sequence duration in the aged mice.

3) Related to point 2, inspection of Figure 1F indicates that aged mice are checking the magazine more than the young mice. These data should be included and the possibility that the altered sequence performance is secondary to altered reward checking behavior should be discussed.

4) What is the rationale for analyzing only the 5 fastest sequences for the main data (Figure 3B,C), but all sequences for the D1-MSN inactivation data (Figure I, J)? It would be preferable to analyze the sequence speed and IPI comparison for all sequences for both data sets.

5) Please include a quantification and statistical comparison of p-MAPK following CNO/GBR12783 on the hM4Di expressing v. non-expressing hemisphere, or inside v. outside the hM4di-mCherry expression zones.

6) Please include individual data as in Figure 1F for the DLS D1 MSN inactivation data presented in Figure 5.

https://doi.org/10.7554/eLife.29908.029

Author response

Essential revisions:

The reviewers identified 3 areas that required revisions that were deemed essential for publication. The first and most important is to clarify that the behavior shown by young rats is in fact chunked and that this is lost in aged rats. While the current detailed analysis was appreciated by all 3 reviewers, it lacks some of the traditional measures to show this. This is captured most clearly by the comment by reviewer 2 on sequence variability in Figure 1 and also analysis of Figure 3.

This is an important point and we agree that better measurements of chunking could have been provided in our initial version. We have now included new analyses of our data to adequately measure differences in the generation and development of action sequences in young and aged mice (see response to specific comments from reviewers below). Our new analysis of action chunking provides unprecedented detail to the study of sequence learning, and confirms and extends our previous finding that action sequence learning is dramatically impaired in aged mice.

The second critical area was to address whether the deficit shown is primarily in sequence generation or whether this is secondary to another problem, such as inability to hear the pellet dispenser or differences in the use of the trigger in the sequence in aged rats or perhaps simply locomotor problems.

We have now experimentally ruled out that aged mice are impaired in detecting pellet delivery. These results have been included in Figure 1—figure supplement 2. In addition, we did not observe overall decreased performance of aged animals as would be predicted by a conditioned reinforcer scenario (see response to reviewer 2 below). Thus, we believe that there should be no doubt that the deficit shown is primarily in sequence generation.

Finally there were concerns about the dose of CNO used and also generally the differences between the pattern of effect of the deficit in this experiment versus what is shown by the aged rats. These must all be addressed to satisfy the reviewers.

The reviewer’s concern on the dose of CNO is specifically addressed in our response to reviewer 2 below. We have now included a new analysis to address more directly which aspects of the aged behaviour are reproduced in the chemogenetics experiment, and which ones are not. These are detailed below. We did not intend to claim that this manipulation in young mice mimicked the aged brain. We have rephrased our text throughout to clarify that this experiment provided mechanistic substrate to only part of the problem in aged mice.

Reviewer #1:

[…] One question I had is whether there is any relationship between the aged rats behavior and the unusual requirement for a short burst of presses to access the RR20 routine. I am not sure I understand fully what this means and its implications, but it seemed to me initially that the microchunks could be influenced by this unusual requirement. Is there any possibility of this?

It is important to clarify that we did not apply any temporal limits: the sequence trigger (ST) does not require to speed up lever press responses but only to execute them consecutively. In our RR20+ST task, the best behavioural strategy is to sustain actions for an extended period of time, irrespective of the speed of action – the generation of action sequences and related automaticity on the way occur spontaneously. Young mice comfortably incorporated the ST requirement, while aged mice could only manage to extend their actions for a few seconds, which allowed them to feed only on the low proportion of “low requirement” trials that were randomly picked by the RR schedule. But even in those, there was never a timeout to access the reward. Therefore, the temporal constraints and the speed established during microchunking are indeed evidenced by the ST (e.g. see new panel C on Figure 1), but cannot be a direct consequence of its unusual requirements. Rather, we interpret this microchunking as the inability of aged mice to temporally sustain actions. We have now introduced a diagram (new Figure 1—figure supplement 1A) clarifying the ST procedure, and expanded the explanation of what it does in the Materials and methods (subsection “Standard instrumental training”).

Second I think the authors need to do a better job clarifying the relationship they see between the aging deficit and the effects of the chemogenetic manipulation. It reproduced some features of the aged mice, but it did not cause the loss of efficiency or (oddly) the pattern of IPI's between lever presses versus magazine-lever interactions (panels K). This seems like it deserves to be noted more clearly in the results (assuming I am not misunderstanding) and hit more head on in the discussion. Obviously there is no reason to require one specific pathway to account for all the changes in aging. Just saying this more clearly would be helpful, unless there is a more specific or different explanation.

With the introduction of quantitative chunking analyses in the study (see further comments below), we have now clearly identified which specific aspects of the aged behaviour are reproduced in the chemogenetics experiment, and which ones are not. Our new data revealed that interrupting intracellular signalling in dSPNs of the DLS during periods of sequence development can increase the speed of individual action sequences, although the number of elements fitted within each sequence as well as the extra sequence intervals remained unaltered. Therefore, this manipulation reproduced within-sequence but not extra-sequence features of action chunking in ageing, which, we argue, could explain the increased—rather than reduced—ability of dSPN-hM4Di(+) mice to obtain rewards during ST training. This has now been clearly explained in our Results section and a paragraph has been included in the Discussion (paragraph six).

Lastly I think the authors should make clearer in the discussion why one would think the microchunking in the aged mice is a form of habit or model-free learning. They discuss nicely why their data suggests it is not. But there was not much of a lead in to this section to convince me this was an important question.

We based this discussion on the suggestion that limited computational resources can prioritise model-free over model-based operations. This is proposed in studies such as Otto et al. 2013, where they limited the working memory (WM) capacity of healthy humans through application of an acute stressor. They found that WM compromise in this way attenuated the contribution of model-based decisions to behaviour, but did not alter model-free processing. The reason why one would think the microchunking behaviour in the aged can encourage model-free behaviours in the first place is that it is well known that the aged brain has substantial working memory limitations (e.g. Bo et al., 2009; Wang et al., 2011; Wingfield et al., 1988). We argue, however, that different features in our data suggest that this is not the case. We have now expanded this discussion in the text to make this point clearer (Discussion, first paragraph).

Reviewer #2:

[…] Overall, while the data approach was interesting, it largely appears that older mice might just be checking more, or at least aging predisposes an increase in checking behaviors.

We see the anticipatory response of checking the magazine as an unavoidable part of the instrumental program, and in fact we used these data to define the different elements of the action sequences throughout the study. While it is true that aged animals display, overall, more magazine checks than younger mice by the end of training, this is a reflection of their overt increase in the number of shorter action sequences executed throughout the session. In the new experiment presented in Figure 1—figure supplement 2B-D, we submitted food-restricted animals to 6 days of magazine training, where no instrumental action was required to access to the food pellets (they were delivered randomly). During this time, animals learn that the discriminative stimulus (likely dispenser vibration) predicts the delivery of the outcome (food pellet), and accordingly produce more anticipatory responses, particularly immediately after the stimulus (see Figure 1—figure supplement 2D). As shown in Figure 1—figure supplement 2C, we recorded the same increase of anticipatory responding (magazine check rates) in both young and aged groups. From these data, we can assume that whatever differences of magazine check rates arise during instrumental training may be consequence of differences in the way instrumental programs are established, rather than different baselines of anticipatory behaviour.

[…] Figure 1: In relation to Figure 1, necessary data and analyses for interpretation are missing. Do young and old mice differ on the amount of sequences performed?

Yes, they do. The conclusion Figure 1 was intending to reach is that aged mice, despite showing the same overall LP performance (previous Figure 1C, current Figure 1—figure supplement 1B), organise their behaviour into a larger number of sequences that are much shorter in length (number of elements) and duration (seconds). This is very evident in our plots in Figure 1F. In order to clarify that aged animals do produce a larger number of sequences, we have now substituted the panel in 1C by a graph showing average sequence rates (i.e. # sequences per min) developed throughout training. Corresponding results and statistics have been integrated in the new version of the manuscript, and we have highlighted this point in the Results section (paragraph one).

Data should be included on how well mice were able to meet the sequence trigger (the efficiency graph on Figure 3D addresses overall performance, but not performance of the ST). It is possible that the aged mice never learned the contingency between the ST and access to the RR, particularly as age is known to engender behavioral inflexibility, and pre-training without the ST requirement may have prevented its acquisition.

We believe that it is especially the aged group that undergoes strong learning of the ST, while young mice may passively overcome it (so learning in this latter group may be weak). We have now directly evidenced this by plotting the number of sequences that meet the sequence trigger (ST5) from day 10 (first day of ST5) onwards in both age groups (see new Figure 1—figure supplement 1C and related text). Data clearly show that only aged mice increased the number of sequences meeting the ST, whereas young animals kept those numbers stable until the end. This is supported by an overall significant effect of training (F(3.065,42.908) = 7.757, p < 0.001) and a significant training x age interaction (F(3.065,42.908) = 6.027, p < 0.01) (Supplementary file 1: StatsReport26). These results and corresponding statistical analysis have now been integrated in the manuscript.

It would similarly be of interest to examine sequence behavior in these two groups prior to the introduction of the ST requirement.

Behavioural structure data corresponding to day 7 (pre-sequence) has now been incorporated in the manuscript (see new Figure 1—figure supplement 1D and corresponding text).

In addition, the authors should include analysis and discussion of how variable their sequence behavior is. In Figure 1F it appears as though, even by day 17 of training, there is substantial variability associated with the sequences length and duration, both within and across subjects. To what extent are the mice actually performing stereotyped sequences vs. variable ones? This is an important concern, since automaticity would be expected to lead to relatively invariable sequences, yet it appears as though the sequences are highly variable, and therefore, it is unclear how mice are understanding and solving the task.

We disagree with the reviewer in that the establishment of automatic action sequences involves a homogenisation of the length of these sequences. A highly automatised behavioural chain (such as, for example, pedalling on a bicycle) can be sustained for as long as it is required by the current task. We purposely modelled this aspect of automatic behaviour by applying pseudorandom, rather than fixed, schedules of reinforcement. Therefore, the variability of sequence length observed in our data is likely to be the result of the variable requirement of the RR schedule, rather than within and between subjects variabilities. However, we do share with the reviewer that other reductions of variability throughout training can be used to quantitatively reflect automatisation.

Also missing are appropriate post hoc analyses that would arise from the significant interactions reported for D and E, which may affect the conclusions proposed. For example, a significant interaction is reported for Figure 1D, and indeed the magnitude of sequence length is visually different between young and aged. However, all additional follow up analyses were not performed. They report a main factor of day for both young and old, but no comparison is made between young and aged for the different days. It seems likely that a difference would arise between ages on Day 17. This could then raise an additional hypothesis that aged mice do not extend sequences in the same manner, and importantly, and such, changes in the variability of such sequences needs to be examined.

We agree with the interpretation of the reviewer and believe is the same we were intending to provide. As we mention in our text, both young and aged mice were able to extend the number of execution elements in their sequences throughout training. This is supported by an overall effect of training (two-way mixed ANOVA F(1,14) = 158.282, p < 0.001) (Figure 1D). However, as indicated by a significant day x age interaction (F(1,14) = 40.03, p < 0.001), this effect is of a different magnitude (i.e. aged animals extend less their sequences). We therefore ran subsequent simple effects analysis (one-way RM ANOVA in young and aged separately) to ensure that, despite this interaction, both groups were indeed extending the number of execution elements in each sequence. We believe that the significant interaction (rather than between subjects comparisons) is the most appropriate statistical measure to demonstrate that the increase of sequence length was significantly different across groups (see Supplementary file 1: StatsReport2 for full statistical analysis of these data). We have now rephrased our text (Results, paragraph one) to make sure that this interpretation is not missed.

Figure 3: The chunking analyses are quite interesting, however, they rely on visual interpretation of differences. The authors should apply some quantitative methods to these data to assess group differences. For instance, visually (figure supplement) it appears as though aged mice simply perform many more sequences than young mice (see concern above about amount of sequences), which might explain some of the chunking results. Proportional analyses may be appropriate here. As it stands, this data does not provide evidence that aged and young animals chunk sequences differently (and may suggest different amounts of overall lever pressing). Average inter-event intervals of various types could also be compared and would be informative.

We agree with the reviewer that more quantitative data should be used to evidence chunking. We have now substantially modified our dataset and provide straightforward measures of chunking throughout the manuscript (see new panels G-K in Figure 3 and its figure supplement 2; new Figure 5G-H and new Figure 6H and all related text). In order to better study age-related differences in chunking, we have categorised the action repertoire into two classes of elements: within-sequence elements (inter-press-intervals [IPIs] that reside within each sequence) and sequence boundary elements (a combination of press-check and check-press intervals that lay in-between sequences) (see expanded Materials and methods). We have also modified the visual representation of chunking: we now combine scatter plots with kernel density curves to represent the distribution of all within-sequence IPIs and all sequence boundary intervals that occur in one session (e.g. see new Figure 3G). These graphs continue to provide a good visual representation of the emergence of chunking during training (e.g. Figure 3G), and allow extracting different quantitative measures of chunking (new Figure 3H-K).

With respect to the proportionality of analyses, we agree that in our initial representation aged mice had more data points than young mice as they took longer to finish the final sessions, and understand the reviewer’s concern for interpretation. In the new panels J and K in Figure 3, we have now plotted all the action element intervals (within-sequence and sequence boundary) performed by all young and all aged mice during the first 600 seconds of day 18 (uninterrupted sequence performance, see Materials and methods). These data are now proportional, as they include an equal period of continuous instrumental performance. Note how the overall number of within-sequence elements is similar in both age groups, whereas aged mice produce a higher number of boundary elements, which reflects a higher overall number of shorter sequences (see inset graphs in panels 3J and K).

Overall, our new analysis evidences clearly different automatisation processes shaping within-sequence and sequence boundary elements in young and aged mice. While young mice appear to exclusively chunk the elements comprised within the sequences, aged mice show evidence of early chunking of boundary elements. We refer the reviewer to the re-written paragraph in the Results section (subsection “Sequence learning leads to rapid response bursts and aberrant chunking in aged mice”) where these new results are laid out, which further support and extend our initial conclusion that aged mice develop aberrant automaticity characterised by ultrafast microchunks.

Figure 4: The investigation into how recruitment of activity in corticostriatal networks represented in Figure 4 is somewhat troubling, in that it may be inherently confounded by general differences in lever pressing and have nothing to do with sequence generation. Are the groups matched for total lever presses? Is it that one group (aged) is just making more lever presses and hence, that could account for the differential activation in the circuit (based on action execution and nothing to do with sequence generation)? In Figure 4C, the large variability between subjects is apparent, and while various sequence factors were correlated with counts observed, whether overall lever presses correlated was not presented. Given this variability within a given group, do similar relationships arise for within group correlations?

Yes, our groups are matched in time of performance and overall amount of lever presses. On day 18, we submitted both young and aged groups to an extra session of training with uninterrupted instrumental performance for 20 minutes prior to transcardial perfusion. Aged mice did not produce more lever presses during this time (mean ± SEM, young: 949.62 ± 50.7; aged: 968.5 ± 82.8), as confirmed by the lack of statistical difference (t(14) = -0.194, p = 0.849). We have now included in our correlation matrix (Figure 4D) an extra row showing the correlation of the total amount of presses with the activation recorded in each region (see new panel D, “Total LP” at the bottom). None of these new correlations was significant (Figure 4D, p value inset), which supports that the significant correlations initially found in postero-lateral areas were related to sequence generation.

Further, for the analyses in D and E, how is the variability in each animal factor into the variability in the overall analyses? This is not clear in the Materials and methods. Are total activity levels normalized is some manner?

In our correlation analyses, the neuronal activation level in each of the regions was correlated with either sequence performance (Figure 4D) or activation in every other region (Figure 4E) of that same animal. Therefore, data points in each correlation of the matrix were obtained by contrasting within-individual measures, so normalisations across individuals are not suitable in this case. We have now clarified this in the figure legend and Materials and methods.

Figure 5: The CNO dosage used is quite high (3mg/kg), and this coupled with repeated dosing across days raises concerns about the potential changes in plasticity that may have occurred. This would raise alternative hypotheses other than an effect of altered D1 SPNs underlying sequence deficits in older animals.

We understand the reviewer’s concern on the dose of CNO, especially considering that it is a close molecular derivate of the highly psychoactive drug clozapine. We have however established in our laboratory that in order to achieve clear inhibitory effects on hM4Di-transduced neurons, the effective dose of systemic CNO needs to be pushed up to 3 mg/kg (see Figure 5—figure supplement 1). We are well aware of eventual reminiscent clozapine-like effects derived from dosing CNO up, and this is why we introduced in our study the littermate (rather than vehicle) controls, which received the same amount of CNO throughout training. Clozapine is a pro-cataleptic drug at high doses, which can have clear hipolocomotion effects through blockade of D2 receptors (see for example Valjent, Bertran-Gonzalez et al. Neuropsychopharmacology, 2011). Our data did not show any reduction of overall performance from day 9 (last day of no CNO) to day 10 (first day of CNO) of training (see Figure 5—figure supplement 1F). Moreover, CNO treatment did not modify the slope of the instrumental acquisition curve in any of the groups, which was very similar to that obtained in drug-free mice (see Figure 1—figure supplement 1B). Although this does not rule out that subthreshold clozapine activity may be taking place during this period, this undetectable side effects are also incorporated in our control mice, and therefore the differences detected between these two groups are likely driven by the chemogenetic manipulation.

The need for a relatively high dose of systemic CNO (3 mg/kg in our case) could be explained by the data recently published in Gomez & Bonaventura et al. Science 357, 503-507 (2017). In this work, the authors suggest that the main specific effects in DREADDs are driven by the small fraction of CNO that is peripherally converted to clozapine (subthreshold levels), provided that CNO itself is unable to penetrate the BBB. These authors show that the conversion of CNO to clozapine is rather inefficient, although a small concentration of peripheral clozapine appeared to be sufficient for substantial DREADD receptor occupancy. They also demonstrate that even very high doses of CNO (10 mg/kg) do not decrease baseline locomotion. Therefore, the conversion to clozapine in DREADDs seems to be the solution, rather than the problem, although this conversion appears too poor for clozapine-like side-effects to be detectable.

If specifically, D1 SPNs control over sequence execution is altered in aged mice, then acute attenuation should result in some alteration in performance. Also, it is unclear from the present data if dMSNs in the DLS are important in learning/acquiring these sequences, in their performance, or both.

It is important to clarify that we applied the CNO treatment to prevent the engagement of dorsolateral striatal plasticity at the time of sequence generation (i.e. from day 10 onwards), in an attempt to model the staticity of corticostriatal activation suggested in our experiments with aged mice. We therefore interpret these data as an enduring learning effect rather than an acute effect on performance. In other words, for action sequences to be appropriately established throughout training, it is required that plasticity in corticostriatal circuitry progressively expands to lateral territories, in accordance with previous reports (see refs Balleine 2005, Yin 2009 and Balleine & O’Doherty 2010 in our text). Under this reasoning, in our experiments where the circuitry has failed to mature in this way, sudden activation of dSPNs in the DLS of aged mice is unlikely to extend sequence duration or reduce speed, and, similarly, interruption of CNO treatment in dSPN-hM4Di(+) mice is unlikely to reduce sequence duration or increase speed.

Although not unconvincing, for Figure 5 I and J, the authors should have consistency between the data presented here and in Figure 3 B and C. In Figure 3, the authors show IPI and speed data from only the fastest sequences, while in Figure 5 the speed and IPI data appears to be collapsed across all sequences. Does CNO replicate the effect on the fastest sequences, and additionally, does age have the same overall effect as CNO appears to?

We have now homogenised the analyses of speed in the two experiments (see new Figure 3A and B and new Figure 5F and G). In the ageing experiment, overall sequence speed is higher in the aged group, although the effect falls below threshold of significance (p = 0.054) (new Figure 3A). However, frequency distribution of IPIs clearly showed a much larger number of very short IPIs in aged mice (new Figure 3B), which prompted us to perform the fastest sequence study (Figure 3C-E). In the DREADDs experiment, dSPN-hM4Di(+) mice did show a similar overall increase in speed, although this time the effect was significant (Figure 5F). The IPI distribution analysis also showed that dSPN-hM4Di(+) mice produced more very short IPIs (new Figure 5G), although this distribution was not as disproportionate as the one observed in aged mice, and therefore we did not consider relevant a fastest sequence study in this case. This is now clarified in the text: “This [increase in speed] was similar to the effect observed in aged mice (see Figure 3A-B), although peak speeds within fast sequence bursts in dSPN-hM4Di(+) were not as high as the ones recorded in aged mice”.

Is there any effect on the overall number of sequences?

Yes, these data are now included in the new Figure 5C.

Why was the comparison made at day 10 instead of 7 (different schedule requirements were in place)?

As discussed above, the aim of the DREADDs experiment was to interrupt the engagement of lateral corticostriatal plasticity to attempt to recapitulate the unusual sequence features observed in aged mice, and so we compared the performance expressed on day 10 (expected start of sequence implementation and first day of CNO treatment) and day 17 (last day of CNO treatment). During this period, the ratio requirements were kept constant (RR20 + ST) to minimise interference with the CNO effect that was expected to develop across treatment.

Perhaps the authors should temper their comparisons between age and dMSN inhibition. Particularly since CNO did not recapitulate the effects of age on either action chunking or task performance (where indeed, CNO and age had opposite effects).

We did not intend to claim that chemogenetic manipulation of dSPNs mimicked all the deficits in the behaviour of aged mice but we did show that it reproduced some important aspects of action chunking.

Figure 6: This is interesting data, but also seems open to additional interpretations. While the introduction of the auditory cue could be perceived as action feedback, it may also act as a conditioned reinforcer and able to support actions independently. The authors raise the spectrum of cognitive as well as motor deficits in aging, but initiating actions under the control of conditioned reinforcement is a different hypothesis than self-initiated automatic action sequences.

The reviewer suggests an interesting interpretation of our data by which the stimulus (S), by being presented close to the delivery of the outcome (O), could act as a conditioned reinforcer driving the instrumental actions (i.e., A-S(O)). This possibility predicts, however, that, removing S should cause the instrumental actions to extinguish over time (at least partially). However, our data shows no evidence of a change in the rate of instrumental performance when S is removed (Figure 6B, day 19 onwards). In fact, overall performance (LP per min) increased at the same rate as animals for whom S was retained (Figure 6B). Therefore, our interpretation is that the presentation of S at the end of the sequences strengthens A and so the A-O contingency. The loss of S causes a temporal reorganisation of the instrumental chains, but does not precipitate their extinction.

Additionally, the improved performance in aged mice with the addition of a loud salient auditory noise also raises the question of what action feedback is missing in Figure 1? Old mice in these age ranges, especially C57s loose large frequencies ranges in their hearing capacity. They appear to hear the white noise, but it does raise the potential that they are missing some other auditory cue that is there during sequence learning. Are they just hearing impaired and can't rely on any auditory feedback that normally arises during training (pellet dropping, lever, etc.)? This may not reflect anything on the ability to form sequences exactly, but to use auditory information?

We report additional experiments to address this issue, which is related to the second concern point by reviewer 3. Please see answer below.

Reviewer #3:

[…] 1) It is not clear how the DLS D1-MSN inactivation data relate to the aging data. Both findings are very interesting in their own right. But the manuscript (e.g., these statements from the Abstract: "Pharmacogenetic disruption of a striatal subcircuit in young mice reproduced most age-related action features" and Results "we hypothesized that age-related changes in corticostriatal function and resulting hypoactivity of dSPNs in the DLS could be generating deficits in action sequence latencies. We directly tested this possibility […]") implies that disrupted DLS D1 MSN activity underlies the deficits in aged mice. There are no data to support this and the DLS D1 MSN inactivation does not recapitulate the full or even most of the phenotype of aged mice. Gathering data to link these two data sets (e.g., evaluation of DLS D1 MSN activity in aged mice), as far as I can tell, is not feasible on any sort of reasonable timeframe. One option is to remove these data from the report. If they are included, then their presentation should be changed throughout to make clear this is not an evaluation of the mechanism of the aging effect and the limitations linking this result to the aging result should be thoroughly discussed.

We have now included new analyses that allow us to be more specific about which features of the aged behaviour are reproduced by dSPN manipulations in the DLS. We have also rephrased our text throughout to clarify that this experiment provided mechanistic substrate to only part of the problem in aged mice (see Abstract, Introduction, Results subheading and text, and Discussion). Although this experiment does not entirely reproduce the deficits observed in the aged, we believe that this manipulation provides support for the notion that dorsolateral striatum is involved in sequence learning.

2) Is it possible that differences in the ability to hear explain the differences in sequence performance in young v. aged mice? The click of the pellet dispense provides a subtle cue signaling reward delivery. If the aged mice cannot hear this, they cannot use it as a cue to signal they should terminate pressing and check the magazine, as a result they have shorted sequences because they have to check the magazine more often. Data to support or refute this possibility are needed. Perhaps there are data from the literature to draw on. This possibility might also explain why the explicit auditory feedback cue that the aged mice can hear facilitates sequence duration in the aged mice.

We believe that all animals are capable of detecting the delivery of the pellets, irrespective of their hearing capacity. We would like to clarify that, at least in our set up, there is not only an auditory sound (which is mostly masked by the fans providing continuous background noise), but also a very obvious vibration in the chamber due to the motor engagement in the pellet dispenser. We refer the reviewers to the new Figure 1—figure supplement 2, where we conducted a series of experiments specifically addressing this issue. In panel A, we recorded in parallel overall sound level (at different frequencies) and vibration inside the chamber while pellets were delivered at random times over a period of ~1.5 min. The data showed that although sound could predict the delivery of the pellets (at least at A frequency weighting), the vibration offered a much more salient signal. Considering that mice have at least two paws and tail on the grid floor at all times, and that they are exquisitely sensitive to mechanical stimulation through vibrissae detection of minute perturbations (e.g. Prescott et al. (2011) Scholarpedia, 6(10):6642), we are confident that pellet delivery is “sensed” rather than “heard” by the mice in the operant chambers. We directly tested pellet drop detection in a new experiment in young and aged mice by extracting the first check intervals (i.e. time elapsed between the delivery of the pellet and the subsequent magazine check) across 6 days of magazine training, where the pellets were randomly delivered into the magazine (Figure 1—figure supplement 2B-D). Our data showed that both groups similarly reduced their first check intervals throughout training (panels B and D) while showing equivalent overall magazine check rates (panel C), thereby suggesting similar capacity to detect pellet deliveries. These results and corresponding data and statistics have been integrated in the new version of the manuscript.

3) Related to point 2, inspection of Figure 1F indicates that aged mice are checking the magazine more than the young mice. These data should be included and the possibility that the altered sequence performance is secondary to altered reward checking behavior should be discussed.

We see the anticipatory response of checking the magazine as an unavoidable part of the instrumental program, and in fact we used these data to define the different elements of the action sequences throughout the study. While it is true that aged animals display, overall, more magazine checks than younger mice by the end of training, this is a reflection of their overt increase in the number of shorter action sequences executed throughout the session. In the new experiment presented in Figure 1—figure supplement 2B-D, we submitted food-restricted animals to 6 days of magazine training, where no instrumental action was required to access to the food pellets (they were delivered randomly). During this time, animals learn that the discriminative stimulus (likely dispenser vibration) predicts the delivery of the outcome (food pellet), and accordingly produce more anticipatory responses, particularly immediately after the stimulus (see Figure 1—figure supplement 2D). As shown in Figure 1—figure supplement 2C, we recorded the same increase of anticipatory responding (magazine check rates) in both young and aged groups. From these data, we can assume that whatever differences of magazine check rates arise during instrumental training may be consequence of differences in the way instrumental programs are established, rather than different baselines of anticipatory behaviour.

4) What is the rationale for analyzing only the 5 fastest sequences for the main data (Figure 3B,C), but all sequences for the D1-MSN inactivation data (Figure I,J)? It would be preferable to analyze the sequence speed and IPI comparison for all sequences for both data sets.

Data analysis has now been homogenised across the two experiments.

5) Please include a quantification and statistical comparison of p-MAPK following CNO/GBR12783 on the hM4Di expressing v. non-expressing hemisphere, or inside v. outside the hM4di-mCherry expression zones.

We have now included this quantification and this dataset has been moved to the new Figure 5—figure supplement 1. Results (subsection “Chemogenetic inhibition of direct pathway projection neurons in dorsolateral striatum increases speed of action sequences”), source datasets and statistical analyses of this quantification have been integrated in the new version of the manuscript.

6) Please include individual data as in Figure 1F for the DLS D1 MSN inactivation data presented in Figure 5.

These data have now been included as Figure 5—figure supplement 2.

https://doi.org/10.7554/eLife.29908.030

Article and author information

Author details

  1. Miriam Matamales

    1. Clem Jones Centre for Ageing Dementia Research, Queensland Brain Institute, The University of Queensland, Brisbane, Australia
    2. Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, Australia
    Contribution
    Conceptualization, Data curation, Software, Supervision, Validation, Investigation, Methodology, Writing—original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9978-0091

  2. Zala Skrbis

    Clem Jones Centre for Ageing Dementia Research, Queensland Brain Institute, The University of Queensland, Brisbane, Australia
    Contribution
    Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing—review and editing
    Competing interests
    No competing interests declared

  3. Matthew R Bailey

    Psychology Department, Columbia University, Broadway, United States
    Contribution
    Conceptualization, Software, Investigation, Visualization, Methodology, Writing—review and editing
    Competing interests
    No competing interests declared

  4. Peter D Balsam

    Psychology Department, Barnard College, Columbia University, Broadway, United States
    Contribution
    Conceptualization, Software, Investigation, Visualization, Methodology, Writing—review and editing
    Competing interests
    No competing interests declared

  5. Bernard W Balleine

    Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, Australia
    Contribution
    Conceptualization, Resources, Supervision, Funding acquisition, Investigation, Methodology, Writing—review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-8618-7950

  6. Jürgen Götz

    Clem Jones Centre for Ageing Dementia Research, Queensland Brain Institute, The University of Queensland, Brisbane, Australia
    Contribution
    Conceptualization, Resources, Funding acquisition, Validation, Writing—review and editing
    Competing interests
    No competing interests declared

  7. Jesus Bertran-Gonzalez

    1. Clem Jones Centre for Ageing Dementia Research, Queensland Brain Institute, The University of Queensland, Brisbane, Australia
    2. Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, Australia
    Contribution
    Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Writing—original draft, Project administration, Writing—review and editing
    For correspondence
    j.bertran@unsw.edu.au
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3794-1782

Funding

Australian Research Council (DE160101275)

  • Jesus Bertran-Gonzalez

Australian Research Council (DP130101932)

  • Jürgen Götz

National Health and Medical Research Council (APP1037746)

  • Jürgen Götz

National Health and Medical Research Council (APP1003150)

  • Jürgen Götz

National Health and Medical Research Council (GNT1079561)

  • Bernard W Balleine

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

The authors thank Dr Amir Dezfouli and Prof Mark Bouton for insightful discussions, and Dr Richard Faville and Matthew Van De Poll for assistance in MATLAB programming.

Ethics

Animal experimentation: All procedures were approved by the University of Queensland Animal Ethics Committee (QBI/412/14/NHMRC and QBI/027/12/NHMRC) in accordance with the Animal Care and Protection Regulation (Queensland Government, 2012) and the Australian Code of Practice for the Care and Use of Animals for Scientific Purposes (National Health and Medical Research Council, 2013). All surgery was performed under isoflurane gas anesthesia and butorphanol analgesia, and every effort was made to minimize suffering.

Reviewing Editor

  1. Geoffrey Schoenbaum, NIDA Intramural Research Program, United States

Publication history

  1. Received: June 25, 2017
  2. Accepted: October 22, 2017
  3. Accepted Manuscript published: October 23, 2017 (version 1)
  4. Version of Record published: November 8, 2017 (version 2)

Copyright

© 2017, Matamales et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,576
    Page views
  • 253
    Downloads
  • 7
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Miriam Matamales
  2. Zala Skrbis
  3. Matthew R Bailey
  4. Peter D Balsam
  5. Bernard W Balleine
  6. Jürgen Götz
  7. Jesus Bertran-Gonzalez
(2017)
A corticostriatal deficit promotes temporal distortion of automatic action in ageing
eLife 6:e29908.
https://doi.org/10.7554/eLife.29908

Categories and tags

Research organisms