Cortico-basal ganglia circuits are thought to mediate goal-directed learning by a process of outcome evaluation to gradually select appropriate motor actions. We investigated spiking activity in core and shell subregions of the cortical nucleus LMAN during development as juvenile zebra finches are actively engaged in evaluating feedback of self-generated behavior in relation to their memorized tutor song (the goal). Spiking patterns of single neurons in both core and shell subregions during singing correlated with acoustic similarity to tutor syllables, suggesting a process of outcome evaluation. Both core and shell neurons encoded tutor similarity via either increases or decreases in firing rate, although only shell neurons showed a significant association at the population level. Tutor similarity predicted firing rates most strongly during early stages of learning, and shell but not core neurons showed decreases in response variability across development, suggesting that the activity of shell neurons reflects the progression of learning.https://doi.org/10.7554/eLife.26973.001
Recurrent cortico-basal ganglia circuits mediate procedural skill learning, which involves goal-oriented evaluation of behavioral outcomes to gradually select appropriate motor actions. Vocal learning in songbirds provides a powerful model for studying the control of experience-dependent skill learning by cortico-basal ganglia circuits during development. Similar to infants learning speech, juvenile songbirds memorize vocal sounds of an adult tutor. They then progressively refine their own vocal behavior to imitate the tutor song through iterative comparisons between feedback of their own vocalizations and tutor sounds. Successful acquisition requires the evaluation of behavioral feedback against a representation of the goal (tutor song) to guide the gradual acquisition of an accurate imitation during the sensorimotor stage of vocal learning.
Neural control of vocal learning in juvenile zebra finches (Taeniopygia guttata) is vested in cortico-basal ganglia loops that emanate from the cortical nucleus LMAN (lateral magnocellular nucleus of the anterior nidopallium; Figure 1) (Aronov et al., 2008; Bottjer et al., 1984; Scharff and Nottebohm, 1991). Core and surrounding shell subregions of LMAN make parallel connections through the basal ganglia and thalamus (Bottjer, 2004; Luo et al., 2001; Johnson et al., 1995; Gale et al., 2008; Person et al., 2008; Iyengar et al., 1999; Paterson and Bottjer, 2017) and appear functionally similar to sensorimotor and associative cortico-basal ganglia loops that contribute to different aspects of motor learning in mammals (Thorn et al., 2010; Yin et al., 2008; Samejima and Doya, 2007; Ashby et al., 2010; Redgrave et al., 2010; Graybiel, 2008; Gremel and Costa, 2013; Yin et al., 2009; Kupferschmidt et al., 2017; Alexander and Crutcher, 1990). The core pathway mediates vocal motor production in juvenile songbirds (Elliott et al., 2014; Scharff and Nottebohm, 1991; Aronov et al., 2008), whereas the shell pathway is involved in evaluating sensorimotor learning; lesions in the shell pathway of juvenile birds impair the ability to copy tutor song syllables, but do not cause motor disruption of song (Bottjer and Altenau, 2010) (Figure 1—figure supplement 1). This disruption of learning, but not motor performance, supports the idea that shell circuitry helps to evaluate whether self-generated vocalizations match learned tutor sounds.
Within the shell subregion of LMAN in juvenile birds, distinct subpopulations of neurons respond selectively to playback of either their learned tutor song or their own self-generated song (Achiro and Bottjer, 2013), indicating that the shell pathway has access to neural representations of the goal behavior and the current version of the bird’s own song (Figure 1—figure supplement 2).
In contrast, core neurons are not selective for different song types at the onset of sensorimotor learning, but gradually become selectively tuned to playback of their own song (Doupe, 1997; Solis and Doupe, 1997). In addition, core neurons that project to vocal motor cortex (RA, Figure 1) send transient axon collaterals into the shell pathway, such that a copy of the motor signal is conveyed into the shell circuit only in juvenile birds (Miller-Sims and Bottjer, 2012) (Figure 1—figure supplement 3). Remarkably, the large population of tutor-tuned neurons in LMAN-shell is gone by late stages of sensorimotor learning, suggesting a key role for this subpopulation during a restricted period of development. These developmental changes suggest that the shell pathway may evaluate feedback about current vocal performance in relation to a memory of the tutor song during early sensorimotor learning and transmit that evaluation to core and other motor pathways (Achiro and Bottjer, 2013; Bottjer et al., 2010). We evaluated this idea by testing the activity of core and shell neurons in singing juvenile birds as they are actively involved in sensorimotor learning. The results support the idea that core neurons participate in motor-related actions and that activity in both core and shell neurons reflects evaluation of feedback of motor performance against the goal of memorized tutor sounds.
We recorded single neurons in core and shell subregions of LMAN in juvenile zebra finches that had completed memorization of a tutor song and begun to practice their song vocalizations (43–60 dph). The majority of neurons in both core and shell showed significant modulation of firing rate during singing (Table 1; see Materials and methods). Among neurons that exhibited a significant change in firing rate during song production, approximately 65–70% showed excitation in both core and shell whereas the remainder were suppressed. Excitatory response strength was marginally higher in shell neurons, whereas suppressed response strength did not differ between core and shell (Table 1; Mann-Whitney tests: singing-excited neurons U = 1768, p=0.06, singing-suppressed neurons U = 459, p=0.90). Figure 2 shows spiking activity of example core and shell neurons that exhibited average levels of excitation (top) or suppression (bottom) during singing. Singing-excited neurons in both core and shell showed a significant increase in bursting during singing (fraction of spikes with interspike intervals <10 ms; Figure 2—figure supplement 1) as shown previously for LMAN-core neurons in juvenile birds (Olveczky et al., 2005).
Firing rates of neurons in core and shell were also similar during production of different syllable types and lacked temporal specificity across renditions of the same syllable type (Figure 2—figure supplement 2). Thus, spiking was relatively sparse and variable within and across syllable types in both core and shell neurons, as reported previously in core for both juvenile and adult birds during playback and singing (Doupe, 1997; Doupe and Solis, 1997; Kao et al., 2008; Olveczky et al., 2005; Solis and Doupe, 1997). In summary, the basic profile of neural activity during song production was similar between shell and core neurons.
Despite these overall similarities in neural activity between core and shell, the population of shell neurons would not be expected to show a coordinated increase in firing rate prior to the onset of singing if the shell pathway lacks a role in vocal motor production (Bottjer and Altenau, 2010). In accord with this idea, population histograms of mean-subtracted responses in pre-singing excited neurons showed synchronous increases in firing rates prior to syllable onsets in core neurons, whereas shell neurons revealed no evidence for coordinated pre-singing increases in firing rate (Figure 3). The lack of time-locked premotor activity in shell neurons is consistent with the absence of motor abnormalities following lesions that disrupt shell circuitry (Bottjer and Altenau, 2010), indicating a non-motor role in learning.
A large proportion of shell neurons are specifically tuned to playback of either the tutor song or the current version of self-generated song during early stages of sensorimotor integration when core neurons drive vocal motor output and lesions of LMAN pathways cause disruption of song learning (Achiro and Bottjer, 2013; Aronov et al., 2008; Bottjer et al., 1984; Olveczky et al., 2005; Scharff and Nottebohm, 1991; Bottjer and Altenau, 2010). This pattern suggests that LMAN is involved in sensorimotor learning in juveniles, but direct support is lacking for the idea that the activity of individual LMAN neurons represents how well syllables match the tutor song during singing. We tested this idea in juvenile birds (43–60 dph) by comparing neural activity during singing with the acoustic similarity of self-generated syllables to tutor syllables (see Materials and methods): regressions of baseline-corrected firing rates against tutor similarity were performed for each neuron. Unexpectedly, this analysis revealed that firing rates of cells in both core and shell could either increase or decrease as a function of tutor similarity: approximately half of all neurons in each subregion showed increased firing rates for syllables with higher tutor similarity (positive slopes, r values > 0), whereas the other half showed increased firing rates for syllables with lower tutor similarity (negative slopes, r values < 0) (Table 2). Figure 4A shows examples of syllable utterances with relatively high or low acoustic similarity to the closest tutor syllable, along with the corresponding firing rate of a shell neuron that showed a negative correlation with tutor similarity (Figure 4B). This neuron showed a low firing rate during production of syllables with high tutor similarity and excitation during production of syllables with low tutor similarity. Thus, LMAN neurons can encode similarity between self-generated utterances and tutor song via either increases or decreases in firing rate.
To estimate the fraction of single neurons showing a significant relationship between firing rate and tutor similarity, we performed repeated permutation tests by generating 1000 random shuffles of the relationship between firing rate and tutor similarity for each cell (O'Connor et al., 2010); see Materials and methods). The percent of significant neurons compared to these random distributions was 5.5% in core and 10.8% in shell (Table 2) (chi-square test between core and shell proportions = 1.98, p=0.18). core and shell neurons were evenly split between positive and negative associations of firing rate to degree of tutor song matching, and had fairly comparable r values (averages ranging from +0.22 to −0.31). In summary, tutor similarity predicted the firing rates of a relatively small percentage of single neurons in both core and shell, with somewhat more neurons in shell showing a significant correlation.
To test whether similarity to tutor syllables modulates neural activity at the population level, we analyzed the regressions of baseline-corrected firing rates with tutor similarity across all neurons using a mixed-effects linear regression model (fixed and random effects for tutor similarity nested within a random intercept for neurons; see Materials and methods). This analysis yielded a significant effect for shell neurons (t = −2.23, p=0.035) but not for core neurons (t = 0.91,p=0.37), indicating that the population activity of shell neurons during singing reflected tutor-matching performance. Because positive or negative relationships between firing rate and tutor similarity may reflect unique processes of evaluating tutor similarity (Table 2), we examined these two categories separately. Using the mixed-effects linear regression model to provide a descriptive assessment of the magnitude of positive and negative associations, we found significant effects for positive and negative relationships between firing rate and tutor similarity in both core and shell neurons (core positive: t = 3.42, p=0.003; negative: t = −2.90, p=0.004; shell positive: t = 4.02, p=0.001; negative: t = −2.80, p=0.009). To illustrate these relationships, Figure 5 shows response strengths during production of syllables with low versus high tutor similarity (bottom versus top 50% ranked by tutor similarity; Materials and methods) separated by whether cells showed positive (left panels) or negative (right panels) slope values. These data indicate that the majority of neurons in both core and shell showed either higher or lower firing rates during production of syllables with higher tutor similarity.
An alternative interpretation of the significant association between firing rate and tutor similarity in shell is that firing rates were modulated by more prototypical juvenile utterances, that is, those closer to the center of the distribution of acoustic features for a given syllable type (see Materials and methods). We tested whether the significant correlation to tutor similarity we observed in shell neurons reflected a tendency to encode highly prototypical syllables using the mixed-effects regression model described above. This analysis showed a non-significant effect (t = −1.53, p=0.136), and there was no relationship between indices of tutor similarity and prototypicality (data not shown), indicating that tutor-similar syllable renditions are not more prototypical. Thus, the relationship between firing rates and tutor similarity in shell neurons does not appear to be based on prototypicality of self-generated syllables.
To test if degree of similarity to tutor song also modulates firing rate variability of LMAN neurons in 43–60 dph birds, we calculated the CV (coefficient of variation) of firing rate during production of syllable renditions with low versus high similarity to tutor syllables (bottom versus top 50% of tutor similarity scores). Both core and shell neurons showed increased firing rate variability during production of syllable utterances with low tutor similarity compared to those with high similarity (Figure 6A; Wilcoxon signed-rank tests, core Z = −5.70, p<0.001; shell Z = −4.48, p<0.001). Thus, variability of firing rates during singing reflected the degree of tutor song matching in both core and shell neurons such that responses were less variable for syllable renditions that were more similar to tutor song.
One possible caveat is that the decreased variability of firing rate for utterances with higher tutor similarity might simply reflect a smaller number of syllable types in this category, whereas low-similarity utterances might include many different syllable types. To test this idea, we assessed the fraction of syllable types (across all syllable renditions) that were produced within the high- versus low-similarity categories for each day of singing and found no significant difference (Figure 6B left; Mann-Whitney U = 693, p=0.06). In addition, there was no difference in the scatter of syllable renditions within a syllable type between the high- and low-similarity to tutor categories (Figure 6B right; Mann-Whitney U = 867.5, p=0.90). In summary, these data indicate that variability in firing rate was higher across the populations of both core and shell neurons during production of syllable renditions with low tutor similarity compared to production of syllable renditions with high tutor similarity.
The developmental span over which we recorded birds’ vocal behavior (43–60 dph) corresponds with a transition from highly variable ‘subsong’ to ‘plastic song’ with less behavioral variability and a corresponding increase in similarity to tutor song. If the modulation of neural activity by tutor similarity we observed above reflects some aspect of learning, then it should change during the progression of learning. To test this idea, we assessed the degree of song development for each day of singing for each bird by calculating goodness-of-fit coefficients from an exponential fit to the distribution of syllable durations (see Materials and methods; Figure 7—figure supplement 1). Juveniles in subsong (but not plastic song) produce a graded distribution of syllable durations, which is well-fit by an exponential function (lower numbers indicate better fit; (Aronov et al., 2011; Tchernichovski et al., 2004; Aronov et al., 2008; Johnson et al., 2002). Comparing degree of song development with acoustic similarity to tutor across average syllable types yielded a significant correlation (Figure 7A; r = 0.23, p=0.03), supporting the canonical understanding that juvenile birds produce syllable renditions that are more similar to tutor syllables as sensorimotor integration progresses.
We then examined individual syllable utterances with high versus low tutor similarity within each day of singing. Surprisingly, we found that birds produced many utterances with relatively high tutor similarity during subsong (Figure 7B, right). Furthermore, the level of tutor matching among these utterances was fairly constant across development such that there was no significant correlation between tutor similarity and song development (r = 0.13, p=0.42). This striking finding indicates that some of the highly variable vocalizations produced by birds during early sensorimotor learning are good matches to tutor syllables. Birds in subsong also produced many low-similarity utterances (Figure 7B, left), and the similarity between these poorly-matched renditions and tutor syllables increased as song development progressed, reflecting the expected increase in tutor matching (r = 0.34, p=0.03; cf. Figure 7A). Thus, birds in subsong produce some utterances with tutor similarity as high as that of birds in later stages of song development, and renditions with lower tutor similarity at the beginning of sensorimotor integration are gradually eliminated.
These data raise the possibility that syllable renditions with high versus low tutor similarity (Figure 7B) were distinct in other ways. For example, high-similarity syllable renditions in subsong birds might consist of simple harmonic stacks, whereas low-similarity renditions might consist of syllable types with complex modulations. We examined whether individual syllable types were selectively represented in high- versus low-similarity categories for each day of singing, and found that the syllable type that was produced most frequently in the high-similarity category (highest percentage of renditions among all syllable types) was also produced most frequently in the low-similarity category. The most common syllable type produced was the same for both high- and low-similarity renditions in 57% of vocal recording sessions across all birds, indicating that a single syllable type did not typically predominate in either low- or high- similarity categories. This pattern is consistent with the finding above that syllable renditions within both high- and low-similarity categories represent multiple syllable types (Figure 6B). In addition, qualitative examination revealed that syllable types with the most renditions in high- and low-similarity categories included both simple and complex syllables, although we noted a slight tendency for low-similarity renditions to include more frequency modulation. Thus, syllable renditions with high versus low tutor similarity did not show a strong pattern of syllable type.
How did neural activity in LMAN encode these changes in tutor similarity as learning progressed? Absolute values of response strength across song development increased in both core and shell neurons regardless of whether syllable renditions had high or low similarity to tutor syllables (data not shown). This pattern agrees with previously published work showing increases in firing rate of RA neurons as song learning progresses (Ölveczky et al., 2011), suggesting generic increases in firing rate with development. However, LMAN circuitry is critical for early sensorimotor learning when the number of tutor-tuned shell neurons is high (Achiro and Bottjer, 2013), indicating that the involvement of LMAN may be high initially when the goal representation is strong and decrease as learning progresses. In accord with this idea, correlations between firing rate and tutor similarity across development revealed that the activity of LMAN neurons tracked vocal performance during sensorimotor learning (Figure 8). Both core and shell neurons showed stronger associations between firing rate and tutor similarity during early stages of song development, and weaker associations as sensorimotor learning progressed. These trends were significant in core and shell neurons with positive associations and in core neurons with negative associations; shell neurons with negative slopes showed a weak but non-significant association between firing rate and tutor similarity across development. However, when the two bottom outliers in the panel for SHELL-negative slopes were removed the association was significant, r = 0.35, p=0.006. This pattern indicates that the activity of both core and shell neurons reflects the degree of song learning: tutor similarity is a better predictor of firing rate during early sensorimotor learning when many utterances have low tutor similarity (Figure 7B), and the strength of this association decreases in parallel with developmental increases in motor performance as song learning progresses.
The variability of firing rate paralleled the behavioral changes in tutor similarity across development (Figure 7B) in shell but not core neurons. CV of firing rate in shell neurons decreased over the course of song development only during production of syllable renditions that had low similarity to tutor during early sensorimotor learning (Figure 9; low similarity, r = −0.24,p=0.01; high similarity, r = −0.08, p=0.42). Thus, variability of firing in shell neurons did not change for utterances with high similarity to tutor song across development. In contrast, the CV of firing rate in core neurons did not change as a function of song development regardless of whether syllable renditions had high or low tutor similarity (Figure 9; low similarity,r = 0.04, p=0.73; high similarity, r = 0.05, p=0.61). These data show that the variability of firing in shell neurons reflected the progression of learning, with higher neural variability early in development for renditions with lower similarity to tutor song, and lower neural variability later in development as the incidence of poorly-matched syllable renditions decreased.
We tested whether spiking activity in LMAN of juvenile birds reflects the acoustic similarity of self-generated vocal utterances to memorized tutor syllables. Neurons in both core and shell subregions of LMAN showed singing-related activity that varied as a function of tutor song similarity during early stages of sensorimotor integration. This pattern represents the first discovery of an online correlate of song performance during learning in juvenile birds and suggests LMAN as a key component of circuitry that evaluates current behavior in relation to a goal behavior during procedural learning. Although neural activity across the population in both core and shell was modulated by degree of similarity between self-generated sounds and memorized tutor sounds, this trend was stronger across the population of shell neurons. As in prior work, we found that spiking activity in core neurons supports the idea that core drives vocal motor output in juvenile birds (Aronov et al., 2008). In contrast, neural activity in shell neurons did not exhibit coordinated premotor increases in firing rate, which is consistent with the lack of projections from shell to downstream motor circuitry and the absence of motor disruption following lesions to the shell pathway (Bottjer and Altenau, 2010; Bottjer et al., 2000) (Figure 1).
One interpretation of this pattern of results is that shell circuitry acts primarily as a ‘critic’ to evaluate comparisons between self-generated and tutor sounds, while core circuitry serves as an ‘actor’ that drives vocal motor output and receives instruction from the critic in order to bias action selection over the course of learning (Barto et al., 1983; Graybiel, 2008). The idea of an instructive function by ‘critic’ circuitry is complicated (here as elsewhere), since the recurrent loop architecture of cortico-basal ganglia circuits means that error or reinforcement signals (which are themselves widely distributed) may be propagated into multiple pathways (Lau et al., 2017). Thus in the current results it is difficult to know whether evaluative signals may originate in shell and be transmitted to core in order to instruct future motor actions. We favor this idea as a working model; the current data serve as a foundation for tests of this idea in future studies.
Interestingly, some individual syllable renditions were similar to tutor syllables even in subsong birds during early sensorimotor integration, indicating a surprising capacity to produce relatively mature syllabic utterances. This result is reminiscent of the finding that juvenile birds exposed to adult females are able to produce more stereotyped song patterns than normally seen in young birds (Kojima and Doupe, 2011). The ability to produce syllable renditions that represent a range of matches to the goal (i.e. both higher and lower similarity) may be an important component of the behavioral variability that is necessary for skill acquisition involving reinforcement learning.
Only syllable utterances with low levels of tutor matching during subsong showed a progressive increase in tutor similarity over the course of development, and the activity of shell but not core neurons reflected this difference in behavioral trajectory. Developmental changes in firing rate variability mirrored that seen for behavioral development only in shell (Figures 7 and 9): firing rate CV in shell neurons decreased throughout development only among utterances that showed an increase in similarity over the course of learning. In contrast, variability of firing in core neurons did not change developmentally. The decrease in variable firing in shell neurons in parallel with the decrease in incidence of poorly matched syllable renditions indicates that the activity of shell neurons tracks the degree of song matching, consistent with the hypothesis that these neurons are involved in comparing song behavior to a tutor song memory.
As vocal behavior was refined to achieve a more accurate imitation of tutor song, the correlation between tutor similarity and firing rate decreased in both core and shell neurons (Figure 8). The tendency toward weaker encoding of tutor similarity by firing rate as song learning progresses coincides with the loss of a large population of shell neurons that respond selectively to playback of tutor song (Achiro and Bottjer, 2013). Thus, tutor similarity is a better predictor of firing rate during early learning when behavioral variability is high and shell contains a large population of tutor-selective neurons. The loss of tutor-tuned neurons along with a progressive weakening of the association between firing rate and tutor similarity suggests that birds may be switching to a different strategy such as comparing their own utterances to a ‘self’ template more than to a tutor template. In accord with this idea, neural selectivity for self-generated sounds increases in core neurons between 45 and 60 dph (Achiro and Bottjer, 2013; Doupe, 1997; Doupe and Solis, 1997; Solis and Doupe, 1997). These patterns raise an important question: do individual shell neurons in which firing rate reflected tutor similarity (10.8%, Table 2) fall within the population of tutor-tuned neurons? Tutor-tuned neurons in juvenile LMAN-shell may act as a gate or filter for self-generated utterances that are tutor-similar, suggesting that evaluation of tutor similarity within LMAN may be selectively vested in the firing rate of neurons within this subtype during early sensorimotor integration.
Tutor similarity could be encoded by either increases or decreases in firing rate for both core and shell neurons. This intriguing aspect of the current results is subject to different interpretations. One possibility is that increases in firing rates for syllables with higher tutor similarity (positive slopes) could provide a reinforcement signal to increase the probability of producing that vocal pattern, whereas increases in firing rates for syllables with lower tutor similarity (negative slopes) could provide an error signal to decrease the probability of making that incorrect sound; both types of information could be conveyed to downstream neurons and used to guide accurate refinement of syllables during learning.
The construction of cortical circuits has been studied extensively in primary sensory cortex of mammals (Katz and Shatz, 1996; Espinosa and Stryker, 2012; Levelt and Hübener, 2012), but few studies have probed developmental changes in cortico-basal ganglia circuits as part of the mechanisms of skill learning. The involvement of LMAN circuitry in goal-oriented learning is likely to be strongly dependent on developmental changes that occur in these pathways. In juveniles, but not adults, many core neurons that project to vocal motor cortex send a collateral branch into the shell pathway (Figure 1—figure supplement 3) (Miller-Sims and Bottjer, 2012). This transient pathway may transmit a copy of the premotor signal generated by core neurons to shell neurons. In addition, the incidence of shell neurons tuned to learned tutor sounds decreases sharply (Achiro and Bottjer, 2013). Because the overall volume of shell regresses sharply during development, it may be that tutor-tuned neurons are eliminated due to naturally-occurring cell death (Johnson and Bottjer, 1992; Johnson et al., 1995), thereby helping to close the sensitive period for learning. Topographic specificity develops in the projection from LMAN core to RA (vocal motor cortex) during early development and is dependent on normal auditory experience (Iyengar and Bottjer, 2002b). In addition, a high proportion of synapses formed by thalamic axons in LMAN are ‘silent synapses’ in early development (Bottjer, 2005); a decrease in their number may contribute to a period of synaptic refinement to confer greater specificity of connectivity in thalamo-cortical connections (Iyengar and Bottjer, 2002a; Nixdorf-Bergweiler, 2001); cf. Garst-Orozco et al., 2014) as well as to curtail the sensitive period (Huang et al., 2015). These developmental changes suggest that differences in the extent and plasticity of neural mechanisms may be causally related to goal-oriented learning during development, and help to explain decreases in behavioral plasticity in older animals.
A functional segregation of vocal learning into parallel core and shell cortico-basal ganglia loops is reminiscent of corresponding architecture in sensorimotor and associative cortico-basal ganglia loops of mammals. Studies in mammals have shown that neurons in associative loops show increased modulation early in learning of goal-directed tasks, whereas sensorimotor circuits increase their activity throughout training and may encode learned motor performance (Joel and Weiner, 1997; Histed et al., 2009; Yin et al., 2009; Thorn et al., 2010; Gremel and Costa, 2013; Kim et al., 2013; Samejima and Doya, 2007; Graybiel, 2008; Yin et al., 2008; Ashby et al., 2010; Redgrave et al., 2010; Ito and Doya, 2015; Lehéricy et al., 2005; Thorn and Graybiel, 2014; Atallah et al., 2007; Nakahara et al., 2001; Parent and Hazrati, 1995). Such evidence has suggested that associative loops function to evaluate motor performance during early stages of learning and sensorimotor loops encode behaviors as they become more habitual (Makino et al., 2016).
The overall patterns of neural activity we observed in shell and core are consistent with recent data showing that both associative and sensorimotor cortico-striatal circuits are engaged in skill acquisition, but that associative circuits disengage early in learning whereas sensorimotor circuits remain engaged (Kupferschmidt et al., 2017). The decline in spiking variability in shell neurons is consistent with a decreased involvement of associative circuits during early skill acquisition (Thorn et al., 2010; Yin et al., 2009). In addition, although variability of firing rate did not change developmentally in core neurons, firing rate CV was higher for syllable renditions with low tutor similarity in core neurons when averaged across all ages (Figure 6). This pattern suggests that core neurons may retain an exploratory (variable) mode (Graybiel, 2005), maintaining a higher CV for poorly-matched syllable renditions and a lower CV for well-matched renditions (Figure 6). Such an outcome would be consistent with the idea that core neurons retain some level of engagement at later stages of sensorimotor learning. Overall, the present results support the idea that motor learning across taxa entails the integrative product of multiple circuits whose functions reflect specific aspects during the progression of learning.
All procedures were performed in accordance with Protocol 9159 approved by the University of Southern California Animal Care and Use Committee and in accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health. Ten juvenile male zebra finches (Taeniopygia guttata) were used, 43–60 days post hatch (dph) at the time of neural recordings (30 min recording sessions were collected across this age span on an average of 4 different days for each bird; thus each neuron represents a single 30 min recording session; see below). The period of sensorimotor integration begins in zebra finches when young males produce variable babbling sounds (subsong) starting at approximately 35 dph, and continues until the birds begin to produce a stereotyped imitation of the tutor song around 70–90 dph. Birds were raised under naturalistic conditions (by their parents within group aviaries) until they were at least 38 dph, ensuring normal tutor song exposure and social experience (Böhner, 1990; Immelmann, 1969; Roper and Zann, 2006; Böhner, 1983; Eales, 1985; Clayton, 1987; Mann and Slater, 1995; Chen et al., 2016).
Birds were anesthetized with isoflurane (1.5% inhalation) and an electrode assembly consisting of seven tungsten-wire stereotrodes was implanted into LMAN core and shell; each stereotrode consisted of two twisted-pair polyesteramide-insulated and imide-overcoated tungsten wires (diameter 25 μm, California Fine Wire Company, Grover Beach, CA) routed through fused silica capillary tubing. One stereotrode was implanted 1.5 mm dorsal to LMAN to serve as a reference electrode, and silver wire (diameter 250 µm) was placed between the skull and skin for the animal ground. Signals were acquired through a unity gain headstage (Neuralynx, Bozeman, MT) with a flexible cable that connected the electrode assembly to a commutator (Neuralynx) and two 8-channel amplifiers (Lynx8, Neuralynx). Vocalizations were recorded through a microphone mounted in the cage (Sanken COS-11D) and saved coincident with neural activity (band passed 300–6000 Hz; digitized at 32 kHz; Spike two software, Cambridge Electronic Design, UK). At the end of each experiment, small electrolytic lesions (7 µA for 20 s) were made to confirm recording locations. To verify the borders of LMAN core and shell, 50-µm-thick coronal sections were Nissl-stained or stained with a monoclonal antibody against calbindin-D-28K (Sigma-Aldrich Cat# C9848, RRID:AB_476894) using standard immunohistochemical procedures; calbindin expression specifically labels thalamic axons terminating in LMAN (Pinaud et al., 2007; Achiro and Bottjer, 2013) (Figure 1). The border between core and shell subregions of LMAN can be distinguished based on the density of magnocellular somata, and the outer borders of shell can be distinguished from surrounding regions based on the limit of calbindin expression (Figure 1). Recording sites were considered for analysis if they were confirmed histologically to be in either core or shell, excluding 50 µm on either side of the core-shell border (see Achiro and Bottjer, 2013).
Single units were isolated offline from stereotrode recordings made during each 30 min session using KlustaKwik (Ken Harris, Rutgers University) for automatic clustering as described previously (Achiro and Bottjer, 2013), and refined manually using MClust (A. David Redish, University of Minnesota) in MATLAB (Mathworks, Natick, MA, California). Single units were included if signal to noise ratio was >3 and if less than 1% of spikes had an interspike interval <2 ms (n = 127 neurons from core, 171 neurons shell). To compute an appropriate sample size, we used preliminary electrophysiological data to measure average response strength that was different from zero during singing (including all neurons regardless of whether they showed a significant response during singing by t-test, see below). One-sample, two-tailed power analysis indicated we would need a sample size of 97 core neurons and 45 shell neurons in order to determine that neurons in each subregion showed a significant response during singing at 90% power with a 0.05 two-sided significance level:
where n is sample size, µ is mean, µ0 is 0, σ is standard deviation, α is Type I error and 1 – β is power.
Analyses of neural activity were based on all neurons from the total of all daily 30-min recording sessions across birds and days (each neuron was recorded for 30 min on 1 day). Baseline periods were defined as periods of silence (non-singing) lasting at least 2 s that were 2 s or more away from singing, calls or movement/cage noise. A neuron was considered responsive if the average firing rate (spikes/s) during singing showed a significant change from the average firing rate during baseline (independent t-test, due to differing number of baseline and singing episodes, p<0.05). Neurons in both core and shell showed significant modulation of firing rate during singing and/or the 50 ms interval prior to syllable onsets compared with quiet baseline periods (core: 81%, 103/127 neurons; shell: 82%, 141/171 neurons).
Analysis of pre-singing related activity was restricted to cells that showed a significant increase in average firing rate between baseline periods and the 50 ms prior to onsets of all syllables (independent t-tests, p<0.05). The proportions of CORE versus SHELL neurons showing an increase in average firing rate during this 50 ms period did not differ [CORE 48% (61/127 neurons), SHELL 40% (68/171 neurons); chi-square test = 2.27, p=0.13]. To analyze the temporal pattern of firing leading up to syllable onsets across these neurons, histograms of population activity were made by calculating the mean-subtracted firing rate for each neuron: the average firing rate was calculated during ±200 ms surrounding syllable onsets in bin sizes of 2 ms and the rate during each bin was subtracted from the mean firing rate over all bins for each cell before smoothing with a Gaussian (40 ms smoothing); population functions were generated by averaging across neurons (Goldberg and Fee, 2012). To determine significance, we used the inverse student t cumulative distribution function to get a t-statistic for 95% probability. We calculated the two-tailed critical value of the t distribution for α = 0.05 with degrees of freedom equal to the number of neurons minus one; a bin of the population histogram was deemed significant if the critical value of the t distribution multiplied by the population histogram ±the s.e.m. for that bin was less than or greater than zero.
To compare changes in activity during singing across neurons with differing firing rates, responses for each neuron were calculated as standardized response strength:
where S is the firing rate during singing, B is the firing rate during local baseline periods (the average of the two baseline periods nearest in time to each singing episode), and n is the number of singing/baseline pairs. A positive value indicates an increased firing rate during singing, and a negative value indicates a decreased firing rate during singing compared to baseline periods. We refer to this standardized measure throughout the text as response strength. To measure the incidence of bursts during singing episodes and local baselines, we calculated a burst fraction by measuring the percentage of spike events with an interspike interval of less than 10 ms (Wilcoxon signed-rank tests, due to one average local baseline period per singing episode,p<0.05).
Because juvenile birds in early stages of song learning do not produce stereotyped syllable sequences (song motifs), episodes of singing were defined as periods of continuous singing separated by gaps of at least 300 ms. Episodes of singing and individual syllables contained within episodes were detected automatically using amplitude threshold crossings and checked manually to remove cage noise and to adjust syllable start or stop boundaries where needed.
Juvenile birds produce highly variable sequences of syllables, making it impossible to align the temporal pattern of neural activity across song motifs (Tchernichovski et al., 2001). However, one can align spiking activity of single neurons to multiple renditions of individual syllables of the same type. Because of the high variability of juvenile syllables, we created custom software in MATLAB using many features created for Sound Analysis Pro (Tchernichovski et al., 2000) to automatically classify syllables in juvenile birds, available online (Shen, 2017; https://github.com/BottjerLab/Acoustic_Similarity. A copy is archived at https://github.com/elifesciences-publications/Acoustic_Similarity). To assign syllables to different types, we employed a combination of two measures of the acoustic distance between syllables that was then used to cluster syllables.
The first distance measure was based on summary statistics of the following syllable features: amplitude modulation, frequency modulation, center frequency, fundamental frequency, length, maximum frequency modulation within each frequency, maximum amplitude modulation across each frequency’s power, pitch goodness (estimate of the amount of periodic energy), total power, derivative of total power, and Wiener entropy (estimate of spectral disorder). We calculated the following summary statistics for each feature (except length) over the duration of each syllable: mean, standard deviation, maximum, minimum, onset (average of samples 1–3%), middle (average of samples 49–51%), offset (average of samples 97–100%), correlation coefficient of the linear trend, time of maximum, and time of minimum. Each syllable was thus represented as a point in high-dimensional space where each dimension was a summary statistic of one of the features (n = 101 feature values total). We then calculated the Euclidian distances between each point.
The second distance measure was based on time-varying changes in song features; we calculated the following syllable features for each time point in the syllable (9.27 ms window size, 7.91 ms overlap): Wiener entropy, frequency modulation, amplitude modulation, fundamental frequency and goodness of pitch (Tchernichovski et al., 2000)(Figure 10A). Syllables were represented as multi-dimensional feature vectors in time. The distance between the vectors was calculated with a dynamic time-warping algorithm including a warping penalty (Vintsyuk, 1972). The time-warping algorithm tolerates small perturbations in the timing of sub-syllabic events common to juvenile syllables.
In order to normalize the measures, we converted each distance measure to percentiles based on the empirical distribution (computed over all pairs of syllables). To obtain a combined distance measure we calculated the geometric mean of the two percentile distance measures for each syllable pair. We then generated a matrix consisting of the combined distance measure for each syllable to all other syllables. A final distance score was calculated as a dissimilarity index by taking one minus the correlation between points of the combined distance measure matrix, and ranged between 0 (perfect similarity) and 2 (no similarity; i.e., if the syllables were completely anticorrelated, the correlation would be −1 and the distance would be 1 - (−1)=2). Syllables were clustered into 4–25 types by hierarchical agglomerative clustering using the dissimilarity index as the distance metric and complete linkage as the linkage criterion (Figure 10B). We then manually selected the number of types for each bird for each day, confirmed cluster quality, and merged clusters which were similar. Some clusters were rejected due to high variability; thus not all syllables were clustered into types, especially those from recordings of birds producing more immature vocalizations. The total percent of classified syllables (assigned to clusters) was 49% overall (across song recordings for all birds and ages); 38% of syllables were classified from song recordings in early stages of development (representing the bottom half of song recordings by goodness-of-fit coefficients of exponential fit to syllable duration distributions, see ‘Song development’ below) and 57% of syllables were classified for song recordings in later stages of song development (representing the top half of song recordings by goodness-of-fit coefficient).
We calculated firing rates during each syllable produced and the 50 ms prior to syllable onsets in order to include pre-syllable-related activity; this syllable-based firing rate was used in analyses of neural selectivity for syllable types and similarity to tutor song (see below). The average gap between syllables in juvenile birds is ~60 ms (Glaze and Troyer, 2013; Aronov et al., 2011), making it unlikely that activity during a previous syllable was included in these firing rates. In order to align syllables to construct PSTH’s of spiking activity, we linearly time warped each spike train to the average length for that syllable type following procedures of (Kao et al., 2008). We calculated the average duration of all syllables within each cluster and used that value as the ‘reference duration’. Then, each syllable in the cluster was linearly stretched/compressed (syllables were aligned at onsets) and spike trains were projected onto the time-warped axis for each syllable. To determine selectivity of responses to specific syllable types, we calculated a sparseness/activity fraction (AF) (Meliza and Margoliash, 2012; Vinje and Gallant, 2000) for all neurons which responded significantly to at least one syllable type:
where r is the firing rate to the ith syllable type and n is the number of syllable types. Thus a score of 0 indicates no selectivity for a specific syllable type (equal firing rates across all syllable types) and one indicates maximum selectivity (change in firing rate during only one syllable type).
Analyses of similarity to tutor song included all neurons for which at least 40 classified syllables (syllables which were able to be clustered into a given type) were produced during each neuron’s 30-min recording period. We used only classified syllable renditions for the tutor similarity analyses in order to use the same set of neurons and syllables for an analysis of prototypicality as a control (see below). We tested whether using only classified syllables influenced the tutor similarity results by examining the outcome using all syllables (both classified and not classified) and observed the same trends (data not shown); thus, use of classified syllables did not bias the results. These analyses included both singing-excited and singing-suppressed neurons, as well as neurons that did not show a significant change in firing rate during singing episodes (the latter cases ensured inclusion of cells that showed excitation during syllables with high similarity and suppression during syllables with low similarity, for example). To evaluate the similarity between juvenile syllables and tutor syllables we employed the final combined distance score described above. Similarity to tutor was calculated as the acoustic distance score between each syllable and its closest tutor syllable. We defined what constituted the closest tutor syllable in two ways: either as the tutor syllable which was closest in distance to the center of the syllable cluster to which each rendition belonged, or as the tutor syllable which was simply closest in distance to each rendition. These two approaches yielded highly similar results, and we therefore chose the latter method to reduce any effects of the syllable clustering routine on tutor similarity calculations. All syllable renditions produced during each neuronal recording were ranked by tutor similarity; for many analyses, a median split was employed in which the ranked juvenile syllables for each neuron were divided into the top 50% versus bottom 50% of tutor similarity – these two categories are referred to as high and low similarity, respectively (Figure 10—figure supplement 1).
For each neuron, we calculated the linear regression between baseline-corrected firing rate during each syllable rendition and similarity to closest tutor syllable (Figure 4B). We used repeated permutation tests to estimate the fraction of neurons with significant correlations (O'Connor et al., 2010): for each neuron, we generated 1000 permutations by randomly shuffling the relationship between firing rate and tutor similarity and calculating the resultant r values. This provided a distribution of values under the null hypothesis of no relationship for each test. The fraction of neurons exceeding chance for each permutation test was computed as the fraction of actual r values falling above the. 975 percentile (positive correlations) or below the. 025 percentile (negative correlations) of the null distribution, and the average of this significant fraction across all permutation tests was taken as the fraction of neurons with significant r values.
To assess the association of baseline-corrected firing rates on tutor similarity at the population level, we employed a mixed-effects linear regression model of baseline-corrected firing rates across all neurons with fixed and random effects for tutor similarity nested within a random intercept for neurons using an unstructured covariance matrix; because firing rate was measured at multiple tutor similarities within each neuron, our model fit a random intercept for neurons, with tutor similarity treated as both a fixed and random slope.
To examine if firing rate variability was modulated by similarity to tutor song, we calculated the CV of firing rate during production of syllables representing high and low similarity to corresponding tutor syllables (top and bottom 50% of syllables ranked by tutor similarity). Neurons were included in these analyses if at least 40 classified syllables were produced during the 30-min recording period and if the mean firing rate was non-zero for the most/least prototypical and most/least similar to tutor song syllables (because neurons with mean firing rates of zero would give undefined CV values).
We calculated a prototypicality score for each syllable based on (Niziolek et al., 2013), which measures whether renditions are similar to the center of that syllable’s distribution (more prototypical) or less similar to the center of the distribution (less prototypical). We computed the acoustic distance between each syllable rendition and the center of the syllable cluster to which it belonged, again employing the combined distance score described above. This measure served as a control for effects based on tutor similarity to assess whether correlations based on firing rate reflected prototypical utterances; we used the same mixed-effects regression model as above to assess a relationship between baseline-corrected firing rates and prototypicality. As indicated above, this analysis also included all neurons for which at least 40 classified syllables were produced during individual recording periods; in this way analyses of tutor similarity and prototypicality included the same set of neurons and syllables.
In order to assess how developed each bird’s song was for each day, we utilized methods previously described to define song stage (Aronov et al., 2011). Juvenile birds in the subsong stage of sensorimotor integration produce syllables of variable lengths, the distribution of which is well-fitted by an exponential function. As birds progress to the plastic song phase, they produce more regular syllable types which begin to appear as peaks in the distribution of syllable durations, and therefore are no longer well-fit by an exponential (Aronov et al., 2011; Tchernichovski et al., 2004). Based on this evidence, we fit an exponential to all syllable duration distributions for each bird, for each day of singing (500–8,000 syllables). We used the Lilliefors test (MATLAB) to quantify goodness-of-fit, and the resulting coefficient was scaled by the number of syllables. Therefore, smaller goodness-of-fit coefficients indicate less developed songs (i.e. subsong) and larger coefficients indicate more developed songs (i.e. plastic song). Response strength, CV, and r values (correlations between baseline-corrected firing rate and tutor similarity) for all neurons across all classified syllables (syllables assigned to clusters) produced in each recording session were analyzed as a function of song development using goodness-of-fit coefficients.
Kolmogorov-Smirnov and Shapiro-Wilk tests were used to test for normality; t-tests were used to compare means for normally distributed data, and Mann-Whitney tests were used for non-normally distributed data. Differences in proportions were tested using chi-square tests. Correlations were performed using Pearson’s correlation. The significance of mean-subtracted bins for pre-singing activity (Figure 3) was calculated as the 95% confidence interval outside of zero and is described above. Specific statistical tests used are identified in context in the Results.
Neural representation of a target auditory memory in a cortico-basal ganglia pathwayJournal of Neuroscience 33:14475–14488.https://doi.org/10.1523/JNEUROSCI.0710-13.2013
Functional architecture of basal ganglia circuits: neural substrates of parallel processingTrends in Neurosciences 13:266–271.https://doi.org/10.1016/0166-2236(90)90107-L
Cortical and basal ganglia contributions to habit learning and automaticityTrends in Cognitive Sciences 14:208–215.https://doi.org/10.1016/j.tics.2010.02.001
Neuronlike adaptive elements that can solve difficult learning control problemsIEEE Transactions on Systems, Man, and Cybernetics SMC-13:834–846.https://doi.org/10.1109/TSMC.1983.6313077
Parallel pathways for vocal learning in basal ganglia of songbirdsNature Neuroscience 13:153–155.https://doi.org/10.1038/nn.2472
Connections of a motor cortical region in zebra finches: relation to pathways for vocal learningThe Journal of Comparative Neurology 420:244–260.https://doi.org/10.1002/(SICI)1096-9861(20000501)420:2<244::AID-CNE7>3.0.CO;2-M
Axonal connections of a forebrain nucleus involved with vocal learning in zebra finchesThe Journal of Comparative Neurology 279:312–326.https://doi.org/10.1002/cne.902790211
Developmental regulation of basal ganglia circuitry during the sensitive period for vocal learning in songbirdsAnnals of the New York Academy of Sciences 1016:395–415.https://doi.org/10.1196/annals.1298.037
Silent synapses in a thalamo-cortical circuit necessary for song learning in zebra finchesJournal of Neurophysiology 94:3698–3707.https://doi.org/10.1152/jn.00282.2005
Early acquisition of song in the zebra finch, Taeniopygia guttataAnimal Behaviour 39:369–374.https://doi.org/10.1016/S0003-3472(05)80883-8
A novel basal ganglia pathway forms a loop linking a vocal learning circuit with its dopaminergic inputThe Journal of Comparative Neurology 508:824–839.https://doi.org/10.1002/cne.21700
Development of temporal structure in zebra finch songJournal of Neurophysiology 109:1025–1035.https://doi.org/10.1152/jn.00578.2012
A cortical motor nucleus drives the basal ganglia-recipient thalamus in singing birdsNature Neuroscience 15:620–627.https://doi.org/10.1038/nn.3047
The basal ganglia: learning new tricks and loving itCurrent Opinion in Neurobiology 15:638–644.https://doi.org/10.1016/j.conb.2005.10.006
Habits, rituals, and the evaluative brainAnnual Review of Neuroscience 31:359–387.https://doi.org/10.1146/annurev.neuro.29.051605.112851
Song development in the zebra finch and other estrildid finchesIn: R. A Hinde, editors. Bird Vocalizations. Cambridge University Press. pp. 61–74.
Growth and regression of thalamic efferents in the song-control system of male zebra finchesThe Journal of Comparative Neurology 326:442–450.https://doi.org/10.1002/cne.903260309
Topographic organization of a forebrain pathway involved with vocal learning in zebra finchesThe Journal of Comparative Neurology 358:260–278.https://doi.org/10.1002/cne.903580208
The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal gangliaCurrent Opinion in Neurobiology 46:241–247.https://doi.org/10.1016/j.conb.2017.08.015
Critical-period plasticity in the visual cortexAnnual Review of Neuroscience 35:309–330.https://doi.org/10.1146/annurev-neuro-061010-113813
Emergence of selectivity and tolerance in the avian auditory cortexJournal of Neuroscience 32:15158–15168.https://doi.org/10.1523/JNEUROSCI.0845-12.2012
Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuomotor sequences - a computational approachJournal of Cognitive Neuroscience 13:626–647.https://doi.org/10.1162/089892901750363208
What does motor efference copy represent? Evidence from speech productionJournal of Neuroscience 33:16110–16116.https://doi.org/10.1523/JNEUROSCI.2137-13.2013
Cortical inter-hemispheric circuits for multimodal vocal learning in songbirdsJournal of Comparative Neurology 525:3312–3340.https://doi.org/10.1002/cne.24280
Organization of the songbird basal ganglia, including area XThe Journal of Comparative Neurology 508:840–866.https://doi.org/10.1002/cne.21699
The excitatory thalamo-"cortical" projection within the song control system of zebra finches is formed by calbindin-expressing neuronsThe Journal of Comparative Neurology 504:601–618.https://doi.org/10.1002/cne.21457
Goal-directed and habitual control in the basal ganglia: implications for Parkinson's diseaseNature Reviews Neuroscience 11:760–772.https://doi.org/10.1038/nrn2915
Multiple representations of belief states and action values in corticobasal ganglia loopsAnnals of the New York Academy of Sciences 1104:213–228.https://doi.org/10.1196/annals.1390.024
Studying the song development process: rationale and methodsAnnals of the New York Academy of Sciences 1016:348–363.https://doi.org/10.1196/annals.1298.031
A procedure for an automated measurement of song similarityAnimal Behaviour 59:1167–1176.https://doi.org/10.1006/anbe.1999.1416
Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networksEuropean Journal of Neuroscience 28:1437–1448.https://doi.org/10.1111/j.1460-9568.2008.06422.x
Changes in the neural control of a complex motor sequence during learningJournal of Neurophysiology 106:386–397.https://doi.org/10.1152/jn.00018.2011
Ronald L CalabreseReviewing Editor; Emory University, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
[Editors’ note: this article was originally rejected after discussions between the reviewers, but the authors were invited to resubmit after an appeal against the decision.]
Thank you for submitting your work entitled "Neural activity in corticobasal ganglia circuits of juvenile songbirds encodes performance during goal-directed learning" for consideration by eLife. Your article has been reviewed by three peer reviewers, one of whom, Ronald L Calabrese (Reviewer #1), is a member of our Board of Reviewing Editors, and the evaluation has been overseen by a Senior Editor..
Our decision has been reached after consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that your work will not be considered further for publication in eLife.
This is an interesting manuscript, which compares the properties and developmental changes of neuronal activity in cortico-basal ganglia circuits of the zebra finch during the juvenile song learning period. This work focuses on the well-studied cortical LMAN CORE and on the surrounding SHELL. The hypothesis that individual LMAN neurons may function to compare sensory feedback and efference copy from current motor actions to the song target, with SHELL serving as the critic and CORE as the actor, is very appealing.
1) This is mainly a correlative study here without any attempt to dissociate sensory from motor effects on firing.
2) There were substantial concerns about the appropriateness and correctness of the statistical analyses. The correlative nature of the study makes appropriate statistics critical.
3) The analyses of the data are very complicated and somewhat idiosyncratic to songbirds and their unique song-learning behavior. This make for a difficult read for a non-expert.
4) The issue of overwhelming details and smallness of effect concerned us. The idea of the paper is appealing, but we were not overly convinced by their argument. This seems particularly important while considering a manuscript in a journal with a general readership.
This is an interesting manuscript, which compares the properties and developmental changes of neuronal activity in cortico-basal ganglia circuits of the zebra finch during the juvenile song learning period. During this period, juvenile songbirds are actively engaged in evaluating feedback of self-generated behavior in relation to their memorized tutor song (the goal). This work focuses on the well-studied cortical LMAN CORE and on the surrounding SHELL, which the corresponding author's group has delineated in several previous studies and implicated in the process of learning during song development of juveniles. It also tracks development of songs themselves during this period of song learning so that neuronal responses and songs can be compared as learning progresses. The paper uses rather complicated and multifaceted analyses of songs and of responses of CORE and SHELL neurons recorded in awake singing juvenile birds, which nevertheless present a consistent picture. The firing rate of a subset neurons in both SHELL and CORE reflect the degree of tutor song matching during singing, with a higher incidence of SHELL neurons showing correlations between firing rate and degree of tutor song matching compared to those in CORE. In addition, variability in firing rate was lower during production of the best-matched syllable utterances compared to poorly-matched utterances. Syllable utterances least similar to tutor syllables showed a progressive increase in similarity to the tutor over the course of vocal learning and the activity of SHELL, but not CORE neurons reflected this difference in learning trajectory between best and worst tutor-matched syllable renditions. These results suggest that individual LMAN neurons may function to compare sensory feedback and efference copy from current motor actions to the song target, with Shell serving as the critic and CORE as the actor.
This paper was difficult to read because the analyses are complicated and somewhat abstract, but the text is tightly written.
1) The analyses of the data is very complicated and somewhat idiosyncratic to songbirds and their unique song-learning behavior. This make for a difficult read for a non-expert.
2) This reviewer cannot fully critique the data analytical methods but defers to the other experts. One point that make this reviewer particularly uncomfortable is the use of correlations, as in Figure 13 (but others also), which are critical for the central conclusion concerning developmental progression in the responses of SHELL neurons. The data has virtually no vertical spread and thus r will be very small and difficult to obtain significance or show substantive change. Moreover, I am uncomfortable in general that these analyses focus on the outer quartiles of the distributions only (most similar and least similar groups).
This paper explores important issues regarding sensorimotor processing during vocal learning. However serious problems in the quantitative analyses prevent a clear interpretation of the data.
First, many analyses rest on comparing the proportions of statistical tests that achieve significance at p<0.05. Although such approaches can be useful, in many cases the authors do not perform the correct control analyses to support their claims. For example, the authors cite the fact that between 6-10% of correlations are significant as evidence that "neural activity is correlated with similarity to tutor song in both CORE and SHELL subregions of juvenile LMAN". This conclusion is not supported by the data unless the authors show that, for example, finding that 7% of CORE cells have a different firing rate for the most/least similar syllables is significantly different from chance. That is, running 1,000 t-tests on vectors of random numbers will, on average, produce significant correlations (at p<0.05) in 5% of cases. To provide evidence for their conclusions here the authors must demonstrate that finding 7% of correlations significant is surprising.
Additionally, the analysis presented in the section "Activity correlated with multiple, but not single, features" is not convincing. It is not clear to me whether the basic finding – that many neurons have a significant relationship with 11 features simultaneously, but few have sig relationships in single regression – is actually informative. First, finding a significant correlation with the 11 "aggregate features" might simply mean that activity is correlated with one, rather than several, of the features. It would therefore be unsurprising if a far smaller number of single-feature tests were significant. More importantly, and completely separate from the above concern, there is no guarantee that the acoustic features measured are the features that the bird actually evaluates or controls, so the conclusion about representing individual vs constellations of features is unconvincing, i.e. even if the individual vs aggregate analysis were performed correctly, it could be that the LMAN neurons really do care about single features, but that these features are different from, although possibly correlated with, the 11 features used in the analysis.
The logic of the analysis used in Figure 4 is confusing. As I understand it, the authors first select neurons that fire greater than baseline rate in the 50 msec prior to syllable onset, and THEN ask whether there is a change in average activity change locked to syl onset. (Such data are used to argue that only CORE cells shows "coordinated premotor activity"). But isn't it the case that to be included in this analysis, SHELL neurons by definition must have elevated rates (relative to baseline) that stay elevated throughout syllable onset, rather than dropping/modulating as is apparently typical in CORE cells? If so, it does not seem correct to say that SHELL neurons do not display coordinated premotor activity, or do so to a lesser extend than do CORE cells.
A further fundamental problem is that no attempt is made to separate the sensory from motor tunings of these neurons. All other problems with the analysis aside, finding that neural activity co-varies with syllable acoustics immediately raises the question of whether this covariation reflects premotor control (or efference copy) of future behavior, sensory processing of prior behavior, or some combination thereof. This issue confounds interpretation of most of the results in this paper. For example, the finding that variability in neural "responses" is correlated with similarity to tutor (subsection “Variability in neural responses of both CORE and SHELL neurons correlates with similarity to tutor song but not with syllable prototypicality”) might reflect either auditory or motor differences between the syllables that are most- or least-similar to those of the tutor. Given this concern, and the numerous problems with the quantitative analyses outlined above, it is not clear to me that this paper significantly advances our understanding of sensorimotor processing.
The paper by Achiro, Chen and Bottjer shows evidence for neurons in a motor learning pathway that seem to encode the motor performance during learning. This finding is important as it provides evidence for a goal-directed signal that has been an elusive, to date. This has important implications for investigators studying motor learning, particularly those interested in the role of the basal ganglia in goal-directed behaviors. Overall, this manuscript is a tour de force that combines difficult (hard to get) neural recordings with a great deal of careful analysis. As far as I get tell the statistical analyses are good. Although the analysis is very thorough, the paper seems to suffer under the weight of complexities of the analysis so that as I through the paper the analysis became so detailed that I lost the narrative of the paper. This could be particularly problematic for non-birdsong readers. I am not sure how to fix the problem as most of the analysis seems important to 'make the case' and the authors summarize each section.
1) The results are correlative and there is no manipulation of the system to confirm that results. This requires additional experiments that are far beyond the scope of the current results, although they would add additional support to the important finding.
2) In addition to the analysis, the effect seems to be quite small; only a small percentage of neurons and a small-ish change in firing rate (Figure 8). I do not have a problem with the small percentage of neurons as the bird may not need many neurons to encode the performance, particularly if those neurons 'disappear' after learning. This does not mean it is not true, or unbelievable, it just makes it harder to understand and show easily. For example, the firing rate increases are small and the similarity modulation index shift is very hard to see from the graphs, although there is a statistical shift.
3) I think it would be useful to show some raw (at least spikes) data for Figure 2. In the figure it is difficult to tell if there is a change in firing rate from baseline as not much baseline is shown and for some of the songs, the firing rate does not seem to change, for example, in the first LMAN shell singing-excited neuron shown, the neuron seems to fire at the same rate before and during song, except for a burst of activity at song onset. In other words, if I looked at the firing rate I would not be able to tell when the bird song. One way this could be addressed is by providing more baseline activity for one song in each category.
4) In subsection “Neural responses in juvenile LMAN are variable during repeated renditions of emerging syllable types “, the authors say they time warped the spike train to look at timing. This seems to be a little tricky to warp the timing to look at timing. How much did the timing have to be warped (% of syllable duration)? How much warping occurred compared to the variability of the spiking activity?
5) I found the Similarity Modulation Index graphs a little confusing (Figure 8D and 9A). Where the neurons with positive response strengths and negative response strengths analyzed together? If you separate them, do the positive responses result in a significant result and negative responses result in a non-significant results?
[Editors’ note: what now follows is the decision letter after the authors submitted a revised manuscript for consideration.]
Thank you for submitting your article "Neural activity in corticobasal ganglia circuits of juvenile songbirds encodes performance during goal-directed learning" for consideration by eLife. Your article has been reviewed by three peer reviewers, one of whom, Ronald L Calabrese (Reviewer #1), is a member of our Board of Reviewing Editors, and the evaluation has been overseen Andrew King as the Senior Editor. We had two new referees.
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission. Given that this paper has gone through two full rounds of review and remains in an unacceptable form, we are prepared to offer only one more opportunity to provide an acceptable version of the manuscript.
This interesting manuscript compares the properties and developmental changes of neuronal activity in cortico-basal ganglia circuits of the zebra finch during the juvenile song leaning period. During this period, juvenile songbirds are actively engaged in evaluating feedback of self-generated behavior in relation to their memorized tutor song (the goal). This work focuses on the well-studied cortical LMAN CORE and on the surrounding SHELL, which the corresponding author's group has delineated in several previous studied and implicated in the process of learning during song development of juveniles. It also tracks development of songs themselves during this period of song learning so that neuronal responses and songs can be compared as learning progresses. The paper uses sophisticated analyses of songs and of responses of CORE and SHELL neurons recorded in awake singing juvenile birds, which present a consistent picture. The spiking patterns of a subset neurons in both SHELL and CORE reflect the degree of tutor song matching during singing. Both CORE and SHELL neurons encode tutor similarity either by increases or decreases in firing rate, but only SHELL neurons showed a significant association at the population level. During development, tutor similarity (for syllables with low initial similarity to tutor syllables) predicted firing rates most strongly during early stages of learning, and SHELL but not CORE neurons showed decreases in response variability, suggesting that the activity of SHELL neurons reflects the progression of learning. These results suggest that individual LMAN neurons may function to compare sensory feedback and efference copy from current motor actions, with Shell serving as the critic and CORE as the actor.
This paper was difficult to read because the analyses are complicated and somewhat abstract, and many separate issues are addressed which are peripheral to the main interest of the paper; during development, tutor similarity predicted firing rates most strongly during early stages of learning, and SHELL but not CORE neurons showed decreases in response variability. This paper has thus much inherent interest but is too unwieldly as presented and how it should be revised was the subject of lively constructive discussion by the expert reviewers. In revision, the authors should focus and provide detailed responses to the points below.
1) The main revision must be to simplify the message of the paper by focusing ONLY on the song similarity analysis.a) The 'premotor vs non premotor' analyses are not necessary to support the main finding, and given the relatively weak support in favor of a CORE-SHELL distinction on that issue, it should be removed.b) A large portion of the results is devoted to showing that juvenile syllables become more and more similar to tutor during early song development. Although previous studies have focused on the later stages of learning, song similarity is known to increase even in the early stages of learning (e.g. Aronov et al., 2008). The authors should cut substantially the lengthy description and analysis of song development to concentrate on the core issues; during development, tutor similarity predicted firing rates most strongly during early stages of learning, and SHELL but not CORE neurons showed decreases in response variability.c) The concentration on the relation between LMAN firing and the acoustic properties of the syllables is unwarranted. The issue here is that findings are very weak. The authors make a very problematic statistical claim about representation of multiple features but no single feature. This could be, in principle, true, but the analyses show simply a very weak effect, as opposed to synergy or gestalt representation. It is not possible to see anything interesting in the raw data. To quote one of the expert reviewers in discussion "The entire section about analysis of acoustic features, and all figures showing nothing apparent, should be removed, and replaced by something like "we found a very weak, but statistically significant representation of song features (t=… p=… see suppl.)."d) Another issue that undermines the clarity of the paper is the intermixing of results that apply both to LMAN-core and LMAN-shell neurons and results that contrast between these two areas. The authors should highlight similarities and differences between these two areas. As suggested by the first sentence in the discussion, the main finding of the paper applies to both core and shell neurons, and only minor differences are found between the singing-related firing in these two populations. This could be made more explicit throughout the results.
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Thank you for resubmitting your work entitled "Neural activity in corticobasal ganglia circuits of juvenile songbirds encodes performance during goal-directed learning" for further consideration at eLife. Your revised article has been favorably evaluated by Andrew King (Senior editor), a Reviewing editor, and two reviewers.
The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance, as outlined below:
The manuscript is much improved and all comments were fully addressed.
There are two remaining issues that should be resolved before this paper is published.
1) Regressions of baseline-corrected firing rates against tutor similarity for each neuron revealed that cells in both CORE and SHELL exhibited either positive or negative associations between neural activity and degree of tutor similarity: approximately half of all neurons in each subregion showed positive slopes (r values > 0, increased firing rates for syllables with higher tutor similarity).
We should have noticed this issue in the previous round, but this description is deficient. Saying that half increased and half decreased is like looking at random data, right? Instead, plot a histogram of all those slopes and/or r values, so readers can see the distribution on both positive and negative sides. Then do bootstrap (shuffling tutor similarity) to plot this histogram against a random distribution of similarities. This way, it would be possible to evaluate if this is a real correlation.
2) Response strength did not differ between CORE and SHELL neurons for either excitation or suppression (Table 1; Mann-Whitney tests: singing-excited neurons U = 1768, p = 0.06, singing-suppressed neurons U = 459, p = 0.90).
Maybe you should not say that there is no difference for singing excited neurons since 0.06 is almost significant. Instead, say that you see a borderline difference between core and shell, which did not reach significance level, for song-excited neurons, but no apparent effect in the song-suppressed neurons and then give the p values.
If the statistical tests in comment #1 work out satisfactorily, then further review by the expert reviewers will not be required.https://doi.org/10.7554/eLife.26973.024
- Sarah Bottjer
- Sarah Bottjer
- Jennifer M Achiro
- Sarah Bottjer
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
This work was supported by NINDS grants NS 037547 and NS 087506, NIDCD Training Grant DC 009975, and NINDS Training Fellowship NS 073323. The authors declare no competing financial interests. We thank Rachel Yuan for comments on the manuscript, Arthur Shau for technical assistance, and Nicholas Jackson for expert statistical advice.
Animal experimentation: All procedures were performed in accordance with Protocol #9159 approved by the University of Southern California Animal Care and Use Committee and in accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health.
- Ronald L Calabrese, Reviewing Editor, Emory University, United States
© 2017, Achiro et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.