To study interactions between pupil-indexed arousal and task engagement, we trained four adult ferrets on a go/no-go auditory detection task (David et al., 2012; Yin et al., 2010). Animals reported the presence of a target tone following a random number of broadband noise reference stimuli (Figure 1A,B). They were required to withhold licking a water spout during the reference sounds and were given liquid reward for licking during the target window (0.1–1.5 s after target onset). Targets were pure tones, either presented alone (tone versus noise discrimination, ferret L) or embedded in reference stimuli (tone-in-noise detection, ferrets R, B, and T). After training, all animals performed consistently above chance (Figure 1C).

Figure 1 with 1 supplement see all

Download asset Open asset

To track and record changes in pupil size, an infrared video camera was used to image the eye contralateral to the speaker emitting auditory stimuli (Figure 1A). Constant luminance was maintained throughout the experiment so that fluctuations in pupil size reflected only changes in internal state (McGinley et al., 2015; Schwartz et al., 2020). Pupil was recorded continuously as animals passively listened to the task stimuli and actively performed the task. The distribution of pupil size differed between passive and active conditions (Figure 1—figure supplement 1A). Mean pupil size (measured by the major axis diameter) was consistently larger during the active condition (n = 35 sites total, ferret L: n = 13, B: n = 5, R: n = 16, T: n = 1, p = 0.0133, hierarchical bootstrap, Figure 1D). Within active blocks, pupil size also varied with task performance. Average pupil size was similar for false alarm (FA) and hit trials, but it was smaller during miss trials, more closely resembling passive trials (Figure 1—figure supplement 1B). In addition, the short-term dynamics of pupil depended strongly on task condition. During behavior, trial onset evoked a rapid increase in pupil size, which was consistently larger than any evoked change during the passive condition (p = 4.00 x 10⁻⁶, hierarchical bootstrap, Figure 1E,F; Figure 1—figure supplement 1C). Thus, pupil size tracked slow changes in task-engagement (active versus passive blocks) as well as more rapid changes in trial-by-trial performance during the active condition.

We recorded extracellular spiking activity from A1 (n = 129 units total, ferret B: six sites/88 units, R: one site/13 units, T: one site/28 units) and IC (n = 66 units total, ferret B: four sites/7 units, L: 13 sites/46 units, R: 10 sites/13 units) of ferrets, while they switched between active engagement and passive listening. In the IC, we recorded from central (ICC, n = 18 units) and non-central regions (NCIC, n = 48 units), distinguished by their functional topography (Slee and David, 2015). Results from the IC are presented jointly between the two sub-regions, unless otherwise specified.

To study how behavior state modulated auditory neural activity over time, we recorded activity in the same units over multiple passive and active blocks and recorded pupil size throughout each experiment as a measure of the animal’s state of arousal. For analysis, the peristimulus time histogram (PSTH) response to the reference stimuli was averaged across passive and correct (hit) active trials, and data from target periods were excluded. On hit trials, animals did not lick during the reference stimuli; thus this approach reduced the possibility that lick-related motor signals confounded our results (Brosch et al., 2005; Schneider et al., 2014; Stringer et al., 2019).

For many units, activity appeared to be modulated by changes in task engagement, as would be expected from previous behavioral studies (Fritz et al., 2003; Kuchibhotla et al., 2017; Niwa et al., 2012; Otazu et al., 2009). During active behavior, responses to the noise stimuli either increased or decreased, and then returned to their baseline passive state during subsequent passive periods (Figure 2A,B). For other units, however, no consistent change in activity was observed between active and passive conditions. In these cases, firing rate could change between behavioral blocks, but the effect was not consistent across engagement conditions (Figure 2D,E). Thus, despite our controls for sound acoustics and motor activity, the firing rates of some units varied over time in a way that did not appear to be related to task engagement.

Figure 2

Download asset Open asset

We wondered if changes in activity that could not be attributed to task engagement could instead be explained by fluctuations in pupil-indexed arousal (Lin et al., 2019; McGinley et al., 2015; Schwartz et al., 2020). A simple way to investigate if pupil size predicts changes in neural firing rate is to divide all trials (pooled across active and passive) into two groups, based on the mean pupil size, and compute a separate PSTH for each group (Figure 2C,F). In the example A1 units, activity was often enhanced when pupil was larger than median (large versus small pupil PSTH, Figure 2C,F), indicating that pupil size was positively correlated with firing rate.

Because changes in pupil size were correlated with transitions between active and passive blocks (Figure 1E), we could not dissociate the effects of task engagement and pupil size using just the raw PSTH. Therefore, to test our hypothesis that pupil size accounted for changes in neural activity following task engagement, we fit a generalized linear model (Equation 1) in which the response to each presentation of the reference stimuli was modulated by task engagement and pupil size. Each of these variables could additively modulate the baseline (mean) firing rate or multiplicatively modulate response gain. Task engagement was modeled as a discrete regressor (active or passive) and pupil size as a continuous regressor. This approach allowed us to dissociate effects of pupil size and task engagement on firing rate.

Model performance was measured by the fraction variance explained (squared correlation coefficient, r²) between the predicted and actual single-trial spike rates, using nested cross-validation. To test for significant contributions of each regressor, we introduced these variables stepwise (Fritz et al., 2010; Musall et al., 2019). The full model included both task engagement and pupil size variables. We compared performance of the full model to the performance of a state-independent model, for which both task and pupil variables were shuffled in time (null model), and of two partial models, which accounted for effects of a single state variable while shuffling the values of the other variable in time (task- or pupil partial models).

The regression analysis revealed substantial variability in effects of task engagement and pupil size on auditory responses. For one example A1 unit, the modulation of firing rate between active and passive blocks was almost completely accounted for by the task variable, with little contribution of pupil size (Figure 2A–C). When we incorporated the task variable, predictions significantly improved with respect to the state-independent null model (null model, r² = 0.12; task partial model, r² = 0.30; jackknifed t-test, p < 0.05; Figure 2C, top). Pupil size was also able to account for some changes in firing rate (pupil partial model, r² = 0.21; Figure 2C, middle), but the full model incorporating both task engagement and pupil size showed no additional improvement over the task partial model (r² = 0.30, p > 0.05; Figure 2C, bottom). Thus, all behavior-dependent changes for this unit were accounted for by task engagement. The apparent pupil-related effects were in fact explained by correlations between pupil size and task engagement.

Conversely, for other units, changes in firing rate were uniquely accounted for by pupil size. For a second example A1 unit (Figure 2D–F), the task partial model (r² = 0.39) indicated a small improvement over the null model (r² = 0.40, p > 0.05, jackknifed t-test). However, the pupil partial model showed a significant improvement over the null model, and it was able to account for changes in the PSTH between passive and active conditions (r² = 0.51, p < 0.05, Figure 2F, middle). The full model showed no additional improvement over the pupil partial model (r² = 0.53, p > 0.05; Figure 2F, bottom). Thus, we found evidence for A1 units whose activity could be modulated either by task engagement or by pupil size.

Next, we investigated the prevalence of task- and pupil-related modulation across the population of A1 and IC units. To quantify the relative contribution of each variable to model performance, we compared the variance in single-trial spike rate explained by each of the four models (r²). Across the entire data set, 66/129 (51%) A1 units and 37/66 (56%) IC units showed a significant increase in r² for the full model over the null model, indicating an effect of either task engagement or pupil size (p < 0.05, jackknife t-test; Figure 3A). By computing the difference between the cross-validated r² for the full model and for the two partial models separately, we determined the variance that could be uniquely explained by either pupil size or task engagement in each unit. This produced four categories of state-modulated neurons (Figure 3A): units for which both task and pupil uniquely contributed to the modulation (black); units for which only one variable, task (purple, example unit in Figure 2A) or pupil (green, example unit in Figure 2D), contributed uniquely; and units for which changes in activity could not be uniquely attributed to either task or pupil (dark gray).

Figure 3 with 1 supplement see all

Download asset Open asset

A comparison of variance explained across the entire data set shows consistent improvement by the full model over the null model in both A1 (mean r² null: 0.282, full: 0.340, p < 10⁻⁵, hierarchical bootstrap test, Saravanan et al., 2020) and IC (mean r² null: 0.290, full: 0.399, p < 10⁻⁵, Figure 3B). Although the proportion of units with any significant state-related modulation was comparable between A1 and IC, only about a quarter of those A1 units showed unique pupil-dependent modulation (15/66), compared to about a half for IC (20/37). The average variance uniquely explained by each state variable also differed between areas. In A1, the variance explained by task (mean r² task-unique: 0.017) was significantly greater than variance explained by pupil (mean r² pupil-unique: 0.007, p = 0.0335, hierarchical bootstrap, Figure 3C). In IC, the variance explained was not significantly different between task and pupil (mean r² task-unique: 0.016, mean r² pupil-unique: 0.019, p = 0.769). These effects were not significantly different between single- and multiunit data in either area (Figure 3—figure supplement 1). Together, these results suggest that in A1, both the magnitude and prevalence of modulation by task engagement are more pronounced than by pupil size. In IC, modulation is equal between the two state variables.

The analysis above treated task engagement as a discrete, binary variable, switching between active and passive states. However, behavioral performance, measured by d’, varied substantially between animals and behavior blocks (Figure 1C). We reasoned that performance represents a more graded measure of task engagement and thus may explain variability in task-related changes across experiments. Indeed, we found a significant correlation between d’ and the unique variance explained by task in A1 (r² task-unique; r = 0.303, p = 0. 007, hierarchical bootstrap; Figure 4A, left). The same relationship was not observed in IC (r = 0.069, p = 0.318; Figure 4A, right), nor was there a relationship between d’ and unique variance explained by pupil in either area (r² pupil-unique; A1: r = 0.113, p = 0.158; IC: r = −0.017, p = 0.552, Figure 4B). Thus, in A1, effects of task engagement cannot be described by a binary variable, even after dissociating effects of pupil. Instead, effects of task engagement are comprised of at least one process that varies continuously between more and less accurate behavior.

Figure 4 with 1 supplement see all

Download asset Open asset

We also observed that both d’ and task engagement effects in A1 differed between animals (Figure 4—figure supplement 1). We wondered if the differences in neural modulation could be fully explained by behavioral performance or if they reflected additional between-animal differences. To answer this question, we performed a multiple regression with both d’ and animal identity as independent variables. This analysis revealed that in A1 d’ could fully explain the differences in modulation for task engagement (d’: F = 16.0, p = 0.000093; animal: F = 0.66, p = 0.52). Neither d’ or animal significantly predicted task engagement effects in IC (d’: F = 1.11, p = 0.29; animal: F = 0.22, p = 0.80). A similar analysis of pupil-related effects revealed that r² pupil-unique did not depend on d’, although it did depend on the amount of pupil variability within an experiment (Figure 4—figure supplement 1). Thus, differences in task engagement effects between animals could be explained by differences in the accuracy with which they performed the task.

So far, we have shown that neurons in A1 and IC show state-dependent modulation of activity that can be explained variably by task engagement and/or pupil size. The observation that pupil size is correlated with task engagement (Figure 1) suggests previous characterizations of task-related plasticity that did not measure pupil might have attributed changes in neural activity to task engagement when they could, in fact, be better explained by pupil size (Downer et al., 2015; Fritz et al., 2003; Knudsen and Gentner, 2013; Kuchibhotla et al., 2017; Lee and Middlebrooks, 2011; Otazu et al., 2009; Ryan and Miller, 1977; Yin et al., 2014). Therefore, we asked to what extent pupil-related modulation could explain changes in activity between active and passive blocks that would otherwise be attributed to task engagement alone.

To quantify the magnitude and sign of the modulation by task engagement, we used a modulation index (MI_AP), which measured the fraction change in mean predicted firing rate between active and passive conditions (Equation 5, Otazu et al., 2009; Schwartz and David, 2018). In previous work, MI was measured directly from differences in the raw firing rate. However, pupil size was correlated with task engagement, and both could contribute to changes in firing rate. To tease apart their respective contributions, we used logic similar to the calculation of unique variance explained (above, Figure 3C). To measure task-related modulation while ignoring possible effects of pupil size, MI was computed from predictions of the task only model (MI_AP task-only). To account for effects of both task engagement and pupil size, MI was computed from predictions by the full model (MI_AP full). Modulation that could be explained by pupil size was computed using the pupil-only model (MI_AP pupil-only). Then the unique task-related change, MI_AP task-unique, was computed as the difference between MI_AP full and MI_AP pupil-only (Equation 6). We used a converse approach to quantify modulation uniquely attributable to pupil between large and small pupil states (MI_LS, see below).

The model predictions for the example A1 units above (Figure 2) illustrate the method for dissociating contributions of pupil and task engagement to MI_AP. The first example shows a strong task engagement effect that cannot be explained by pupil size (MI_AP task-unique = 0.25, Figure 2C), and the second shows weak task-only modulation that can be completely explained by pupil (MI_APtask-unique = −0.07, Figure 2F).

A comparison of MI_AP task-only and MI_AP task-unique across the entire data set showed that a task-only model often overestimated the magnitude of changes due to task engagement (Figure 5A). To measure this effect, we normalized the sign of MI for each unit so that the means of MI_AP task-only and MI_AP task-unique were always positive. This normalization accounted for the bidirectionality of modulation while avoiding bias that would result from normalizing sign for just a single condition. After sign normalization, accounting for pupil size led to a decrease in MI_AP magnitude across units in A1 and IC (A1: p < 10⁻⁵; IC: p = 0.0140, hierarchical bootstrap). Effectively, accounting for pupil size led to a 33% reduction in the magnitude of MI_AP in both areas (A1: mean magnitude MI_AP task-only = 0.141; MI_AP task-unique = 0.095; IC: MI_AP task-only = 0.069; MI_AP task-unique = 0.046). The effect of removing pupil-related activity on the magnitude of active versus passive modulation was also not different between central and non-central regions of IC (ICC versus NCIC, p = 0.467, hierarchical bootstrap, Figure 5—figure supplement 1). Taken together these results show that pupil-indexed arousal accounted for a significant portion of the change in activity between passive and active blocks in A1 and both regions of IC.

Figure 5 with 1 supplement see all

Download asset Open asset

In some units, the effects of task engagement and pupil size on firing rate were difficult to dissociate by simply looking at raw activity (e.g., Figure 2D). However, the regression analysis could disambiguate their unique contributions and showed that often only one state variable, task or pupil, modulated the activity of individual units. Based on this observation, we wondered if effects of task engagement and pupil size operated via functionally distinct or common pathways. While identifying specific anatomical circuits was outside the scope of this work, we reasoned that if the two modulations were mediated by the same inputs to A1 and IC, a unit modulated by task engagement would also tend to be modulated by pupil size. To test this prediction, we measured the correlation between modulation that could be uniquely attributed either to task engagement or pupil. To measure modulation by pupil, we computed changes in spike rate associated with pupil size by measuring MI_LS (Equation 5), analogous to MI_AP, but dividing trials according to whether pupil size was larger or smaller than the median for the experiment, respectively. For calculation of task-unique effects, MI_LS pupil-unique was the difference between MI_LS full and MI_LS task-only (Equation 7). Thus, we could compare the change in firing rate uniquely attributable to a change in task engagement to a change in pupil size for each unit (Figure 6).

Figure 6 with 2 supplements see all

Download asset Open asset

We found a negative correlation between these quantities in A1 (r = −0.281, p = 0.040, hierarchical bootstrap, Figure 6, left) and no correlation in IC (r = 0.104, p = 0.191, Figure 6, right). The absence of a positive correlation between task- and pupil-related effects suggests that independent populations of neurons in A1 and IC are modulated by these variables. To test directly for dependence between task engagement and pupil size, we performed a permutation analysis, estimating the number of units that would show both task- and pupil-unique effects if the likelihood of these effects was independent (p < 0.05, jackknife t-test, counts from Figure 3). The frequency of joint occurrences was no greater than expected for independent distributions in both areas (A1: p = 0.41, IC: p = 0.59, permutation test). This independence is consistent with the hypothesis that separate neural mechanisms mediate effects of task engagement and pupil-indexed arousal.

Based on previous studies that used a similar positive reinforcement structure, we expected the average sign of task-related modulation to be positive in A1 (David et al., 2012) and negative in IC (Slee and David, 2015). In A1, MI_AP task-only, computed without accounting for effects of pupil, was in fact positive on average (median 0.069, p = 0.0240, hierarchical bootstrap). However, MI_AP task-unique, which accounted for pupil size, was not significantly different from zero (40/69 positive, median MI_AP task-unique = 0.027; p = 0.232). In IC, average MI_AP task-only was not different from zero (MI_AP task-only median: −0.010, p = 0.310), but MI_AP task-unique showed a trend toward being negative (17/49 positive, median: −0.037; p = 0.062). In contrast to the variable task-related effects, the average effect of increased pupil size was positive in A1 (55/69 positive, median MI_LS pupil-unique = 0.0139, p = 0.032, hierarchical bootstrap). In IC, there was also a trend toward increased firing rate with larger pupil (31/49 positive, median MI_LS pupil-unique = 0.022; p = 0.090). Thus, in both areas, accounting for higher firing rate during larger pupil led to decreased in MI associated with task engagement (Figure 5).

Some previous work has also reported that the effect of task engagement can depend on the relationship between task-relevant sound features and the receptive field of individual neurons (David et al., 2012; Slee and David, 2015). During a tone detection task, response gain in IC is selectively suppressed for neurons with best frequency (BF) near the target tone frequency (Slee and David, 2015). In A1, frequency tuning is suppressed at the target frequency, despite there being no systematic difference in overall response gain between units with BF near or far from the target (David et al., 2012). After accounting for pupil size, we observed similar trends in the data but no significant effects, likely due to the small sample size in the on-BF and off-BF groups (Figure 6—figure supplement 1). Task engagement effects could also depend on the difficulty or other structural elements of the task (Carcea et al., 2017; Knyazeva et al., 2020). We considered whether state-dependent effects varied with task difficulty but found no differences between pure tone, high SNR tone-in-noise (easy), and low SNR tone-in-noise (hard) conditions (Figure 6—figure supplement 2).

While all units were recorded from areas functionally and anatomically characterized as auditory, neurons within a single auditory field can vary substantially in their tendency to respond to sound (Atiani et al., 2014; Gruters and Groh, 2012). We noticed that some units, particularly in IC, did not show strong auditory responses but were modulated during behavior. We asked whether the magnitude of the state-dependent modulation in each recorded unit was related to how reliably it responded to repeated presentations of the same auditory stimulus. Auditory responsiveness was quantified by the variance explained by the null model, since it described how accurately the PSTH response to the reference stimuli, averaged across the entire recording, predicted responses on single trials. Larger or smaller values of null model r² were associated with units whose auditory responses were more or less reliable across the experiment, respectively. We compared this measure of auditory responsiveness to the change in r² between the full and null models, which indicates additional variance explained by the state variables. In IC, but not in A1, units that responded less reliably to sound also showed greater dependence on state (A1: r = −0.134, p = 0.130; IC: r = −0.323, p = 0.0129, hierarchical bootstrap; Figure 7).

Figure 7 with 1 supplement see all

Download asset Open asset

These same relationships were observed when auditory responsiveness was defined as the signal-to-noise ratio (SNR) of the spectro-temporal receptive field (STRF) measured during passive listening (Figure 7—figure supplement 1B; Klein et al., 2000). The results also held true for a larger pool of neurons, which included data collected without pupillometry and analyzed with the task-only model (Figure 7—figure supplement 1C). High prediction accuracy by the null model limits the variance that can be explained by behavioral state (note relatively weak state effect in highest quintile in Figure 7). To explore how this bound might impact the analysis of auditory responsiveness, we performed a similar analysis but measured the correlation between null model performance and the fraction of remaining variance explained by pupil size and task engagement. In this case, the null model performance was positively correlated with the remaining variance explained in A1 (r = 0.301, p = 0.0360, hierarchical bootstrap) and not correlated in IC (r = 0.124, p = 0.178, Figure 7—figure supplement 1A). In this case the overall slope of the relationships changed, but the effects were still relatively larger for IC units with low auditory responsiveness. Thus, across multiple measures of sensory responsiveness, we observed that effects of behavior state in IC were more common in units with weak sensory responses, while in A1, state-dependent modulation was independent of sensory responsiveness.

Previous studies in both A1 (Fritz et al., 2003) and IC (Slee and David, 2015) have reported examples of task-related modulation persisting in passive blocks following active engagement (referred to as post-passive blocks). These effects have been interpreted as persistent task-related plasticity, but they are highly variable and difficult to attribute to any specific aspect of the behavior. We observed these effects in our data as well. In addition to units in A1 (e.g., Figure 2D), some units in IC showed persistent changes in activity during passive blocks following behavior. These changes sometimes lasted through the entire post-passive block (Figure 8A) and other times gradually reverted to pre-passive levels (Figure 8B). The time course of these changes often tracked fluctuations in pupil size.

Figure 8

Download asset Open asset

To measure the presence of persistent task-related modulation—and the extent to which pupil size accounts for it—we compared the activity between passive blocks before (P1) and after (P2) a behavior block. We performed a regression analysis similar to that used for measuring task engagement effects, but treating passive block identity (P1 or P2) as a state variable in addition to pupil size. MI was computed for predictions by a model based only on block identity, ignoring pupil (MI_P1P2 block-only), and for the unique contribution of passive block after accounting for pupil effects (MI_P1P2 block-unique, Figure 8A). We assessed the magnitude of MI using the same sign normalization as for the task-unique MI analysis above (Figure 5). In both A1 and IC, the magnitude of firing rate modulation between passive blocks was significantly reduced after accounting for effects of pupil (A1: p < 10⁻⁵, IC: p < 10⁻⁵, hierarchical bootstrap). Accounting for pupil size led to a 20% and a 48% reduction in P1 versus P2 modulation in A1 and IC, respectively (A1: mean signed-normalized MI_P1P2 block-only = 0.140, MI_P1P2 block-unique = 0.100; IC: mean signed-normalized MI_P1P2 block-only = 0.100, MI_P1P2 block-unique = 0.052). This change was not significantly different between animals in A1 (F = 0.669, p = 0.516, one-way ANOVA) or in IC (F = 0.446, p = 0.643). While there was decrease in post-behavior effects on average, there were distinct groups of units in both A1 and IC that showed changes between P1 and P2, even after accounting for pupil size (compare units falling on the unity line versus x-axis in Figure 8C). Thus, while some post-behavior effects previously reported as persistent plasticity induced by behavior can in fact be explained by fluctuations in pupil size, a small number of units showed persistent post-behavior changes, even after accounting for pupil.

The correlation between pupil size and task engagement demonstrates the importance of accounting for interactions between state variables when measuring their effects on neural activity. This observation also leads to a more fundamental question: what constitutes a meaningful measure of behavioral state? Concepts like task engagement and arousal make intuitive sense, but they become slippery when one attempts to pin them to distinct physiological processes. The current results illustrate that a commonly used binary task engagement variable is better described by a combination of pupil size and accuracy of behavioral performance. Other characterizations of task engagement have argued similarly that the degree of engagement can vary, even when relevant acoustic stimuli are held fixed (Atiani et al., 2009; Carcea et al., 2017; Knyazeva et al., 2020; McGinley et al., 2015; Zhou et al., 2014). Larger data sets that track continuous fluctuations in performance are likely to be able to measure fluctuations of engagement effects within behavior blocks. It remains unclear how smoothly internal state can vary. Theories of network attractors suggest that there may in fact be discrete changes in brain state associated with different behavioral contexts (La Camera et al., 2019). Thus, while task engagement is clearly not binary, it could still comprise multiple metastable states.

As a behavioral state variable, arousal also has multiple definitions, including the transition from wakefulness to sleep, the response to emotionally salient stimuli, and a generalized activation of the autonomic nervous system (Satpute et al., 2019). This relatively nonspecific behavioral concept contrasts with pupil size, which has a clear link to neurophysiology (McGinley et al., 2015; Reimer et al., 2016; Zekveld et al., 2018). The detailed circuits that link pupil size to cognitive state are not fully understood, but a connection to behavioral state has been demonstrated repeatedly. Thus, while pupil size may not map trivially onto the concept of arousal, it has a clear link to brain mechanisms.

A clearer understanding of the relationship between behavioral state and sensory processing may be achieved with models that incorporate additional well-defined variables like pupil size, which can be linked to well-defined neurophysiological processes. For example, motor- and reward-related activity are associated with specific neural circuits (Bakin and Weinberger, 1990; Knyazeva et al., 2020; Schneider et al., 2014). The current study excluded epochs that contained licking activity to avoid motor confounds, but motor signals can be included as additional model inputs that reflect aspects of signal detection and decision making (Runyan et al., 2017). These variables may not be directly related to an experimentally imposed task structure, but they capture processes involved in the behavior and can be quantified more concretely than abstract variables based on experimentally imposed task conditions.

Our findings indicate that changes in firing rate previously attributed to task engagement might instead reflect fluctuations in pupil-indexed arousal, particularly in IC, where pupil-related changes predominate. Changes in pupil size can also explain some modulation that persists after behavior ceases (Fritz et al., 2003; Slee and David, 2015). However, pupil size does not account for all the modulation between active and passive blocks; instead, it appears that both pupil-indexed arousal and task engagement act on independent neural populations to shape sound processing.

Previous studies that measured the effects of task engagement on auditory neuronal activity described different effects on overall excitability between A1 and IC. In A1 the effects of task engagement were shown to be enhancing (David et al., 2012), while in IC these changes were predominantly suppressive (Slee and David, 2015). Separate studies have shown that pupil size is positively correlated with excitability in both A1 and IC (Joshi et al., 2016; Schwartz et al., 2020). Here we also found that increased pupil size enhanced excitability in both areas. However, after accounting for pupil-related changes, task-engagement effects were no longer enhancing in A1, and they were more suppressive in IC. Pupil-related changes did not depend on tuning for the target tone. These nonspecific effects of pupil size contrasted with the effects of task engagement, which can be selective for task-relevant features (Fritz et al., 2003; Kuchibhotla et al., 2017; Slee and David, 2015).

In the current study, significant state-dependent A1 units were equally likely to be modulated by task engagement, by pupil size, or by both. In IC, however, we found that a larger portion of state-dependent changes could be explained by pupil size. Furthermore, in IC but not in A1, units with less reliable auditory responsiveness were those with a stronger state-modulation component. This result is perhaps not surprising given that the majority of IC units in our sample were recorded from non-central regions of the IC (NCIC). These are dorsal and lateral regions of IC that receive feedback from cortex (Winer, 2006), input from neuromodulatory nuclei such as the pedunculopontine and latero-dorsal tegmental nuclei (Motts and Schofield, 2009), as well as multisensory information from somatosensory, visual, and oculomotor centers (Gruters and Groh, 2012). These results suggest that the auditory midbrain is not a passive sound processor. Instead, the network of midbrain nuclei in the IC plays an active role enhancing behaviorally relevant signals. Experiments that sample more units across the different subregions would be needed to confirm the interplay of behavioral and sensory signals across IC.

Multiple regression has been used previously to dissociate effects of pupil size and movement on neural activity (Musall et al., 2019; Stringer et al., 2019). This study used this approach to dissociate the effects of state-related variables and differentiate where effects emerge. Several other variables have been shown to impact auditory responses during behavior, including selective attention, temporal expectation, reward value, and motor responses (David et al., 2012; Huang et al., 2019; Jaramillo and Zador, 2011; Lakatos et al., 2013; Metzger et al., 2006). The same method can be adopted to explore the contribution of other state variables measured in normal conditions as well as in perturbation experiments to identify the circuits through which such variables operate.

The correlation between task engagement and pupil size may help explain a number of results related to neuromodulation in the auditory system. It is well established that neuromodulators can induce short-term changes in activity and sensory tuning in cortex (Bakin and Weinberger, 1996; Goard and Dan, 2009) and midbrain (Gittelman et al., 2013; Habbicht and Vater, 1996; Hurley and Pollak, 2005). Cholinergic fibers projecting to A1 from the nucleus basalis play a key role in rapid switching between passive listening and active engagement by modulating the activity of different populations of cortical inhibitory interneurons (Kuchibhotla et al., 2017). In addition, the activity of cholinergic and noradrenergic terminals in A1 was found to be elevated during pupil dilation and reduced during pupil constriction (Reimer et al., 2016). Thus, these neuromodulatory systems have been implicated in effects of both task engagement and pupil-indexed arousal. The correlation between these state variables could be explained by shared cholinergic and noradrenergic signaling.

Animal care and procedures were similar to those described previously for neurophysiological recordings from awake behaving ferrets (Slee and David, 2015). Four young adult male ferrets were obtained from an animal supplier (Marshall Farms, North Rose, NY). After a 2-week quarantine, a sterile surgery was performed under isoflurane anesthesia to mount two head-posts for head fixation and to permit access to auditory brain regions for recordings. A UV light-cured composite (Charisma, Heraeus Kulzer International) was used to mount two custom-made stainless-steel head-posts spaced approximately 1 cm apart along the sagittal crest of the skull. The stability of the implant was achieved using 8–10 stainless steel bone screws mounted in the skull (Depuy Synthes, Raynham, MA), which were covered along with the headpost in additional Charisma and acrylic denture material (Co-oral-ite Dental, Diamond Springs. CA). Two 1.2 × 1.2 cm wells over A1 in the implant were covered with only a thin layer of Charisma to allow access to auditory regions.

Following surgery, animals were treated with prophylactic antibiotics (Baytril 10 mg/kg) and analgesics (Buprenorphin 0.02 mg/kg) under the supervision of University veterinary staff. For the first 2 weeks the wound was cleaned with antiseptics (Betadine 1:4 in saline and Chlorhexidine 0.2%) and bandaged daily with application of topic antibiotic ointment (Bacitracin). After the wound margin was healed, cleaning and bandaging occurred every 2–3 days throughout the life of the animals. This method revealed to be very effective in minimizing infections of the wound margin. After recovery (~2 weeks), animals were habituated to a head-fixed posture inside the sound-booth chamber for about 2 weeks prior to the beginning of the training.

The animals were trained by instrumental conditioning to perform a positively reinforced, tone versus noise discrimination task (ferret L) (Slee and David, 2015) or tone-in-noise detection task (ferrets R, B, and T). During training, animals were provided access to water ad libitum on weekends, but were placed on water restriction during the weekdays (Monday through Friday), allowing them to maintain >90% of their baseline body weight long term. On weekdays, they were given the opportunity to receive liquid reward during behavioral training. Each behavioral trial consisted of a sequence of two to five broadband noise reference sounds (TORCs; 30 samples, five octaves, 0.75 s duration, 0.7 s inter-stimulus interval) (Klein et al., 2000), followed by a target tone, either alone (tone versus noise task) or overlapping with another TORC (tone-in-noise task). Animals reported the occurrence of the target by licking a water spout (Figure 1A). Licks were detected by a piezoelectric sensor glued to the spout. Licks occurring within the target window, 0.1–1.5 s after target onset, were rewarded with one to three drops of a 2:1 solution of water and a high-protein, high-calorie dietary supplement (Ensure). FAs, licks occurring before the target window, and misses, a failure to lick during the target window, resulted in a 5–8 s timeout. Regardless of trial outcome, the next trial began only after animals did not lick for a random period (2.5 ± 0.5 s), reducing the prevalence of accidental, short-latency FAs. The number of TORCs per trial was distributed randomly with a flat hazard function to prevent behavioral timing strategies (Heffner and Heffner, 1995). Behavioral performance was measured using d’, the z-scored difference between hit rate (HR) and FA rate (Green and Swets, 1966).

A behavioral block consisted of 60–100 trials with the same target tone frequency (100–20,000 Hz) and same distribution of target SNR. Animals completed one to three behavioral blocks per day, between which target frequency and/or SNR could change. During training, target frequency was chosen at random to span the audiogram of a ferret (Kelly et al., 1986). During electrophysiological recordings, target tone frequency was selected to match the BF of the recording site.

During the training on the tone-in-noise variant of the task, the TORC sample masking the target varied randomly in each trial to prevent animals from using TORCs’ spectro-temporal features to identify targets. At the beginning of training, the tone was presented at +40 dB SNR (ratio of peak-to-peak amplitude) relative to the TORCs. This difference was gradually reduced over the course of 2–3 weeks until the animal consistently performed above chance (three behavioral blocks with performance yielding d’>1, see below) at 0 dB SNR.

For the tone-in-noise task, the SNR of the tone with respect to the overall level of the TORCs (fixed at 55 or 60 dB SPL depending on the animal) varied between +5 and −20 dB SNR, in 5 dB steps. Each session included five target/noise SNRs. To manipulate task difficulty within each session, the probability of each of the five target/noise SNRs varied, yielding two difficulty conditions: a high SNR (‘easy’) condition in which 60% of the trials the target/noise SNR was either the highest or the second to the highest SNR within the target/noise SNR distribution; and a low SNR (‘hard’) condition in which the two lowest target/noise SNRs occurred in 60% of the trials. For example, for ferret B, the distribution after training was completed and kept between 0 and −20 dB SNR, so that in the easy condition 0 and −5 dB SNR targets would appear 60% of the time, −10 dB SNR 20% of the time, and the remaining −15 and −20 dB SNR would be presented another 20% of the trials. During electrophysiological tone-in-noise experiments, the tone was embedded in a single TORC sample, which also occurred in the reference period. We confirmed that animals were not biased to respond to this TORC exemplar in the reference phase.

Behavioral training and subsequent neurophysiological recording took place in a sound-attenuating chamber (Gretch-Ken) with a custom double-wall insert. Stimulus presentation and behavior were controlled by custom MATLAB software (code available at https://bitbucket.org/lbhb/baphy). Digital acoustic signals were transformed to analog (National Instruments), amplified (Crown), and delivered through two free-field speakers (Manger, 50–35 000 Hz flat gain) positioned ±30° azimuth and 80 cm distant from the animal. Stimuli were presented either from the left or the right speaker, contralaterally to the recording site. Sound level was equalized and calibrated against a standard reference (Brüel and Kjær).

During experiments, infrared video of one eye was collected for offline measurement of pupil size as an index of arousal (pupil-indexed arousal). Ambient light was maintained at a constant level to prevent light-evoked changes in pupil and to maximize the dynamic range of pupil-related changes (Schwartz et al., 2020). Recordings were collected using a CCD camera (Adafruit TTL Serial Camera 397) fitted with a lens (M12 Lenses PT-2514BMP 25.0 mm) whose focal length allowed placement of the camera 10 cm from the eye. To improve contrast, the imaged eye was illuminated by a bank of infrared LEDs. Ambient luminance was provided using a ring light (AmScope LED-144S). At the start of each recording day, the intensity of the ring light was set to a level (~1500 lux measured at the recorded eye) chosen to give a maximum dynamic range of pupil sizes. Light intensity remained fixed across the recording session. Pupil size was measured from the video signal offline using custom MATLAB software, which is detailed in Schwartz et al., 2020.

Data were recorded during behavior and pupillometry from 129 units in A1 and 66 units in IC. For one supplementary analysis (Figure 7—figure supplement 1), additional data were collected during behavior but without pupillometry (IC: 135 additional units, A1: 125 additional units). After animals demonstrated consistent above-chance performance (target versus reference d’ > 1, see below), a small craniotomy was opened to access either A1 or IC. Extracellular neuronal activity was recorded in non-anesthetized ferrets either using a tetrode (Thomas Recording Inc) or a linear 64-channel silicon probe (Shobe et al., 2015). The impedance of the tetrode was measured to be 1–2 MOhm, and the 64-channel probe was electroplated to reach a 0.7-MOhm impedance in each channel. The tetrode and probe were inserted using a motorized electrode positioning system (Alpha-Omega).

Amplified (tetrodes, AM Systems; arrays, Intan) and digitized (National Instruments) electrophysiological signals were stored using the open-source data acquisition software MANTA (Englitz et al., 2013) or Open Ephys (Black et al., 2017). Recording sites in A1 and IC were initially targeted stereotactically (Moore et al., 1983; Shamma et al., 1993). Sites in A1 were confirmed based on tonotopy—which changes from high to low BF as one moves from dorso-medial to ventro-lateral regions—and relatively reliable and simple frequency tuning (Shamma et al., 1993). Recording sites in the IC were classified based on tonotopy and response properties (Aitkin et al., 1975; Moore et al., 1983). Neurons in the central nucleus of the IC (here referred to as ICC) receive input from the auditory brainstem, and have a characteristic short response latency, dorsal-ventral tonotopy, and narrow bandwidth tuning. Conversely, regions around the central nucleus do not receive direct ascending input and present longer response latencies, considerably less sharp tuning, lack consistent tonotopic organization; these areas were grouped together as NCIC (Slee and David, 2015).

For tetrode recordings, upon unit isolation, a series of brief (100 ms duration, 200–400 ms inter-stimulus interval, 50 dB SPL) tones and/or narrowband noise bursts were used to determine the range of frequencies that evoked the strongest response, the BF of the unit(s). If a unit(s) in the site did not respond to the sound stimuli (an evoked increase or decrease in activity compared to spontaneous activity either during or after sound presentation), the electrode was moved to a new recording depth with small, 5 µm incremental steps. For the 64-channel recordings, we lowered the probe so that its entire extent (1 mm) spanned the target region (A1 or IC).

Putative spikes were sorted offline by band-pass filtering the raw trace (300–6000 Hz) and extracting events from the continuous signal that deviated ≥4 standard deviations from zero. To separate single units and stable multiunits from the electrode signal, we used the Catamaran clustering program (kindly provided by D. Schwarz and L. Carney) (Schwarz et al., 2012) for tetrode recordings and the software KiloSort (Pachitariu et al., 2016) for array recordings. In both cases, units were defined based on visual inspection of traces and by having less than 1% of inter-spike intervals shorter than 0.75 ms (single units) and less than 2.5% inter-spike intervals shorter than 0.75 ms (multiunits). Stability of single-unit isolation was verified by examining waveforms and interval histograms. If isolation was lost during an experiment, only activity during the stable period was analyzed.

Data preprocessing and analysis were performed using custom MATLAB and Python scripts. All analyses were performed on biological replicates, either different behavior sessions, different neurons (single- or multiunit, see above), or different neuron-behavior condition pairs. Neural and pupil activity were binned at 20 samples/s before analysis. A Python library used for the modeling portion of this analysis is available at https://github.com/LBHB/NEMS/. Sample size (number of animals, number of units per condition) was determined based on norms established in previous studies of neural activity in awake, behaving animals (Fritz et al., 2003; Niwa et al., 2012; Otazu et al., 2009).

Behavioral performance was measured using signal detection theory (Green and Swets, 1966; Stanislaw and Todorov, 1999). Hits or FAs occurred when the animal licked the piezo waterspout upon presentation of the target or the reference stimuli, respectively. Misses or correct rejections (CRs) occurred when the animal did not lick following presentation of the target or reference. HR was calculated as the proportion of licks that occurred upon presentation of the target during the response window (0.1–1.5 s after target onset), HR = hits/(hits+misses). FA rate (FAR) was calculated as the proportion of reference stimuli that elicited a lick, during the same window following reference onset, FAR = FAs/(FAs + CRs). Sensitivity (d’), a measure of the animals’ ability to discriminate between target tone and reference noise, was measured by the difference between the z-scored HR and the FA rate, d’ = HR_z− FAR_z. No discrimination, that is, equal likelihood of responding to target and reference is indicated by d’ = 0. Animals were considered trained to the task and ready to undergo electrophysiological recordings when they performed consistently above chance (d’ > 1) for three consecutive sessions.

Tuning properties of A1 and IC units were characterized using the STRF estimated by reverse correlation between time-varying neuronal spike rate and the rippled noise used as reference sounds during behavior (Klein et al., 2000). The STRF provides a quick and efficient measure of stimulus frequencies that enhance or suppress neural firing. In the current study, the STRF was used to measure BF as the center of mass of a spectral tuning curve computed from the first principle component of the STRF matrix (Simon et al., 2007). Auditory responsiveness was computed as the SNR of single trial responses to the reference noise stimuli during passive listening (Klein et al., 2000).

We fit a generalized linear model in which time-varying state variables pupil size, $p\left(t\right)$, and task engagement, $b\left(t\right)$, were used to re-weight each unit’s mean evoked response to each noise stimulus, $r_0\left(t\right)$, and spontaneous rate, $s_0$, to generate a prediction of the single-trial spike rate, ${\displaystyle r_{full}\left(t\right)}$, at each point in time. The model included multiplicative gain parameters ($g$) and DC offset parameters ($d$) to capture both types of modulation. We refer to this model as the full model:

To account for nonlinear threshold and saturation of state effects, the summed state signal was passed through a sigmoid nonlinearity, F_d or F_g, before scaling response baseline or gain, respectively (difference of exponentials) (Thorson et al., 2015). With additional constant terms $g_0$ and $d_0$, this model required a total of six free parameters. For comparison, we calculated a state-independent model here referred to as the null model, in which the state variable regressors were shuffled in time. Because shuffling removes any possible correlation between state and neural activity, gain and offset parameters are reduced to $d_0$=$g_0$=1 and $d_p$=$d_b$=$g_p$=$g_b$=0, effectively reducing the model to

In practice, fitting to the shuffled data produces parameter values slightly different from zero, and controls for noise in the regression procedure.

We also considered two partial models, one to predict responses based on pupil size only, $r_p\left(t\right)$, and the other to predict responses based on behavior only, $r_b\left(t\right),$ in which the other regressor was shuffled in time. Thus, the pupil-only model accounted only for effects of pupil size,

and the task-only model accounted only for task engagement,

These models tested the effects of a single state variable while ignoring the other.

By comparing performance of the full model to each partial model, we determined the unique contribution of each state variable to the neuron’s activity. We used a 20-fold, nested cross-validation procedure to evaluate model performance, which permitted using the full data set for both model fitting and validation without introducing bias from over-fitting. The model was fit to 95% of the data and used to predict the remaining 5%. Fit and test data were taken from interleaved trials. This procedure was repeated 20 times with nonoverlapping test sets, so that the final result was a prediction of the entire response. Model performance was then quantified by the fraction of variance explained, that is, the squared correlation coefficient, r², between the predicted and actual time-varying response. Variance uniquely explained by single state variables was calculated as the difference between r² for the full model and for the partial model in which the relevant variable was shuffled in time.

When comparing pupil and neural data, a 750 ms offset was applied to pupil trace to account for the lagged relationship between changes in pupil size and neural activity in A1 (McGinley et al., 2015).

To quantify the modulatory effects of task and pupil size on the firing rate of A1 and IC units, we computed a modulation index, MI (Otazu et al., 2009; Schwartz and David, 2018). The sound-evoked response was computed by averaging firing rate over the 750 ms duration of a stimulus presentation. MI was defined as the difference between the mean response to the same stimulus in two conditions, α and β, normalized by the sum,

MI could be calculated between different behavioral blocks or between state conditions. In the case of task engagement, MI was calculated between active and passive conditions, MI_AP. For pupil-dependent changes, data from an experiment were divided at the median pupil diameter, and MI_LS was computed between large pupil (above median, high arousal) and small pupil (below median, low arousal). To quantify the differences between the first and the second passives, MI_P1P2 was computed between the first and second passive blocks.

To quantify the changes in firing rates due to unique contributions of task condition or pupil size, we used MI to test how well the regression model could predict state-related changes in neural activity. The modulation between conditions α and β predicted by the full model is denoted as MI_αβ full, where α and β are either active/passive (MI_AP full) or large/small pupil (MI_LS full). Similarly, modulations between conditions α and β predicted by the pupil partial model or the behavior partial model are denoted MI_αβ pupil-only and MI_αβ task-only, respectively. The MI uniquely predicted by including task engagement as a regressor is

that is, MI predicted by the full model minus MI predicted by a model in which behavior condition, but not pupil, is shuffled. The net result is the MI predicted by task engagement above and beyond modulation predicted by changes in pupil size alone. Similarly, MI uniquely predicted by including pupil size as a regressor is

Significant effects of regressing out pupil size were quantified by comparing the signed-normalized differences between MI_AP task-unique and MI_AP task-only and differences of the same quantities across areas were quantified using a hierarchical bootstrap test (Saravanan et al., 2020). Sign normalization was achieved by multiplying the difference between MI_AP task-unique and MI_AP task-only in each unit by the sign of their mean.

Significantly modulated units were determined by comparing Pearson’s R coefficients associated with the full model and with the difference between the full model and the task and pupil partial models using jackknifed t-test with α = 0.05 (Efron and Tibshirani, 1986).

Several analyses assessed changes in the median of repeatedly sampled data, e.g., average pupil size measured across multiple behavior blocks (Figure 1) or average state-dependent modulation measured across multiple units (Figures 3 and 5–8). In this case, significant differences were assessed using a hierarchical bootstrap test (Saravanan et al., 2020). The hierarchical bootstrap is nonparametric, like the more traditional Wilcoxon signed-rank test, but it accounts for potential bias resulting from the relatively small number of array recordings in the data set. A statistical test that treats each unit as an independent measure could potentially be biased if a single array recording produced spuriously large effects, and the hierarchical bootstrap controls for this possibility. The bootstrap analysis was run for 10,000 iterations, so that the minimum measurable p-value was 10⁻⁵. Thus, results that returned p = 0 after this many iterations are reported as p < 10⁻⁵.

State-dependent changes in individual neurons were assessed using a combination of nested cross-validation and a jackknife t-test (see above, Efron and Tibshirani, 1986). To determine if any population-level effects depended on task performance or between animal differences, we used multivariate ANOVA (Figure 4).

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

This study was supported by NIH grants F31 DC014888 (DS), F31 DC016204 (ZPS), R01 EB028155 (SVD) and R01 DC0495 (SVD), by NSF grant GVPRS0015A2 (CRH) and by the Tartar Trust at Oregon Health & Science University (DS).

Animal experimentation: All procedures were approved by the Oregon Health and Science University Institutional Animal Care and Use Committee (protocol IP1561) and conform to National Institutes of Health standards.

© 2021, Saderi et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.