Introduction

Dopamine (DA) neurons originating in the ventral tegmental area (VTA) project to diverse forebrain regions, forming distinct but interacting neuromodulatory systems that are thought to play pivotal roles in the regulation of reward-related learning, motivation, and cognition (Sawaguchi and Goldman-Rakic, 1991; Schultz et al., 1993; Goldman- Rakic, 1995; Schultz et al., 1997; Tzschentke, 2001; Schultz, 2002; Pierce and Kumaresan, 2006; Berridge, 2007; Vijayraghavan et al., 2007; Lammel et al., 2008; Robbins and Arnsten, 2009; Hauber, 2010; Cohen et al., 2012; Salamone and Correa, 2012; Howe et al., 2013; Matsumoto and Takada, 2013; Hamid et al., 2016; Mohebi et al., 2019). A wealth of electrophysiological recordings from midbrain DA neurons, complemented by in vivo microdialysis data indicate that midbrain DA activity promotes behaviors associated with motivation (Wise, 2004; Berridge, 2007; Salamone and Correa, 2012; Howe et al., 2013; Matsumoto and Takada, 2013; Hamid et al., 2016; Mohebi et al., 2019) and supports reward-based learning by encoding reward prediction error (RPE) signals (Schultz et al., 1993; Schultz et al., 1997; Cohen et al., 2012).

Also, DA is of central importance to higher cognitive functions, such as updating and maintaining short-term memory (Sawaguchi and Goldman-Rakic, 1991; Miller and Cohen, 2001; Ott and Nieder, 2019). Pioneering behavioral studies which pharmacologically manipulated the activity of DA receptors in the PFC revealed the significant role of DA signals on short-term memory. In fact, an inverted-U-shape effect was discovered, where too little or too much DA receptor stimulation impairs PFC engaging short-term memory (Sawaguchi and Goldman-Rakic, 1991; Vijayraghavan et al., 2007; Robbins and Arnsten, 2009). Moreover, at the origin of the DA system, electrophysiological recordings at the VTA showed that DA neurons are not active in the delay period of memory tasks (Ljungberg et al., 1991; Schultz, 2002; Phillips et al., 2004; Matsumoto and Takada, 2013; Choi et al., 2020).

Motivated by the response of DA neurons to reward-related stimuli and memory delays, several lines of computational modeling studies sought to answer when and how DA signals support short-term memory “update” and “maintenance”. They proposed the “gating theory”, which provided a unified computational framework for reward prediction and short-term memory (Cohen et al., 2002; Dreher and Burnod, 2002; Montague et al., 2004; Ott and Nieder, 2019). According to the model, reward predicting cues, elicit phasic DA release which opens the gate for the afferent signals to be stored in memory (update). But, in the delay period, low, tonic DA levels close the gate for interfering signals to enter the PFC and overwrite the short-term memory component (maintenance). Although the “gating theory” fits adequately the behavior unique responses of DA neurons to the coding schemes of short-term memory, it relies mainly upon empirical evidence of the population activity of DA neurons and the longstanding consensus that short-term memory depends on the unbroken chain of persistent neuronal activity (Durstewitz et al., 2000; Curtis and D’Esposito, 2003).

However, recent advances in the study of the brain’s functional organization suggest that persistent neuronal activity might not be the only candidate mechanism for the active maintenance of goal representation over short delays, leading to the proposal of new coding schemes for short-term memory (Stokes, 2015; Miller et al., 2018). One of these candidate mechanisms regards the memory-dependent dynamic changes in functional connectivity. Neural oscillations are abundant in the mammalian brain and are thought to offer the networking framework for the temporal organization of neuronal activity and information processing in short-term memory (Uhlhaas and Singer, 2006; Buschman et al., 2012; Miller et al., 2018). Calculating the phase coherence of neural oscillations between distributed brain regions provides an estimation of the functional connectivity between them (Fries, 2005). Among other basal ganglia regions, the VTA engages dynamically in the large-scale network of brain systems that support memory related information processing. Simultaneous electrophysiological recordings were performed in the PFC and the VTA while rodents executed memory-guided behavioral choices in a T-maze task (Fujisawa and Buzsáki, 2011). Neural oscillations (4Hz) were prominent in both regions throughout the task, but their power and coherence were adaptively increased in memory delay. In the same task, another short-term memory related coding scheme was reported, this time at the single neuronal level. It was shown that while rodents navigate the maze, performing memory-guided decisions, PFC and parietal neurons differentiate their firing activities between opposite behavioral choices (Fujisawa et al., 2008; Harvey et al., 2012). Neuronal preference for choices was prominent in the delay period of the memory task but diminished in control non memory behavioral tasks. To summarize, this novel empirical evidence from rodent studies on the Tmaze behavioral apparatus complements the coding framework of short term memory with more dynamic and adaptive information-processing mechanisms other than persistent activity. Notably, by averaging neuronal population activities, none of these computational schemes would have been revealed.

In studying the role of DA neurons in short-term memory, we should take into consideration that the DA neuronal circuit is by no means self-contained and therefore it should not be investigated in isolation. Neurons utilizing GABA as a neurotransmitter constitute approximately 30% of the VTA neuronal population. The memory-related encoding properties of these inhibitory neurons have been largely overlooked, despite evidence of a strong inhibitory influence on neighboring DA neurons (Nair-Roberts et al., 2008; Omelchenko and Sesack, 2009; Tan et al., 2012; van Zessen et al., 2012) and well-established interconnections with the PFC circuit (Carr and Sesack, 2000a, b).

In light of the above, we wished to investigate with fine temporal and spatial resolution the firing activity of optogenetically identified DA and GABA neurons while mice performed a T-maze reward-seeking task with memory load. We took into consideration that (i) earlier studies analyzed either the population activity of putative DA neurons or drew inferences of the population activity from voltammetry measurements (Ljungberg et al., 1991; Schultz, 2002; Phillips et al., 2004; Matsumoto and Takada, 2013; Choi et al., 2020), and (ii) field potentials (like the 4Hz oscillations recorded in the VTA) stem mainly from phase-aligned excitatory or inhibitory post-synaptic potentials, whereas spiking activity is sparse (Traub et al., 2004; Buzsaki, 2006). (iii) A recent report revealed the causal relationship of DA activity with short-term memory, by inhibiting DA neurons with optogenetic tools. However, they did not report the encoding properties of single VTA DA neurons (Choi et al., 2020). Therefore, we looked for hints of memory-related encoding activities in single DA and GABA neurons by characterizing their firing preference for opposite behavioral choices.

Results

Optogenetic identification of DA and GABA neurons in the VTA

In the present study, we sought to investigate the encoding properties of DA and GABA neurons of the VTA while mice engage in memory-dependent reward-seeking behavior. To identify neurons, we expressed the light-gated cation channel, channelrhodopsin-2 (ChR2), in DA and GABA neurons by injecting an adeno-associated virus containing FLEX-ChR2 into DAT-Cre and VGAT-Cre transgenic mice, respectively (Bäckman et al., 2006; Tsai et al., 2009; Vong et al., 2011) (Figure 1A). Optogenetic identification and parallel electrophysiological recordings were performed using a custom-made diode-probe system (diode-fiber assemblies attached to high-density silicon probes(Stark et al., 2012), (Figures 1A and S1). For each neuron, we assessed the response to light pulse trains delivered before and after behavioral sessions (Figures 1 and fig. S1 and S2). We identified 104 neurons recorded from five DAT-Cre mice (hereafter referred to as DA neurons) and 74 neurons recorded from four VGAT-Cre mice (GABA neurons) with significant excitatory responses to light pulses (Figure 1B). Light-induced spikes from these neurons were almost identical to spontaneous spikes (waveform correlation coefficient > 0.9, Figure S1E and 1F). In addition, the electrophysiological profiles of the identified neuronal populations resembled those of previous studies (i.e., DA neurons fired action potentials with both wider waveforms and slower spontaneous firing rates than GABA neurons; Figure S1G), confirming the selective expression of ChR2 in DA and GABA neurons (Cohen et al., 2012; Tan et al., 2012).

Figure 1.

Behavioural Performance in a memory-dependent decision-making task

Mice were trained to perform sensory-guided and memory-dependent decisions in the “Memory Task” (Figures 2A, S3A and S3B). This task required animals to associate a visual cue presented at the beginning of the trial with a rewarded side arm of a figure eight T-maze. A short memory delay was introduced between cue presentation and action selection. Following a correct response, they received water (5 μl) from a waterspout located at the end of each arm. Depending on the individual features of cognitive demand, the maze apparatus was divided into separate sections (i.e., “start,” “cue,” “delay,” “side arms,” and “reward”). To ensure that the mice made choices guided by the visual cues and had minimal influence from other behavioral parameters on decisions, we eliminated imbalances between the left and right trials in key task parameters (e.g., reward amount, visual environment, effort, and motor skill requirements).

Figure 2.

At the time of neurophysiological data collection, all mice performed memory task trials with high accuracy. Averaging across sessions, the total correct rate was 86.8 ± 7.9% (mean ± standard deviation [SD]; left: 88.1 ± 10.0%, right: 87.2 ± 11.6%; paired t-test evaluating left vs right performance rate: t(59) = 0.46, P = 0.65, 60 sessions in nine mice, Figure 2B). In addition, performance was independent of individual preference for the left-or-right arm visits in any of the recorded sessions (test of independence, χ²(1) < 3.84, P > 0.05, Ho: correct rate is independent of arm choice, Figure S3C).

We also assessed the contribution of memory-related processing to task performance. To achieve this, we delivered blocks of trials with different memory loads in separate training sessions. Across all sessions, the correct performance rate dropped with higher memory load demands (mean ± SD; low load: 83.1 ± 8.3%, high load: 73.9 ± 9.5%; paired t-test on correct performance rate: t(12) = 4.33, P < 0.001, 13 sessions in seven mice, Figure 2C). This result is consistent with earlier reports (Floresco and Phillips, 2001; Floresco and Magyar, 2006) and highlights the important role of memory in supporting decisions in the present task.

The population activity of DA neurons is not elevated during the memory task trials

DA neurons are not known to be active in the delay period of short-term memory tasks (Ljungberg et al., 1991; Schultz et al., 1993; Phillips et al., 2004; Matsumoto and Takada, 2013; Choi et al., 2020), even though DA is a key neurotransmitter in the regulation of prefrontal cortical mnemonic functions (Goldman-Rakic et al., 1989; Smiley et al., 1992; Smiley and Goldman-Rakic, 1993; Goldman-Rakic, 1997; Tzschentke, 2001). This well-established notion has been mainly inferred from the average analysis of putative neuronal population activities and striatal voltammetry recordings (Ljungberg et al., 1991; Schultz, 2002; Phillips et al., 2004; Matsumoto and Takada, 2013; Choi et al., 2020). Corroborating these earlier reports, the average discharge rate of identified DA neurons in the present study remained essentially constant (Figure 2D). Simple linear regression analysis, with the neuronal firing rate as the response variable and the animal’s position on the maze as the single predictor variable, showed that from the beginning until the end of the trial (a 1.5-meter distance), the population activity of DA neurons deviated slightly by 0.17 ± 0.62 Hz (mean ± standard error of the mean [SEM], did not differ from a distribution with a mean equal to zero; one-sample t-test on the position coefficient, t₍₁₀₃₎ = 0.275, P = 0.78). Notably, in the memory-delay period, the discharge rate of DA neurons declined by - 0.72 ± 2.3 Hz (mean ± SEM, one-sample t-test on the position coefficient, t₍₁₀₃₎ = -0.31, P = 0.75). On the other hand, the GABA neurons elevated their discharge rate by 4.29 ± 1.10 Hz in the delay period (mean ± SEM, one-sample t-test on the position coefficient, t₍₇₃₎ = 4.09, P < 0.001), confirming evidence from an earlier report (Cohen et al., 2012).

DA and GABA neurons in the VTA show trajectory-specific encoding preferences in short-term memory-dependent behaviors

Making interpretations of the encoding properties of single neurons from population rate averages is highly challenging in tasks with many behavioral choices, especially for functionally heterogeneous populations such as the DA neurons. To overcome this limitation, we analyzed the firing activity of single neurons, by taking into consideration two important behavioral parameters. First, in every trial, the animals visited either the left or right rewarded side arms. Therefore, we grouped and averaged trial spike trains of single neurons by the corresponding lap trajectories (left or right; see also Methods and Figure S4A). Also, in the present task, significant behavioral events (including visual cue presentation, memory delay, and reward delivery) were inherently bound to fixed positions in the maze (Figure 2A, S3A and S3B). Thus, we arranged spiking events according to their position, to get an estimate of the behavioral correlates of neuronal activity. To this end, individual trial trajectories were linearized and represented as a one-dimensional vector consisting of 100 linearly spaced points (trial start: point 0; trial reward: point 100).

Examples of discharge patterns arranged by position and trajectory are shown in Figures 3A, S5A and S5B. These representative neurons differentiate their responses between left and right trials at certain maze positions in a robust manner. To assess the trajectory-specific effects on neuronal firing activity, we used the permutation method (Figure S4). First, we calculated the original difference between the average firing rates in the left and right trials. We then randomly reassigned the trajectory labels (left or right) on the trial spike trains and produced the permuted average firing rate differences. If neurons were modulated by trajectory, the original and permuted firing rate differences were significantly different. Since spiking events are arranged by position, the permutation method can also detect positions with significant differences.

Figure 3.

Figure 3B summarizes the results from the permutation analysis applied to the populations of 104 DA and 74 GABA neurons. In the left and middle heatmaps, we organized the normalized mean firing rates for the preferred and non-preferred lap trajectories (i.e., trajectories with higher and lower firing rates respectively). The right heatmap contains the maze positions with significant firing rate differences. In both neuronal populations, there was abundant trajectory-specific activity, concentrated mostly in the delay and side-arm sections. Almost 20% of DA neurons differentiated their response between left or right trajectories in those maze sections (21% in the delay section and 22% in the side-arm section, 104 neurons, permutation test, P < 0.05, Figure 3B and Table S1). In GABA neurons, the percentage was even higher, with almost 50% of these cells eliciting trajectory-specific activities (47% in the delay section and 47% in the side-arm section, 74 neurons, permutation test, P < 0.05, Figure 3B and Table S1).

There have been reports of DA neurons discriminating between visual cues in a T-maze task (Engelhard et al., 2019) or choice selections in delayed-match-to-sample tasks (Matsumoto and Takada, 2013; Choi et al., 2020). However, in the present study, VTA neurons did not exhibit different levels of activity in the cue section. Furthermore, neuronal preference for trajectories was not restricted to the turning point, which could indicate neuronal engagement in motor preparation for choice execution. Instead, it was spread in a wider area, covering a distance from the memory delay onset until the end of the side arms.

A plausible explanation for the trajectory-specific responses in the side arms is that neurons were under the control of the sensory, motor, or goal-directed behavioral processes triggered by the opposite trajectories (Howe et al., 2013; Hamid et al., 2016; Mohebi et al., 2019). However, in the memory-delay section, trajectories were identical for the left and right trials, which could be suggestive of the engagement of these neurons in short-term memory processing. Neuronal preferences to arm visits in memory delay are not uncommon in T-maze tasks. They have been reported in prefrontal and post-parietal cortical neurons and have been attributed to short-term memory-dependent decisions (Fujisawa et al., 2008; Harvey et al., 2012). So, is the trajectory-specific activity in our task reminiscent of internally generated, memory representations, or can be attributed to the well-known DA-linked neuronal computations (Schultz, 2002; Cohen et al., 2012; Berke, 2018; Engelhard et al., 2019)? To test this hypothesis, we proceeded to a series of statistical analyses and control behavioral tasks.

Multiple regression analysis confirms the trajectory-specific effect on DA and GABA neurons

We discovered that significant proportions of VTA neurons fired preferentially for left or right trajectories at specific locations on the maze when we arranged discharge patterns by arm visit and position. This result does not attest that trajectory and position alone contribute to the neuronal firing rate. Midbrain DA neurons are known to respond to a wide variety of behavioral parameters (i.e., choice accuracy, reward history, running speed, and distance to rewards (Engelhard et al., 2019)) which could also exert a significant effect on neuronal firing activity. However, their effect could be dampened due to the specific firing range arrangement.

Since these behavioral variables are difficult to control with behavioral tasks, we assessed their contribution to neuronal responses using multiple regression analysis (Figure S6 and Supplementary Text 1). We found that all the examined variables (lap trajectory, trial number, speed, trial accuracy, and reward history) contributed to the firing activities of neuronal subpopulations; however, only the lap trajectory predictor could explain better the trajectory-specific activities observed in the ∼20% of DA and ∼50% of GABA neurons that were identified with the permutation analysis.

Memory-dependent but not motivated behavior is related to trajectory-specific activity in VTA neurons

Next, we investigated the contribution of short-term memory in decision-making on the trajectory-specific activity of VTA neurons. Memory-dependent decision-making depends on three major computational components. These are (i) sensory input gating, (ii) maintaining and manipulating memory contents and (iii) generating and executing appropriate motor plans (Cohen et al., 2002; Dreher and Burnod, 2002; Montague et al., 2004; Ott and Nieder, 2019).

We eliminated all three components in a variation of the memory task. Specifically, we trained mice in the no-cue-no-choice task, in which they were not presented with a visual cue and, therefore, could not make predictions about the location of the reward (Figures 4A, S3A and S3B). Furthermore, the choice selection was prevented by the presence of blocked side arms when they arrived at the T-intersection. After a short delay (approximately 1 s), access to one of the side arms (chosen pseudo-randomly) was permitted, which always led to a reward.

Figure 4.

In recording sessions, mice received mixed protocols composed of randomly interleaved memory task and no-cue-no-choice task trials. We evaluated the trajectory-specific activities on each task separately using the permutation method. In the delay section of the no-cue-no-choice task, we observed a significant reduction in the number of positions with a significant firing rate difference (mean ± SD; DA: memory task 5.9 ± 3.8 points, no-cue-no-choice task 1.5 ± 3.4 points, paired t-test, P = 0.002, four animals; GABA: memory task 10.7 ± 3.9 points, no-cue-no-choice task 1.5 ± 2.0 points, paired t-test, P < 0.001, three animals, Figures 4B to 4E). The attenuating effect on trajectory specific activity was also reflected by a marked reduction in the number of trajectory specific neurons (Figure 4E, numbers in parentheses). However, the firing rate difference between the left and right-side arms was strong in both tasks (DA: memory task 8.5 ± 7.8 points, no-cue-no-choice task 5.9 ± 7.4 points, paired t-test, P = 0.139, four animals; GABA: memory task 12.6 ± 8.1 points, no-cue-no-choice task 9.9 ± 8.8 points, paired t-test, P = 0.234, three animals, Figures 4B to 4E).

However, the significant reduction in trajectory-specific encoding preference in the no-cue-no-choice task could not be entirely attributed to the absence of the memory component. This is because important running speed, motor responses, and motivational discrepancies exist between memory and no-cue-no-choice tasks. With regard to motivation, the important role of DA in adaptive decision-making is widely recognized (Hamid et al., 2016; Berke, 2018; Mohebi et al., 2019). We did not observe animal choice bias in memory task performance (Figures 2B and S3C), but we cannot rule out the possibility that individual neurons were modulated differently by effortful actions to reach the left- and right-sided rewards (Figure S7). Unlike the memory task, in the no-cue-no-choice task, the mice could not direct behavior towards the left- or right-side arms due to the absence of a visual cue. As a result, they were unable to allocate incentive motivational drives to the left-or-right trials (Howe et al., 2013; Hamid et al., 2016; Berke, 2018; Mohebi et al., 2019). Also, the initial access denial to the side arms in the control task eliminates any potential differences in the motor preparation coding schemes (according to the “gating theory”) for the opposite arm visits in the memory task (Engelhard et al., 2019; Ott and Nieder, 2019). Finally, regarding speed, a representative example of running speed differences between the two tasks within a single session is shown in Figure 4F.

To dissociate the short-term memory component of neuronal activity from the modulatory effects of running speed, incentive motivation, and motor-related signaling, we trained mice in a second control task. The cue-no-choice task preserved the same running speed parameters (Figure S8), motor skill requirements, and physical effort demands (i.e., visual cues, maze shape, arm length, and reward amount were the same) as the memory task, but it prevented animals from making decisions. Accordingly, the animals were presented with the same visual cue as in the memory task, which indicated the side arm that was rewarded and enabled them to allocate incentive motivational drive to the left-or-right trials; however, they were always forced to visit the rewarded arm by blocked access to the unrewarded arm (Figures 5A, S3A and S3B). In the same recording session, the mice performed a separate block of memory task trials. Similar to the first control task, in the delay section of the cue-no-choice task we observed a significant reduction in the spatial extent of the firing rate difference (DA: memory task 5.4 ± 3.6 points, cue-no-choice task: 0.4 ± 1.1, paired t-test, P = 0.011, four animals; GABA: memory task 10.7 ± 6.3 points, cue-no-choice task 3.5 ± 4.2 points, paired t-test, P < 0.001, 1 animal, Figures 5B to 5E). In the side arms, however, the trajectory specific effect remained strong and was not significantly different from the effect observed in the memory task (DA: memory task 8.1 ± 10.3 points, cue-no-choice task 5.0 ± 4.6 points, paired t-test, P = 0.146, four animals; GABA: memory task 8.7 ± 5.9 points, cue-no-choice 5.2 ± 6.2 points, paired t-test, P = 0.086, 1 animal, Figures 5B to 5E).

Figure 5.

Together, these results suggest that trajectory-specific responses in the delay period of the memory task could reflect short-term memory representations linked to decision making behavior and cannot be explained by running speed, motor, and motivation related signaling differences.

Neuronal activities in delay and reward are unrelated

It is widely recognized that DA and GABA neurons of the VTA respond to reward and reward-predicting cues (Schultz et al., 1993; Schultz et al., 1997; Cohen et al., 2012; Matsumoto and Takada, 2013; Engelhard et al., 2019; Choi et al., 2020). Therefore, we examined whether the selective neuronal responses to left-or-right trials in the memory delay and side-arm sections could be attributed to encoding biases related to rewards. We defined the first second of reward consumption as the reward section.

DA neurons are known to be excited by rewards (Schultz et al., 1993; Schultz et al., 1997; Cohen et al., 2012; Matsumoto and Takada, 2013; Engelhard et al., 2019; Choi et al., 2020). In agreement with this notion, we discovered that 27 DA neurons (28% of 104 neurons, Figure 6B column 4) responded to reward with significant excitation, which is a high proportion considering the low uncertainty of the task due to the high performance rate and overtraining.

Figure 6.

We subsequently examined whether VTA neurons could discriminate between left- and right-arm rewards. DA neurons are known to discriminate between rewards with different magnitudes and predictabilities(Tobler et al., 2005; Morris et al., 2006). The animals were offered equivalent options in terms of reward magnitude, uncertainty, and effort. Thus, we predicted the presence of a small number of reward-discriminating neurons. However, we found that 23% of DA neurons and 46% of GABA neurons differed significantly in their responses to left-or-right rewards (Figures 6A, 6B column 3, and Table S1; paired t-test comparing mean firing rates, P < 0.05). Therefore, we assessed the relationship between encoding preferences in the reward section and those in the remainder of the maze by correlating the average firing rate difference in the reward section with the average rate difference in the preceding maze sections. In both neuronal populations, we found a significant positive relationship between the reward and side-arm sections (Figure 6C; Pearson’s correlation; DA: R = 0.31, P < 0.001; GABA: R = 0.67, P < 0.001) but not between the reward and delay sections (Figure 6C; Pearson’s correlation; DA: R = -0.05, P > 0.05; GABA: R = -0.03, P > 0.05). These results demonstrate that trajectory-specific firing activities in memory delay do not reflect reward preference during consumption.

Discussion

In the present study, we performed extracellular recordings from optogenetically identified DA and GABA neurons in the VTA while mice performed reward-seeking tasks on a T-maze apparatus. Mice were trained to choose between two spatially separate goals under the instruction of visual cues presented at the beginning of the trials. A short memory delay was introduced between cue presentation and choice selection. We discovered that subpopulations of DA and GABA neurons showed differential responses between the left and right trials, starting from the onset of the memory delay period in the main arm, where the trajectories were indistinguishable. Trajectory-specific preference was not correlated with reward history, running speed, the incentive motivational drive of physical effort, or reward-related encoding differences, and diminished significantly when the memory-dependent decision component was eliminated in control behavioral tasks. This evidence indicates that populations of DA and GABA neurons in the VTA encode internally generated signals that support short-term memory in decision-making.

Activities of midbrain DA neurons in short-term memory

The well-established notion that DA somatic spiking activity is low in short-term memory stemmed mainly from firing rate averages of putative neuronal populations (Ljungberg et al., 1991; Schultz, 2002; Phillips et al., 2004; Matsumoto and Takada, 2013; Choi et al., 2020). Consistent with those reports we did not observe a profound variation in the population activity of DA neurons during the memory task. However, a wealth of recent studies has shown that DA neurons are functionally and genetically segregated (Lammel et al., 2008; Lammel et al., 2011; Engelhard et al., 2019).

Moreover, in many real-life situations, animals have to choose between many options for behavioral responses. In such behavioral conditions, averaging firing rates cannot reveal the fine computational processes of single neurons. To address this limitation, we analyzed the firing activities of identified single neurons, focusing on different discharge patterns between behavioral choices. Here, we demonstrated that memory-specific activities by midbrain DA neurons can be represented as trajectory-specific responses in the delay period of the memory task.

The “gating theory” unifies the signaling activities of DA neurons in reward prediction and short-term memory (Cohen et al., 2002; Dreher and Burnod, 2002; Montague et al., 2004; Ott and Nieder, 2019). Our evidence does not question the validity of this computational model, since we do not provide evidence of how the selective preference for one response over the other translates into the release site. However, it is worth noting that trajectory-specific activities we observed here were rare in the visual-cue section when the updating of sensory information takes place. Moreover, firing rate differences between left and right-arm responses declined in a control behavioral task (cue-no-choice) without memory load, but with the same motor skill requirements as the memory task. Thus, motor preparation coding schemes for different responses (Cohen et al., 2002; Ott and Nieder, 2019) cannot be accounted for the trajectory-specific activities in the memory delay. Instead, our evidence indicates that trajectory-specific activities by DA (also GABA) neurons are functionally linked to the maintaining and manipulating of memory contents.

Memory-related, trajectory-specific neuronal activities have also been reported for PFC and post-parietal neurons while rodents perform reward-seeking responses on the T-maze (Fujisawa et al., 2008; Harvey et al., 2012). Furthermore, in the same task, both the PFC and VTA circuits are functionally engaged with coherent 4-Hz oscillations in the memory delay (Fujisawa and Buzsáki, 2011). Our results complement this line of work, demonstrating that also DA and GABA midbrain neurons elicit memory-related, trajectory-specific activities.

The present study also corroborates important findings from a recent report, which demonstrated that optogenetic perturbations in DA neuron excitability exert a strong effect on short-term memory performance, highlighting the causal role of DA neuronal firing activity in memory-dependent behavior (Choi et al., 2020). Also, in agreement with a previous report (Engelhard et al., 2019) by the same laboratory, we show that subpopulations of VTA neurons are modulated by running speed, cumulative performance rate, current choice accuracy, and reward history. Disparities between this and our study in the proportions of modulated neurons could be attributed to the different recording techniques applied as well as the maze regions of interest; for example, Engelhard et al. analyzed neuronal firing activities in the visual-cue period (Engelhard et al., 2019), whereas we focused on memory delay.

Overall, our results are in agreement with the notion that DA neurons encode a variety of behavioral parameters in complex environments. In addition, we confirmed that in memory-dependent behaviors, DA neuronal populations did not elicit sustained increases in their discharge rate. However, in the present task, DA neurons individually encoded internal representations by differentiating their responses to lap trajectories in memory delay.

Role of motivated behavior in trajectory-specific encoding properties of VTA neurons

Midbrain DA activity is known to be involved in motivated behavior while rodents navigate mazes to receive rewards in trials lasting a few seconds (Hamid et al., 2016; Berke, 2018; Mohebi et al., 2019). When mice approach rewards, striatal DA concentrations increase, scaling flexibly with reward size and proximity, which is proposed to reflect a neural correlate of a sustained motivational drive (Howe et al., 2013). To evaluate the role of motivated behavior in the trajectory-specific preference of midbrain neurons, we compared firing activities between a memory task and a control task without memory-dependent decisions (cue-no-choice task). Although in the cue-no-choice task, the behavioral parameters that determined the incentive motivational drives were the same as in the memory task (visual cues, maze shape, and reward amount), neuronal responses did not differ between the left and right trials. This result strongly indicates that incentive motivational drives (at least for physical effort) do not contribute to trajectory-specific activities of midbrain neurons during the delay period of the memory task.

Memory-specific activities of the VTA neurons are not attributed to reward prediction error signaling

We also assessed the role of reward-related processing in the trajectory-specific activity of the midbrain neurons. In behavioral tasks in which animals estimate the spatial proximity of distant rewards, it has been suggested that DA neurons calculate RPE signals from state-value functions (Hamid et al., 2016; Berke, 2018; Engelhard et al., 2019; Mohebi et al., 2019; Kim et al., 2020). In the present study, the animals received ongoing visual input, facilitating the continuous estimation of reward proximity. Thus, DA neurons can potentially estimate RPE signals from successive state values assigned to each position on the maze track (Figure S7). Therefore, the difference in firing activity between the left and right trials could be the result of differences in the state value functions assigned to these trajectories (Hamid et al., 2016; Berke, 2018). However, significant evidence contradicts this hypothesis.

First, the behavioral parameters that determine the state-value functions for the left and right trajectories were set to be identical in the cue-no-choice task and memory task, by preserving the same maze configurations and delivering equal amounts of reward. In addition, behavior in both tasks was cue-driven; therefore, animals could make predictions about the reward location and orchestrate behavior accordingly. However, we observed a prominent reduction in the firing rate difference between the left and right trials in the cue-no-choice task (Figure 5). Second, a significant subset of DA neurons (approximately 20%) responded differently to the left and right rewards in the memory task, although the same amount of reward was delivered. This unexpected finding raised the hypothesis that the encoding preference for reward could be reflected in the values of the preceding states in the maze and, therefore, could account for the trajectory-specific effect in memory delay. However, the differences in the firing activity elicited by the consumption of left-or-right rewards were unrelated to the firing rate difference in the delay section (Figure 6C). In conclusion, these findings indicate that the encoding preference for lap trajectories exhibited by midbrain DA and GABA neurons cannot be simply explained by discrepancies in RPE signaling.

GABA neurons of the VTA and short-term memory

With the advent of highly selective identification and perturbation techniques, new evidence has emerged regarding the encoding properties and functional roles of local VTA inhibitory networks in reward processing and motivation. There are reports demonstrating that GABA neurons of the VTA suppress reward consummatory behavior (van Zessen et al., 2012), facilitate aversive behavior (Matsumoto and Hikosaka, 2007; Tan et al., 2012), and elicit sustained activities in the delay period between conditioned and unconditioned stimuli (Cohen et al., 2012). During these behaviors, the responses of DA and GABA neurons are often inverse, such that when GABA neurons are excited, neighboring DA neurons decrease their discharge rate. In particular, aversive stimuli excite GABA neurons, which then suppress the neighboring DA neurons (Tan et al., 2012). In addition, during reward consumption, GABA neurons are inactive (Cohen et al., 2012; van Zessen et al., 2012); however, when excited, they inhibit DA neurons and disturb consummatory behavior (van Zessen et al., 2012). Finally, in classical conditioning tasks, DA neurons respond to rewards and reward predicting stimuli, whereas GABA neurons remain silent during such events (Cohen et al., 2012). However, we demonstrated here that midbrain DA and GABA neurons elicit remarkably similar encoding properties. Both neuronal populations respond to short term memory-specific activities manifested by encoding preferences for lap trajectories. Notably though, GABA neurons are more strongly engaged in this dynamic encoding activity since almost twice as many inhibitory neurons responded differently to the left and right trials.

This result presents an activity paradox. Given the abundant and potent synaptic inhibition of DA neurons by neighboring GABA neurons (Omelchenko and Sesack, 2009), it was unexpected that both populations were highly active and similarly engaged in tasks. However, anatomical evidence provides a plausible explanation. Local inhibitory neurons form a dense network of local synaptic innervations that target the dendritic sites of DA and other GABA neurons (Traub et al., 2004; Buzsaki, 2006; Omelchenko and Sesack, 2009). Although potent and well-suited for coordinated network activity, this synaptic inhibition is not as strong as somatic inhibition (Jhou et al., 2009; Omelchenko and Sesack, 2009), and it has been suggested that it is not sufficient to suppress DA neurons when they receive a strong excitatory drive from extrinsic sources (van Zessen et al., 2012).

Conclusion

In summary, we optogenetically probed DA and GABA neurons in the VTA while mice performed a decision-making task with memory load. We discovered that both of these neuronal populations elicited memory-dependent preferences for left-or-right trajectories that could not be explained by motor activity, motivated behavior, or reward-related processes. This evidence indicates that VTA neurons encode mental representations to support short-term memory-dependent decisions and provides insights into novel sophisticated coding strategies employed by the midbrain DA and GABA neurons in reward-related behavior.

Methods & Materials

Key resources table

Contact for reagent and resource sharing

Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Dr. Shigeyoshi Fujisawa (shigeyoshi.fujisawa@riken.jp).

Animals

All experiments were approved by the RIKEN Institutional Animal Care and Use Committee. We used five adult male DAT-ires-Cre (Jackson’s Lab; stock #6660; (Bäckman et al., 2006) and four Vgat-ires-Cre (Jackson’s Lab; stock #16962; (Vong et al., 2011) mice backcrossed to C57BL/6J. Animals were housed in separate cages on a 12-h dark/light cycle and each performed the behavioral tasks at the same time of the day, between 11:00 and 17:00. In the cage, they were provided ad libitum food access but were restrained from water availability.

Behavioral tasks

All behavioral tasks took place on a T-maze apparatus. More information about the maze configuration is provided in Figure S3.

Memory task

Behavioral sessions commenced with the animal being placed at the “starting position” (Figures 2A, S3A and S3B). Then, access to the main corridor was provided and the animal had to run through the “start” section (0-50 cm) before it arrived at the maze segment surrounded by two PC monitors (“visual-cue” section, 50-80 cm). In this section, it was presented with a distinctive visual object (vertical black and gray bars) in one of the two monitors (the other monitor remained dark) indicates which side arm to visit to obtain the reward (i.e., left cue → left reward, right cue → right reward). In the third region of the central arm (“delay” section, 80-120 cm) both monitors turned dark. While running in the delay section, the animal had to maintain the reward-related information and based on that perform the action selection at the T-intersection. The intersection at the end of the main arm designated the end of the delay section and the beginning of the “side-arms” section (120-150 cm) where the animal runs towards the reward position in anticipation of the reward. Reward (5 μl water) was delivered on correct trials at the end of the side-arms section from a waterspout. The first activation of the light-beam sensor at the waterspout triggered the water-delivery pump (Burkert, Ingelfingen, Germany), followed by reward consumption (“reward” section). After consuming the reward, the animal could return of its own will to the starting position, to commence a new trial.

Daily behavioral sessions consisted of 80-100 trials. Only animals with at least three successive sessions with an 80% performance ratio or more in the training phase were been proceeded to surgical operations.

No-Cue-No-Choice task

Trials of this control task were delivered in recording sessions, interleaved with memory task trials. When the animal entered the visual-cue section it was not instructed by the visual cue (Figures 4A, S3A and S3B). Also, access to both side arms was initially denied by closed sliding doors. Approximately 1 s after the animal arrived at the turning point, one of the sliding doors opened (pseudo randomly) providing access to the reward. On every trial, the presentation or absence of the visual cue could instruct the animal about the task rules (i.e., memory task or no-cue-no-choice task).

Cue-No-Choice-task

The settings of this control task were the same as the memory task settings, except for the blockade of the unrewarded side arm (Figures 5A, S3A and S3B). Thus, the animals were always forced to perform correct choices. Because in both tasks, the same cue was presented, the animals could potentially be confused about the trial’s task rules. Therefore, memory task and cue-no-choice task trials were delivered in separate sets within the same recording session. Accordingly, when the animals completed the set of cue-no-choice task trials (approximately 50 trials), they were automatically delivered with another set of memory task trials (approximately 50 trials).

Recording sessions lasted approximately 20-30 minutes.

Intracranial surgeries and electrophysiological recording

The surgical process consisted of two separate operations. First, mice (DAT-ires-Cre or Vgat-ires-Cre) were surgically injected with 200-500 nl of adeno-associated virus AAV5-EF1a-DIO-hChR2(H134R)-EYFP-WPRE-pA (University of North Carolina vector core facility; (Tsai et al., 2009) into the VTA stereotaxically (from inferior cerebral vein AP: ∼6.65 mm, from midline ML: - 0.55 mm on the left hemisphere, from surface 4-4.5 mm, Figure 1A). Ten to fifteen days later, mice were implanted with a silicon probe in the same AP and ML coordinates (vertical insertion was intended, 0 degrees; Figures 1A, S1A to S1D). We used Buzsaki64spL (NeuroNexus, Ann Arbor, MI, USA) silicon probes which are composed of 6 shanks (10 mm long, 15 μm thick, 200 μm shank separation) and each shank has 10 recording sites (160 μm² each site 0.6-1.0 MΩ impedance). The silicon probe was attached to a custom-made micromanipulator and moved gradually to the desired depth position. On every probe shank, optic fibers were firmly attached to secure an accurate and firm insertion of the recording channels in the deep midbrain area (Figure S1B). For experiments where light delivery was required, two of the optic fibers (shanks 2 and 5) were coupled with blue (450 nm) laser diodes (PL450B, OSRAM Opto Semiconductors). Light dispersion could potentially cover the axial and transverse span of all 64 channels (Stark et al., 2012).

During recording sessions, the wide-band neurophysiological signals were acquired continuously at 20 kHz on 256-channel Amplipex systems (KJE-1001, Amplipex Ltd, Hungary; (Berényi et al., 2014)). Following surgery, the probe was inserted 45 μm deeper in intoe brain on a daily basis, until it reached the VTA. Thereafter, the probe was moved deeper by 20 μm / day. The average recording coordinates for the DAT-Cre animals are 3.32 ± 0.32 mm (mean ± standard deviation) rostrocaudal and 0.82 ± 0.17 mm mediolateral, and for the VGAT-Cre animals, 3.52 ± 0.29 mm rostrocaudal and 0.90 ± 0.17 mm mediolateral (Figure S1A).

We cannot exclude the possibility that some neurons were recorded in successive sessions because clustering analysis was performed on individual sessions.

Data analysis

Unless otherwise stated, data analysis was performed with custom-made programs designed in MATLAB with Signal-processing and Statistics toolboxes.

Light-stimulation protocols for optogenetic identification

Light stimulation protocols were delivered before and after the behavioral tasks. They were composed of 1, 2, 3, and 4 mW blocks of 450 nm light pulses. Each block consisted of 150 square pulses (12 ms pulse duration; 0-1 ms and 11-12 ms contained artifacts) delivered at 1, 2, 3 to 10 Hz. Electrophysiological data recorded during light stimulation and behavioral protocols within a single session were merged and clustered together.

Statistical analysis for detection of light-responsive units

Neurons with light-induced responses exceeding the average spontaneous activity were classed as light-responsive. To identify light-responsive neurons we applied the statistical analysis described in detail in Figure S2.

Estimation of firing activity during behaviour

To estimate the neuronal firing activity while animals performed the behavioural task, we took into consideration the primary goal of this study; that is to look for trajectory-specific encoding properties, as well as the inherent limitation of the task; that is the experimenter could not control the temporal precision of the behavioural events. To overcome this limitation, we arranged firing activity by the animal’s position on the maze. To do so, first, we linearized the trial trajectories and assigned them with a lap trajectory label (left or right). Then, the linearized products were divided into 100 position points and normalized so that position 0 corresponded to the starting point of the trial and position 1 to the waterspout. Second, we constructed post-distance histograms, analogous to the peri-stimulus-time-histograms (PSTHs), although the time of spiking events was replaced by the position they occurred (for the purpose of simplicity, also by habit, we will call the post-distance histograms as PSTHs). To construct accurate PSTHs we considered the exact position the spikes were discharged and the time the animal occupied this certain position. Let n^(k)(x) be the number of spikes of a single neuron and t^(k)(x) be the occupation time in the x^th position point of the k^th trial (Figure S4A). Then, where K is the number of trials, represents the average firing rate probability (spikes / sec) at position point x. To examine the trajectory-specific encoding properties of VTA neurons we produced average firing rate histograms for correct left and right trials, separately. Then, both histograms were smoothed with a Gaussian Kernel function (σ= 0.5, length of 20 position points).

Firing rate heatmap construction

To construct the normalized firing rate heatmaps shown in Figures 3B, 4C, 4D, 5C, 5D, 6B, S5C to S5E and S6B we took the following steps. First, for every neuron we produced the average firing rate for left and right correct trials. Second, we normalized both rates by dividing them with the maximum firing rate of the strongest trajectory response (e.g., for the example shown in Figure S4A we divided both average firing rates by the maximum rate of the response to the left trials). Then, the normalized rate of the stronger trajectory response was assigned to the “preferred” heatmap and the rate of the weaker trajectory response to the “non-preferred” heatmap (e.g., for the example shown in Figure S4A, the left normalized rate was assigned to the “preferred” heatmap and the right rate to the “non-preferred” heatmap). Both rates occupied the same row. The row ordering was determined by the position of maximum rate.

Identifying trajectory-specific neurons with the permutation method

To identify neurons with trajectory-specific encoding properties we applied the permutation test reported elsewhere (Fujisawa et al., 2008) and described in detail in Figure S4.

Regression analysis

We designed a generalized linear regression model (GLM) with the neuronal firing rate (FR) modelled as a gaussian function of the lap trajectory (T), speed (S), trial number (TN), performance (R), current trial accuracy (A₀) and previous trial accuracy (A_-1) behavioural variables. With the permutation analysis we observed that the trajectory-specific effect on the firing activity was dependent on position. Thus, we examined the joint effect of trajectory with position (P) on spiking activity. All dependent and independent variables were arranged by position. The values of the trajectory (1 for left and 2 for right), trial number, performance (cumulative correct rate), current trial and previous trial accuracy (1 for correct trial, 0 for error trial) variables remained constant throughout the whole trial. The firing rate, position and speed variables changed their values on every position.

The GLM was:

where the β values are the regressor coefficients for the different predictors (including the intercept β₀) and ε is the Gaussian noise term. The 6^th degree order polynomials of position and speed were chosen for model optimization with the Bayes information criterion.

First, we generated model predictions of the average firing rates for left and right trials, and from those we calculated the predicted firing rate difference Then, we shuffled the trajectory labels assigned to the tested variable (the assigned labels to the rest of the independent variables remained intact) and assessed the effect on the firing rate difference. For every predictor we produced 500 shuffled rate differences, If the absolute mean value of exceeded the top 5% of the values (including Bonferroni correction), then the hypothesis was rejected, and the predictor was significantly contributing to the firing rate difference. We examined every maze region individually, but, here we report only for the delay region.

Reward-related excitation or inhibition

The reward section was defined as the first second of reward consumption. To assess neuronal response to reward consumption and categorize it as excitatory, inhibitory or non-responsive we performed the following analysis. First, we produced the smoothed mean firing rate response in the time domain (as we did in the maze sections in the space domain) for left, and right, trials. For the preferred arm of each neuron, we compared the mean firing rate in the reward section to the mean rate in the 100 ms epoch preceding reward delivery (paired t-test on mean firing rates; P < 0.05; Figure 6B column 4).

Encoding preferences in the reward section

The difference in the intensity of neuronal firing activity between left and right rewards was assessed by comparing the mean firing rate of neuronal activity elicited in the reward section of left and right trials (paired t-test on mean firing rates; P < 0.05; Figure 6B column 3).

Relationship of encoding preferences in the reward section to those in the remainder of the maze

To assess whether the trajectory-specific firing activity in the maze was linked to discrepancies in the response to left and right rewards, we followed the next steps of analysis. First, for every neuron and every maze section, we calculated the mean value of the relative firing rate difference between left and right trials Then, for each neuronal group, we calculated the linear relationship (Pearson’s correlation) between the reward section mean values, to those in the remainder of the maze (Pearson’s correlation; Figure 6C).

Immunohistochemistry

After completion of the recording sessions, which lasted about a month, mice were anesthetized with isoflurane and perfused transcardially with 10 ml PBS and 10 ml paraformaldehyde (4%), before they were decapitated. Brains were then removed, post fixed and coronal slices (100 μm) were prepared. The primary antibodies used were rabbit anti-tyrosine hydroxylase (TH) and chicken anti-GFP. The secondary antibodies used were AlexaFluor 549 anti-rabbit and 488 IgG anti-chicken, respectively. Sections were further stained with DAPI to visualize nuclei. Image acquisition was performed with a fluorescence microscope NanoZoomer (Hamamatsu, Japan) system.

Acknowledgements

This study was supported by the Ministry of Education, Culture, Sports, Science, and Technology (Grants-in-Aid for Scientific Research 18H02711 and 18H05525), the Mitsubishi Foundation, the Naito Foundation, and the Japan Agency for Medical Research and Development (AMED).

In memory of Miles Adrian Whittington. A true mentor and friend.

Author contributions

V.G. and S.F. designed the experiments. V.G. performed the experiments and analyzed the data. V.G. and S.F. wrote the paper.

Competing Interest Statements

The authors declare that they have no competing interests.

Supplementary Materials for

Supplementary Texts

Supplementary Text 1. Evaluating the contribution of behavioral variables in trajectory-specific activity with multiple regression analysis

We designed a generalized linear regression model in which the dependent variable was the position-arranged firing rate, and the independent variables (predictors) were the animal’s running speed, trial number, performance rate (cumulative correct rate), current trial accuracy (reward or not), previous trial accuracy (reward history), and lap trajectory (left or right). Since the permutation analysis of the original spike trains revealed that the trajectory-specific effect on neuronal firing activity was highly dependent on the animal’s position (occurring mainly in the delay and side-arm sections), we included the joint effect of lap trajectory and position in the training model instead of testing the effect of trajectory alone. A major advantage of regression analysis is that one can dissociate the inherently bound effects of the independent variables. For example, we can examine the influence of the lap trajectory variable on neuronal firing activity without including the effect of the running speed variable (which could potentially differ between left and right trials).

For each neuron, we produced model predictions for the correct left and right trial average firing rates (Figure. S6A, dashed lines) from which we calculated the predicted firing rate difference. We subsequently examined the contribution of each independent variable to the trajectory-specific firing activity by shuffling the trajectory labels assigned only to this particular variable. If the tested variable exerts a significant effect on the firing rate, then shuffling the trajectory labels would produce a significant reduction in the predicted firing rate difference. We examined the same pool of neurons that we reported on the memory task, as shown in Figure 3B (n=104 DA and n=74 GABA-identified neurons). In the delay region, the joint effect of lap trajectory and position (trajectory × position predictor) contributed significantly to the predicted average rate difference of 23 DA neurons (Figure S6B and S6C top), 17 of which overlapped with the 22 trajectory-specific neurons identified with the permutation analysis (Figure S6C bottom). Of the GABA neurons, 34 were modulated by trajectory and position (Figure S6B and S6C top), 30 of which were also trajectory-specific (Figure S6C bottom). These results confirm that the trajectory factor was responsible for the firing rate difference shown in Figure 3B. The speed variable significantly modulated 24 DA and 22 GABA neurons (Figure S6B and S6C top), but only four DA and four GABA neurons were co-modulated by trajectory (Figure S6C bottom). The performance and accuracy variables modulated smaller numbers of neurons (Figure S6B and S6C top), and the reward outcome of the previous trial did not co-modulate any of the trajectory-specific DA and GABA neurons (Figure S6C bottom). The trial variable modulated 30 DA neurons and 32 GABA neurons, co-modulating with the trajectory variable of 8 DA and 15 GABA neurons. However, the distribution of the trial predictor coefficient did not differ from a distribution with a mean equal to zero (one-sample t-test, P > 0.05, Figure S6D), indicating that the effect of successive trials on firing rate did not reflect cognitive processing, but was caused by mechanical reasons; due to the animal’s movements, the distance of the recording channel from the targeted neurons changed continuously, which affected the signal-to-noise ratio and eventually spike detection. In agreement with Engelhard et al., a notable proportion of DA neurons (36%) and GABA neurons (74%) were co-modulated by more than one behavioral variable Figure S6E (Engelhard et al., 2019).

Overall, the regression analysis confirmed the results of the permutation analysis regarding the significant effect of trajectory on midbrain neuronal activity during memory-dependent decisions. In addition, it demonstrated that the remaining independent variables included in our model cannot fully explain trajectory-specific firing activities in the delay period of the memory task.

Supplementary Figures

Figure S1.

Figure S2.

Figure S3.

Figure S4.

Figure S5.

Figure S6.

Figure S7.

Figure S8.

Supplementary Table

Table S1.

SI References

All references have been already cited in the main manuscript