1. Neuroscience
Download icon

Dopamine neurons drive fear extinction learning by signaling the omission of expected aversive outcomes

  1. Ximena I Salinas-Hernández
  2. Pascal Vogel
  3. Sebastian Betz
  4. Raffael Kalisch
  5. Torfi Sigurdsson
  6. Sevil Duvarci  Is a corresponding author
  1. Neuroscience Center, Goethe University, Germany
  2. University Medical Center of the Johannes Gutenberg University, Germany
  3. Focus Program Translational Neuroscience, Johannes Gutenberg University, Germany
Research Article
  • Cited 7
  • Views 3,120
  • Annotations
Cite this article as: eLife 2018;7:e38818 doi: 10.7554/eLife.38818

Abstract

Extinction of fear responses is critical for adaptive behavior and deficits in this form of safety learning are hallmark of anxiety disorders. However, the neuronal mechanisms that initiate extinction learning are largely unknown. Here we show, using single-unit electrophysiology and cell-type specific fiber photometry, that dopamine neurons in the ventral tegmental area (VTA) are activated by the omission of the aversive unconditioned stimulus (US) during fear extinction. This dopamine signal occurred specifically during the beginning of extinction when the US omission is unexpected, and correlated strongly with extinction learning. Furthermore, temporally-specific optogenetic inhibition or excitation of dopamine neurons at the time of the US omission revealed that this dopamine signal is both necessary for, and sufficient to accelerate, normal fear extinction learning. These results identify a prediction error-like neuronal signal that is necessary to initiate fear extinction and reveal a crucial role of DA neurons in this form of safety learning.

https://doi.org/10.7554/eLife.38818.001

eLife digest

To survive, animals must identify and react to stimuli in their environment that signal danger. But they must also adapt their behavior when those stimuli no longer signal danger – hiding whenever you hear a loud noise might keep you safe, but it also prevents you from searching for food. In the laboratory, we can study this form of learning using procedures called fear conditioning and extinction.

During fear conditioning, animals learn that a stimulus, such as a tone, signals that an unpleasant event is about to occur. That event might involve receiving a mild shock to the foot, for example. After experiencing the tone and shock paired together multiple times, animals will initially show signs of fear – such as freezing – when they hear the tone. But if later the tone occurs without being followed by the shock, these fear responses fade. This fading process is called extinction. Extinction does not involve erasing the old memory about the tone-shock relationship. That is, it is not a form of forgetting. Instead, the animals learn that the tone no longer signals an impending shock.

By monitoring brain activity in mice trained to associate a shock with a tone, Salinas-Hernández et al. reveal how the brain begins to learn that the shock no longer follows the tone. When the mice do not receive the anticipated shock to the foot, a group of brain cells that produce the chemical dopamine increase their activity. These neurons also fire whenever animals receive a reward, particularly one that exceeds their expectations. The more the dopamine neurons fire, the faster the mice reduce their fear responses to the tone. Preventing the neurons from increasing their activity prevents the mice from extinguishing their fear memory. By contrast, activating the neurons speeds up the extinction process.

Understanding how the brain extinguishes learned fear responses has therapeutic implications. Many anxiety disorders, such as post-traumatic stress disorder, involve impaired fear extinction learning. Indeed, exposure therapy – used to treat anxiety disorders such as phobias – is a form of fear extinction. Manipulating the activity of dopamine neurons during extinction could therefore help to treat anxiety disorders.

https://doi.org/10.7554/eLife.38818.002

Introduction

The ability to learn which stimuli predict danger is crucial for survival but it is equally important to adapt behavior when those stimuli no longer represent a threat. One classic example of this is fear extinction learning, during which the repeated presentation of a stimulus (conditioned stimulus, CS) that no longer predicts an aversive outcome (unconditioned stimulus, US) leads to a gradual decrease in learned fear responses. Many anxiety disorders, such as post-traumatic stress disorder, are characterized by impaired extinction learning (Craske et al., 2017; Graham and Milad, 2011; Mahan and Ressler, 2012; Milad and Quirk, 2012; Pitman et al., 2012) and thus understanding the neural basis of fear extinction has clinical significance.

A large body of evidence indicates that fear extinction represents new learning rather than forgetting or the erasure of the original fear memory (Bouton et al., 2006; Myers and Davis, 2007). In order to initiate extinction learning, the absence of the expected aversive outcome must be detected and signaled to the brain regions mediating fear extinction. Decades of research on fear extinction has revealed that a distributed network of brain structures including the amygdala, medial prefrontal cortex and hippocampus mediates the acquisition, consolidation and retrieval of fear extinction memories (Duvarci and Pare, 2014; Maren et al., 2013; Pape and Pare, 2010; Sotres-Bayon and Quirk, 2010; Tovote et al., 2015). However, none of these structures have been shown to signal the absence of the expected aversive outcome during fear extinction. The neural substrates of such a signal that could initiate extinction learning have therefore remained elusive.

New learning is initiated when outcomes violate expectations (Rescorla and Wagner, 1972). Such violations are thought to cause ‘prediction error’ signals that initiate the neural processes which ultimately lead to changes in behavior (Friston, 2012; den Ouden et al., 2012). During fear extinction, the absence of the US is an unexpected event and likely generates a prediction error signal that initiates extinction learning. More specifically, the omission of the aversive US can be conceptualized as a better-than-expected outcome. It is well-established that the activity of midbrain dopamine (DA) neurons represents the degree to which outcomes are better or worse than expected (Bayer and Glimcher, 2005; Eshel et al., 2015; Eshel et al., 2016; Schultz et al., 1997; Schultz and Dickinson, 2000). For example, many DA neurons increase their firing to rewards that are either unexpected or better than expected and this DA signal is sufficient to drive reinforcement learning (Steinberg et al., 2013). Based on our data from human studies, we have previously proposed that DA neurons could provide a prediction error-like signal during the aversive US omission to initiate fear extinction (Raczka et al., 2011). Consistent with this, an increase in DA release has been observed in the nucleus accumbens (NAc) during fear extinction (Badrinarayan et al., 2012) and pharmacological blockade of DA receptors in the NAc impairs fear extinction (Holtzman-Assif et al., 2010). However, the electrical activity of DA neurons during fear extinction − particularly at the time of the aversive US omission − and its relationship to extinction learning, is incompletely understood. In this study, we hypothesized that the unexpected omission of the aversive US activates DA neurons in the ventral tegmental area (VTA) and that this signal is necessary to initiate normal fear extinction learning. To test this hypothesis, we used in vivo single-unit recordings, DA neuron-specific calcium recordings and bi-directional optogenetic manipulations in behaving mice to examine and causally test the role of VTA DA neurons in fear extinction.

Results

Dopamine neurons signal the unexpected omission of the aversive US during fear extinction learning

We first examined whether DA neurons in the VTA are activated by the unexpected omission of the aversive US during fear extinction. We recorded the single-unit spiking activity of VTA neurons (Figure 1A–C;Figure 1—figure supplement 1) in mice (n = 11) that were trained in a fear conditioning paradigm (Figure 1D–F;Figure 1—figure supplement 2A) where a tone (CS) was paired with an aversive foot shock (US). Twenty-four hours after fear conditioning, mice received an extinction session consisting of CS presentations in the absence of the aversive US. A total of 43 (out of 90) and 40 (out of 75) VTA neurons classified as ‘putative’ DA neurons (see Materials and methods; Figure 1—figure supplement 3) were recorded during day 1 and day 2, respectively. Analysis of neuronal firing rates during the time of the US omission revealed that 25% of putative DA neurons (10 of 40) exhibited a significant increase in firing rate to the omission of the aversive US during the early extinction trials (E-Ext: average of first 10 CSs) when the US omission was unexpected (Figure 1G-I;Figure 1—figure supplement 3B; see Materials and methods for details). This was not simply a response to the CS offset since only 2.3% of neurons (1 of 43; Figure 1H;Figure 1—figure supplement 3A) showed increased firing at the end of the CS during tone habituation (Hab). On the other hand, during late extinction trials (L-Ext: average of last 10 CSs), when the US omission was no longer unexpected and animals showed significant extinction of fear responses (Figure 1F), only 7.5% of putative DA neurons (3 of 40; Figure 1G-I) showed an increase in firing to the absence of the US. Importantly, there was no difference in freezing levels during the CS and the post-CS period in E-Ext (Figure 1—figure supplement 2B) suggesting that the observed increase in DA neuron firing was not due to an increase in movement when the CS terminates.

Figure 1 with 4 supplements see all
Putative dopamine neurons signal the unexpected omission of the aversive US during fear extinction learning.

(A) Schematic of single-unit recordings in the VTA. (B) Histological example showing an electrode placement in the VTA (white arrow). Red: immunostaining against tyrosine hydroxylase (TH), blue: DAPI staining. Scale bar: 0.5 mm. (C) Schematic coronal sections showing the location of the recording sites in the VTA. Numbers represent distance posterior to the bregma. (D) Schematic of the behavioral protocol. Hab: tone habituation, Fear Cond.: fear conditioning. (E) Schematic of CS and US presentations during tone habituation (Hab), fear conditioning (FC) and extinction (Ext). (F) Behavioral data. During tone habituation, mice (n = 11) showed low freezing levels in response to the CS. Twenty-four hours after fear conditioning, mice exhibited significant increase in freezing to the CS during E-Ext trials (first 10 CSs, paired t-test comparing E-Ext to Hab, t(10) = 8.30, p<0.0001). The CS evoked low freezing levels during L-Ext trials (last 10 CSs) indicating successful extinction learning (paired t-test: comparing L-Ext to Hab, t(10) = 1.32, p = 0.21; comparing L-Ext to E-Ext, t(10) = 11.9, p<0.0001). (***p<0.0001). (G) Raster plot (top) and peristimulus time histogram (1 s bins; bottom) of an example putative DA neuron (see inset; scale bar: 50 μV) responding to US omission during the extinction session. (H) The proportion of putative DA neurons that were significantly US omission (or CS offset for Hab) excited, inhibited or nonresponsive during the Hab, E-Ext and L-Ext trials. Proportion of US omission excited neurons significantly increased from Hab to E-Ext (Fisher’s exact test, p = 0.0028) and decreased back to Hab levels during L-Ext (Fisher’s exact test, p = 0.34). Note that there were no US omission inhibited neurons during E-Ext and L-Ext. (I) Peristimulus time histogram showing the z-scored population activity of all putative DA neurons that were significantly US omission excited during E-Ext (n = 10 of 40 putative DA neurons). (J) Peristimulus time histogram showing the z-scored population activity of all putative DA neurons during Hab (left), E-Ext (middle) and L-Ext (right). Note the significant increase in population activity at the time of the US omission during E-Ext (*p<0.05). Shaded regions represent mean ± s.e.m. across neurons.

https://doi.org/10.7554/eLife.38818.003

Furthermore, analysis of the distribution of z-scores during the US omission (Figure 2A) revealed that none of the putative DA neurons (0 of 40) showed a selective decrease in firing to the US omission (see Materials and methods for details) during either E-Ext or L-Ext. This suggests that the dominant response of putative DA neurons to US omission was excitatory. Consistent with this, we observed a significant increase in firing to the US omission at the population level when we examined the average response of all putative DA neurons during E-Ext (paired t-test, t(39) = 2.22, p = 0.03), but not L-Ext (paired t-test, t(39) = 0.20, p = 0.83) or Hab (paired t-test, t(42) = 0.71, p = 0.47; Figure 1J).

Distribution of z-scores during the US omission for putative DA and non-DA neurons.

(A) Top: plot of z-score (CS offset bin) against the ∆ z-score (CS offset bin − CS last bin) during tone habituation (Hab.; left) and z-score (US omission bin) against the ∆ z-score (US omission bin − CS last bin) during early extinction (E-Ext; right). Each dot represents a putative DA neuron. The red shaded areas (x-axis: 2, y-axis: 2) contain the significantly excited (red dots) and the blue shaded areas (x-axis: −2, y-axis: −2) contain significantly inhibited (blue dots) neurons. Black dots represent non-responsive neurons. Bottom: peristimulus time histogram (1 s bins) of an example neuron (see inset; scale bar: 50 μV) that showed significant inhibition to CS offset during Hab (left) and of an example neuron (see inset; scale bar: 50 μV) that showed significant excitation to the US omission during E-Ext (right). (B) The same as in (A), but for putative non-DA neurons.

https://doi.org/10.7554/eLife.38818.008

Local circuit interactions between DA and GABA (γ-aminobutyric acid) neurons in the VTA underlie reinforcement learning (Cohen et al., 2012; Eshel et al., 2015). It is therefore possible that the DA and local GABA neurons also interact to drive fear extinction learning. For instance, the increased firing of putative DA neurons could be mediated by disinhibition resulting from inhibition of local GABA neurons at the time of the US omission. To test this possibility, we analyzed the activity of putative non-DA neurons – the subset of VTA neurons which likely dominantly includes GABA cells – during the time of the US omission and found that 8.5% of neurons (3 of 35) showed a selective decrease in firing to the US omission during E-Ext (Figure 1—figure supplement 3B, Figure 2B). However, this was not significantly different from the proportion of non-DA neurons (4.2%, 2 of 47 neurons) showing decreased firing at the CS offset during Hab (Figure 1—figure supplement 3B, Figure 2B; Fisher’s exact test, p = 0.64). These results therefore suggest that the excitation observed in the putative DA neurons during US omission is unlikely mediated by the activity of the local GABA neurons. Moreover, only a small proportion of putative non-DA neurons showed increased firing at the time of the US omission during E-Ext (2.8%, 1 of 35 neurons; Figure 1—figure supplement 3B, Figure 2B). This proportion was again not different from the 2.1% of neurons (1 of 47) that showed increased firing during Hab (Figure 1—figure supplement 3B, Figure 2B; Fisher’s exact test, p = 1). Together, these results suggest that the putative non-DA, likely local GABA, neurons in VTA do not change their firing at the time of the US omission and that the US omission is signaled specifically by the putative DA neurons in VTA.

In contrast to the uniform response observed during US omission, the responses of putative DA neurons during the CS were diverse. Figure 1—figure supplement 4 shows example neurons that displayed excitation, inhibition and also biphasic response during the CS. Whereas some neurons showed sustained responses to the CS (Figure 1—figure supplement 4B), others showed a transient response at the onset of the CS (Figure 1—figure supplement 4C). In order to quantify these CS-evoked responses, we calculated the mean response to the CS for each neuron during Hab, E-Ext and L-Ext (Figure 1—figure supplement 4A). We found that 5% (2 of 40) of putative DA neurons exhibited CS-evoked excitation and 7.5% (3 of 40) CS-evoked inhibition during E-Ext. However these percentages were not significantly different from the proportion of cells that showed excitation (6.9%, 3 of 43 neurons; Fisher’s exact test, p = 1) or inhibition (0%, 0 of 43; Fisher’s exact test, p = 0.1) during Hab. Furthermore, when we examined the population activity by averaging the response of all putative DA neurons, there was no significant change in the average response during the CS (paired t-tests, Hab: t(42) = 0.58, p = 0.56, E-Ext: t(39) = 1.89, p = 0.065, L-Ext: t(39) = 1.23, p = 0.22; Figure 1J;Figure 1—figure supplement 4A). These results suggest that the average activity of putative DA neurons in the VTA did not change during the CS even though different subsets of cells showed CS-evoked excitation and inhibition. A lack of strong responses to the CS might be due to the cued fear conditioning paradigm that we used in our study. It has recently been shown that DA neurons respond strongly to the CS when a discriminative fear conditioning task is used and the strength of their response increases with increasing discrimination between the aversive and the safe CS (Jo et al., 2018).

The above results show that putative DA, but not non-DA, neurons in the VTA signal the omission of the aversive US during fear extinction, specifically during the beginning of extinction learning when the US omission is unexpected. To further confirm that DA neurons signal the unexpected omission of the US, we next measured activity-dependent calcium signals selectively in DA neurons using fiber photometry. To this end, a Cre-dependent adeno-associated virus (AAV) expressing the genetically encoded calcium (Ca+2) indicator gCaMP6 was injected, and an optical fiber implanted, in the VTA of transgenic mice expressing Cre recombinase under the control of the dopamine transporter (Dat) promoter (DAT-Cre mice; Figure 3A–B and Figure 3—figure supplement 1A). In these mice, Cre expression is highly selective for dopamine neurons (Lammel et al., 2015). Accordingly, we observed a high degree of overlap between Cre-dependent gCaMP6 expression and immunohistochemical staining against tyrosine hydroxylase (TH; Figure 3—figure supplement 1B–C). In control mice, we injected a Cre-dependent AAV expressing GFP to examine whether changes in fluorescence could be independent of neuronal activity. Recordings from gCaMP6-expressing animals revealed transient increases in fluorescence whereas such increases were absent in mice expressing GFP (Figure 3C). Furthermore, consistent with electrophysiological studies in VTA DA neurons (Eshel et al., 2015; Eshel et al., 2016; Roesch et al., 2007), we also confirmed that reward delivery caused large increases in fluorescence in the gCaMP6-expressing mice (Figure 3—figure supplement 2, also see Materials and methods), indicating that this Ca+2 signal is indeed generated by DA neuron activity.

Figure 3 with 3 supplements see all
Calcium recordings in VTA dopamine neurons confirm signaling of the unexpected US omission during fear extinction learning.

(A) Schematic of the surgical procedure showing the virus injection (left) and optical fiber implantation (right) in the VTA. (B) Top: example histological image showing Cre-dependent expression of gCaMP6 (green) along with immunostaining for tyrosine hydroxylase (TH, red) and DAPI (blue) staining in the VTA. White vertical track indicates the optical fiber placement in the VTA. Scale bar: 0.5 mm. Bottom: confocal images showing expression of gCaMP6 (left), TH (middle) and a merged image (right) showing co-expression. Scale bar: 20 μm. (C) Examples of changes in fluorescence (dF/F) over time in an animal expressing gCaMP6 (green) and an animal expressing the control fluorophore GFP (gray). Scale bar: 5 s, 0.2 dF/F. (D) Schematic of the behavioral protocol. Hab: tone habituation, Fear Cond.: fear conditioning. (E) Schematic of CS and US presentations during tone habituation (Hab), fear conditioning (FC) and extinction (Ext). (F) Behavioral freezing to the CS during tone habituation and extinction session for gCaMP6 (n = 10) and GFP (n = 4) groups. During Hab, all mice showed low freezing levels in response to the CS. Twenty-four hours after fear conditioning, both groups of mice increased freezing to the CS during E-Ext (first 10 CSs). The CS evoked low freezing levels during L-Ext (last 10 CSs) indicating successful extinction learning. (G) Top: Average change in fluorescence in animals expressing gCaMP6 (green, n = 10) or GFP (gray, n = 4) around the time of CS presentation (gray area) during Hab, E-Ext and L-Ext. Note the increase in fluorescence at the offset of the CS (the time of the unexpected US omission) during E-Ext. Bottom: Average change in fluorescence in the 5 s after CS offset during Hab, E-Ext and L-Ext. gCaMP6-expressing animals exhibited a significant increase in Ca+2 signal during E-Ext compared to the GFP control group (**p<0.01, rank-sum test). (H) Correlation between the average change in fluorescence in the 5 s after CS offset during E-Ext and the change in percent freezing from E-Ext to L-Ext (n = 10 mice; Spearman’s correlation = 0.83, p = 0.0037). (I) Correlation between the average change in fluorescence during the 0–3 s after reward delivery and change in percent freezing from E-Ext to L-Ext (n = 10 mice; Spearman’s correlation = 0.32, p = 0.36). Shaded regions and error bars represent mean ±s.e.m across animals.

https://doi.org/10.7554/eLife.38818.009

Both gCaMP6- and GFP-expressing animals underwent the same fear conditioning protocol as in the electrophysiology experiment (Figure 3D–E) and showed comparable levels of freezing to the CS across sessions (two-way repeated measures ANOVA; main effect of group: F1,24 = 0.04, p = 0.83; group × trial interaction: F2,24 = 0.04, p = 0.96; Figure 3F and Figure 3—figure supplement 3A). During E-Ext, we observed a significant increase in the Ca+2 signal of gCaMP6 animals at the time of the US omission compared to the pre-CS baseline (p<0.01, sign-rank test) and the GFP control group (p<0.01, rank-sum test; Figure 3G). On the other hand, during L-Ext, when the US omission was no longer unexpected, the Ca+2 signal did not change during the post-CS period (p = 0.85, sign-rank test; Figure 3G). No changes in fluorescence were observed in GFP-expressing animals during either E-Ext or L-Ext (Figure 3G). Furthermore, we did not observe any change in the Ca+2 signal during Hab (gCaMP6 group, p = 0.32, sign-rank test; Figure 3G) suggesting that the increase during US omission is unlikely a response to the CS offset. Furthermore, freezing levels during the CS and the post-CS period in E-Ext were comparable (Figure 3—figure supplement 3B) ruling out the possibility that an increase in movement when CS terminates might have resulted in the observed increase in Ca+2 signal at the time of the US omission during E-Ext. Notably, these results obtained by measuring the population Ca+2 signal from DA neurons are consistent with the results of the electrophysiology recordings where we found a significant increase in the average population activity of all putative DA neurons at the time of the US omission during E-Ext (Figure 1J).

If activation of DA neurons at the time of the unexpected US omission drives extinction learning then this Ca+2 signal during E-Ext should be larger in animals that exhibit better extinction learning. To test this, we took advantage of the variability in the extinction learning rates of individual mice and asked whether they were correlated with the Ca+2 signal at the time of the US omission during E-Ext. This revealed a significant correlation between the Ca+2 signal during E-Ext and the change in freezing from E-Ext to L-Ext (Spearman’s correlation = = 0.83, p = 0.0037; Figure 3H). A significant correlation was also observed between the Ca+2 signal during E-Ext and the freezing levels during L-Ext (Spearman’s correlation = 0.92, p = 0.0003). However, it is possible that the variation in the Ca+2 signal during E-Ext might be due to the variation in recording locations in the VTA across animals rather than reflecting the relationship with extinction learning. We reasoned that differences in recording locations would likely result in variation in reward responses. We therefore examined the correlation between the change in freezing from E-Ext to L-Ext and reward responses and did not find a significant relationship between these two variables (Spearman’s correlation = 0.32, p = 0.36; Figure 3I). These results therefore suggest that the magnitude of the Ca+2 signal during E-Ext correlated with the level of extinction learning and the variations in the magnitude of this signal were not due to differences in the recording location.

Contrasting with the uniform increase in DA neuron activity during US omission (Figure 3G), the responses to the CS varied across animals (Figure 4A). Some animals showed increased and some decreased fluorescence during the CS in E-Ext (Figure 4A–B). Accordingly, consistent with single unit results which showed no change in CS-evoked population activity of putative DA neurons (Figure 1JFigure 1—figure supplement 4A), the Ca+2 signal evoked by the CS was not significantly different from the baseline during E-Ext (p = 1.0, sign-rank test) or Hab (p = 0.43, sign-rank test) when we averaged the CS responses of all animals (Figure 4A).

Changes in the calcium signal during CS presentations.

(A) Average change in fluorescence (dF/F) during the CS (10 s) during tone habituation (Hab), early extinction (E-Ext) and late extinction (L-Ext). There was no difference in responses to the CS between the gCaMP6-expressing (n = 10) and GFP-expressing (n = 4) animals (rank-sum test: Hab, p = 1.0; E-Ext, p = 0.37; L-Ext, p = 0.11). (B) Example CS responses in gCaMP6-expressing animals that showed decreased (top), unchanged (middle) and increased fluorescence during the CS (gray area). Left: histology image showing the location of the optical fiber in VTA. Green: cre-dependent gCaMP6 expression, red: immunostaining for tyrosine hydroxylase (TH), blue: DAPI staining. White vertical tracks indicate the optical fiber placement in the VTA. Right: Changes in fluorescence during Hab, E-Ext, L-Ext and Reward.

https://doi.org/10.7554/eLife.38818.013

Overall, these findings together with our single-unit results demonstrate that DA neurons signal the unexpected omission of the aversive US during fear extinction and that the magnitude of this DA signal predicts the strength of extinction learning.

Inhibition of dopamine neuron firing at the time of the US omission impairs fear extinction learning

We next asked whether the observed increase in DA neuron firing at the time of the unexpected US omission is necessary for fear extinction learning. To address this question, we optogenetically inhibited DA neurons in the VTA at the time of the US omission during fear extinction. DAT-cre mice received bilateral injections of a Cre-dependent AAV expressing either the light-activated inhibitory opsin halorhodopsin fused with enhanced yellow fluorescent protein (NpHR-eYFP) or eYFP only (eYFP control) into the VTA, as well as bilateral implantation of optical fibers above the VTA to allow for selective inhibition of VTA DA neurons (Figure 5A–C and Figure 5—figure supplement 1A). We observed a high degree of overlap between Cre-dependent NpHR-eYFP expression and immunohistochemical staining against TH (Figure 5—figure supplement 1B–C) suggesting DA neuron-selective expression. Furthermore, we confirmed that optical stimulation of NpHR inhibits DA neuron firing in awake DAT-cre mice (Figure 5—figure supplement 2).

Figure 5 with 2 supplements see all
Inhibition of dopamine neuron firing at the time of the US omission impairs fear extinction learning.

(A) Schematic of the surgical procedure showing bilateral virus injection (left) and optical fiber implantation (right) in the VTA. (B) Example histological image showing Cre-dependent expression of NpHR-eYFP (green) along with immunostaining for tyrosine hydroxylase TH (red) and DAPI (blue) staining in the VTA. White vertical tracks indicate the bilateral optical fiber placements in the VTA. Scale bar: 0.5 mm. (C) Confocal images showing expression of NpHR-eYFP (left), TH (middle) and merged image (right) showing co-expression. Scale bar: 20 μm. (D) Schematic of the behavioral protocol. Fear Cond.: fear conditioning, Ext Recall: extinction recall. (E) Schematic of paired optogenetic inhibition of DA neurons at the time of the US omission. (F) Schematic of unpaired optogenetic inhibition during intertrial intervals. (G) Percent freezing to the CS during fear conditioning (FC), extinction and extinction recall sessions. The Paired-NpHR group showed impaired extinction learning and extinction recall. (**p<0.01, *p<0.05). (H) No difference in freezing to the CS between groups at the start of extinction (first CS). Ext: extinction. (I) Freezing levels during E-Ext (average of first 10 CSs) and L-Ext (average of last 10 CSs; two-way repeated measures ANOVA, main effect of group: F2,19 = 9.05, p = 0.0017; group × trial interaction: F2,19 = 7.38, p = 0.0043). The Paired-NpHR group (n = 7) showed significantly higher freezing to the CS compared to the Paired-eYFP (n = 7) and Unpaired-NpHR (n = 8) groups during L-Ext trials (***p<0.001). E-Ext: early extinction, L-Ext: late extinction. (J) Freezing levels during E-Ext Rec (average of first 10 CSs) and L-Ext Rec trials (average of last 10 CSs; two-way repeated measures ANOVA, main effect of group: F2,19 = 7.21, p = 0.0047). The Paired-NpHR group exhibited significantly higher freezing to the CS compared to the control groups during E-Ext Rec (***p<0.001). E-Ext Rec: early extinction recall, L-Ext Rec: late extinction recall. Error bars represent mean ± s.e.m. across animals.

https://doi.org/10.7554/eLife.38818.014

Mice were trained in a fear conditioning protocol (Figure 5D) consisting of 4 CS-US pairings on day 1. Twenty-four hours after fear conditioning, mice received an extinction session. In the experimental group expressing NpHR-eYFP light was delivered bilaterally to the VTA to inhibit DA neurons specifically at the end of each CS presentation, that is during the time of the US omission (Paired-NpHR, n = 7; Figure 5E). The behavior of the experimental group was compared to two control groups: one group consisted of mice expressing eYFP only which received the identical light delivery (Paired-eYFP, n = 7) and a second group consisted of mice expressing NpHR-eYFP that received light delivery to inhibit DA neurons during the intertrial intervals (ITIs; Unpaired-NpHR, n = 8; Figure 5F).

Compared to the two control groups, the Paired-NpHR group exhibited high freezing levels to the CS throughout the extinction session, suggesting impaired extinction learning (Figure 5G). A two-way repeated measures ANOVA revealed a significant main effect of group (F2,456 = 7.03, p = 0.0052) and a significant interaction between group and trial (F48,456 = 2.19, p<0.0001). Consistent with this, there was a significant difference between the Paired-NpHR group and the Paired-eYFP (p<0.001) or Unpaired-NpHR (p<0.001) controls during L-Ext (Figure 5I). Furthermore, impaired extinction learning resulted in a weaker memory for extinction when tested the next day (extinction recall test; two-way repeated measures ANOVA; main effect of group: F2, 456 = 7.1, p = 0.005; group × trial interaction: F48, 456 = 1.49, p = 0.02; Figure 5G). Consistently, during the early extinction recall trials (E-Ext Rec: first 10 CSs) the Paired-NpHR group froze significantly more compared to the Paired-eYFP (p<0.001) or the Unpaired-NpHR (p<0.001) controls (Figure 5J). In contrast to the Paired-NpHR group, the Unpaired-NpHR group behaved comparable to the Paired-eYFP control group during both extinction and extinction recall tests (Figure 5G–J) suggesting that optical inhibition of DA neurons per se did not result in a nonspecific increase in freezing levels, and that the behavioral effect was dependent on the temporally specific inhibition of DA neurons during the time of the US omission. Furthermore, there was no difference between the groups in their fear acquisition on day 1 (two-way repeated measures ANOVA, no main effect of group, F2,57 = 0.3, p = 0.74 or no group × trial interaction, F6,57 = 0.94, p = 0.47; Figure 5G) and all groups showed comparable levels of freezing at the start of extinction (first CS; one-way ANOVA, F2,19 = 2.06, p = 0.15; Figure 5H) before any experimental manipulation took place, ruling out the possibility that differences in the strength of fear memory on day 2 between groups might have caused the observed effect.

Importantly, freezing levels of the NpHR group at the beginning of extinction recall were comparable to freezing levels at the beginning of extinction (paired t-test comparing first CS of extinction and first CS of extinction recall, t(6) = 0.56, p = 0.59), suggesting that no significant extinction learning happened in these animals. Furthermore, we found that the extinction rate of the Paired-NpHR group during extinction recall, in the absence of optogenetic inhibition, was comparable to the extinction rate of the Paired-eYFP and Unpaired-NpHR groups during extinction (two-way repeated measures ANOVA; no main effect of group F2, 456 = 1.02, p = 0.37 and group × trial interaction: F48, 456 = 0.86, p = 0.73). This suggests that our manipulation did not have a nonspecific long-term effect on the ability of the Paired-NpHR group to exhibit extinction learning. These results also suggest that our optogenetic manipulation likely did not affect the strength of the fear memory. Taken together, these findings demonstrate that DA neuron activation by the unexpected omission of the US is necessary for fear extinction learning.

Enhancing dopamine neuron firing at the time of the US omission accelerates fear extinction learning

If DA neuron firing at the time of the unexpected US omission drives fear extinction, then enhancing this DA signal should accelerate fear extinction learning. To test this, we optogenetically excited DA neurons precisely at the time of the US omission during fear extinction learning. DAT-cre mice were bilaterally injected with a Cre-dependent AAV expressing either channelrhodopsin-2 (ChR2) fused with eYFP (ChR2-eYFP) or eYFP only (eYFP control) into the VTA, and implanted bilaterally with optical fibers above VTA (Figure 6A–C; Figure 6—figure supplement 1A). There was again a high level of overlap between Cre-dependent ChR2-eYFP expression and immunohistochemical staining against TH (Figure 6—figure supplement 1B–C) suggesting DA neuron-selective expression of ChR2. Furthermore, we confirmed that optical stimulation of ChR2 induces firing of DA neurons in awake DAT-cre mice (Figure 6—figure supplement 2).

Figure 6 with 3 supplements see all
Enhancing dopamine neuron firing at the time of the US omission accelerates fear extinction learning.

(A) Schematic of the surgical procedure showing the bilateral virus injection (left) and optical fiber implantation (right) in the VTA. (B) Example histology showing Cre-dependent expression of ChR2-eYFP (green) along with immunostaining for tyrosine hydroxylase (TH, red) and DAPI (blue) staining in the VTA. White vertical tracks indicate the bilateral optical fiber placements in the VTA. Scale bar: 0.5 mm. (C) Confocal images showing expression of ChR2-eYFP (left), TH (middle) and merged image (right) showing co-expression. Scale bar: 20 μm. (D) Schematic of the behavioral protocol. Fear Cond.: fear conditioning, Ext Recall: extinction recall. (E) Schematic of the paired optogenetic excitation of DA neurons at the time of the US omission. (F) Schematic of the unpaired optogenetic excitation during intertrial intervals. (G) Percent freezing to the CS during fear conditioning (FC), extinction and extinction recall sessions. The Paired-ChR2 (P-ChR2) group showed accelerated extinction learning and better extinction recall. (***p<0.001, **p<0.01, *p<0.05). (H) No difference in freezing to the CS between groups at the start of extinction (first CS). Ext: extinction. (I) Freezing levels during E-Ext (average of first 10 CSs) and L-Ext (average of last 10 CSs; two-way repeated measures ANOVA, significant group effect: F2,18 = 3.88, p = 0.03). The P-ChR2 group (n = 7) exhibited significantly lower freezing compared to the Paired-eYFP (P-eYFP; n = 7) and Unpaired-ChR2 (U-ChR2; n = 7) control groups during E-Ext (*p<0.05). E-Ext: early extinction, L-Ext: late extinction. (J) Freezing levels during E-Ext Rec (average of first 5 CSs) and L-Ext Rec (average of last 5 CSs; two-way repeated measures ANOVA, main effect of group: F2,18 = 5.5, p = 0.01 and group × trial interaction: F2,18 = 4.81, p = 0.021). The P-ChR2 group exhibited significantly lower freezing to the CS compared to control groups during E-Ext Rec (**p<0.01). E-Ext Rec: early extinction recall, L-Ext Rec: late extinction recall. Data are presented as means and error bars represent s.e.m.

https://doi.org/10.7554/eLife.38818.017

Mice were trained in a fear conditioning protocol (Figure 6D) similar to the optogenetic inhibition experiment. The experimental group consisted of ChR2-eYFP expressing mice which received light stimulation of DA neurons specifically at the time of the US omission (Paired-ChR2, n = 7; Figure 6E). Two control groups, one expressing eYFP only that received the identical light delivery (Paired-eYFP, n = 7) and the other expressing ChR2-eYFP that received light excitation during the ITIs (Unpaired-ChR2, n = 7; Figure 6F), were used to control for nonspecific effects of light and DA neuron stimulation, respectively.

As expected, all groups showed a gradual decrease in freezing to the CS during the extinction session. However, in the Paired-ChR2 group, freezing decreased faster than in the control groups suggesting accelerated extinction learning (Figure 6G). A two-way repeated measures ANOVA comparing freezing levels confirmed this observation by revealing a significant main effect of group (F2,432 = 4.1, p = 0.03). Comparison of freezing levels in the three groups, particularly during E-Ext, revealed a significant difference between the Paired-ChR2 group and the Paired-eYFP (p<0.05) or Unpaired-ChR2 (p<0.05) controls (Figure 6I) suggesting accelerated extinction learning. On the other hand, the two control groups behaved comparably (p>0.05). These results suggest that optical excitation of DA neurons per se did not result in a nonspecific decrease in freezing levels and that the temporally precise excitation during the US omission is necessary for the observed behavioral effect. Furthermore, there was no difference between the groups in their fear acquisition on day 1 (two-way repeated measures ANOVA, no main effect of group, F2,54 = 0.17, p = 0.84 and no group × trial interaction, F6,54 = 0.65, p = 0.68, Figure 6G) and all groups showed comparable levels of freezing during the first CS of extinction (one-way ANOVA, F2,18 = 0.81, p = 0.66, Figure 6H) before light stimulation began. Thus, excitation of DA neurons precisely at the time of the unexpected US omission is sufficient to accelerate fear extinction learning. Finally, Paired-ChR2 mice spent less time freezing to the CS compared to the control groups during the extinction recall test (two-way repeated measures ANOVA; main effect of group: F2,162 = 5.5, p = 0.013 and group × trial interaction: F18,162 = 2.52, p = 0.0011, Figure 6G) suggesting that the accelerated extinction learning resulted in a stronger extinction memory.

An alternative possibility is that the low level of freezing in the Paired-ChR2 group during Ext Rec was due to an effect of optical stimulation of DA neurons on the fear memory. For instance, the optogenetic manipulation could result in the erasure of the fear memory by impairing the memory reconsolidation process (Nader, 2015) rather than accelerating extinction learning and strengthening extinction memory. To rule this possibility out, we tested the animals on a fear renewal test on day 4 by presenting 5 CSs in the conditioning context. It is well established that extinction learning is context-dependent such that if the animals are tested in a different context than the one they are extinguished in, fear responses return, a phenomenon called fear renewal (Bouton, 2004). Therefore, if low level of freezing during Ext Rec was due to impaired reconsolidation then we would expect to see impaired fear renewal in Paired-ChR2 group (Duvarci and Nader, 2004). However, we found that all groups showed high freezing to the CS during the renewal test and that there was no difference in the freezing levels between the groups (Figure 6—figure supplement 3; two-way repeated measures ANOVA, no significant effect of group, F2, 72 = 0.33, p = 0.72 and group × trial interaction, F8, 72 = 0.86, p = 0.55) suggesting that all three groups showed comparable levels of fear renewal. This suggests that the low level of freezing in Paired-ChR2 group during extinction recall was not due to an affect of our optogenetic manipulation on the fear memory but rather was due to enhanced extinction learning and memory formation. Taken together, these findings demonstrate that increasing DA neuron activity at the time of US omission — and thus enhancing an endogenous extinction mechanism — is sufficient to accelerate extinction learning and strengthen extinction memory.

Discussion

Here we demonstrated that DA neurons were activated by the omission of the aversive US during fear extinction, specifically during the beginning of extinction when the US omission is most unexpected. Importantly, the magnitude of this DA signal correlated with the strength of extinction learning. Furthermore, temporally specific optogenetic inhibition of DA neurons at the time of the US omission prevented extinction, demonstrating that this signal is necessary for normal fear extinction. Conversely, enhancing this DA signal using temporally-specific optogenetic excitation was sufficient to accelerate extinction learning. Together, these results identify a crucial role of DA neurons in signaling the unexpected omission of aversive outcomes and thereby driving fear extinction learning.

Previous studies have shown that DA neurons encode a reward prediction error, or the discrepancy between expected and actual rewards, which acts as a teaching signal for reinforcement learning (Bayer and Glimcher, 2005; Eshel et al., 2015; Eshel et al., 2016; Schultz et al., 1997; Steinberg et al., 2013). Specifically, presentation of unexpected or better than expected rewards induces increased firing in DA neurons (Bayer and Glimcher, 2005; Eshel et al., 2016; Roesch et al., 2007; Schultz et al., 1997). Our results suggest that DA neurons might also signal a better than expected outcome during fear extinction. We found that a subpopulation of putative DA neurons in the VTA increased their firing selectively at the time of the US omission during fear extinction. This US omission-responsive firing was observed specifically during the early trials of extinction when the absence of the US was unexpected, and was significantly reduced during the late stages of extinction when the US omission was no longer unexpected and animals showed significant extinction of fear responses. Importantly, these responses were not observed in putative non-DA neurons. This suggests that DA neurons encode a prediction error-like signal during fear extinction learning. Ca+2 recordings selectively in DA neurons further confirmed these results and revealed that this DA signal correlated with the strength of extinction learning. Recent studies have shown that DA neurons not only encode reward prediction errors but also signal prediction errors to gate fear learning (Groessl et al., 2018) and to drive threat avoidance (Menegas et al., 2018). Interestingly, omission of aversive stimuli during fear extinction in fruit flies is encoded by the DA system that mediates reward, but not aversive, learning (Felsenberg et al., 2018). Whether the DA signal during fear extinction in mammals is also similar to prediction error signals for reward and mediated by the brain’s reward circuitry (Wise, 2002) will be important questions for future studies.

Consistent with our results, previous studies using partial reinforcement paradigms have shown that DA neurons exhibit increased firing to the unexpected omission of aversive stimuli (Matsumoto and Hikosaka, 2009; Matsumoto et al., 2016; but see Tian and Uchida, 2015). However, responses to aversive US omission in these studies were much smaller compared to what we observed. There are several differences between these studies and ours that might account for this. The aversive US used in our study is a painful footshock whereas an air-puff was used in previous studies (Matsumoto and Hikosaka, 2009; Matsumoto et al., 2016). Furthermore, it has been shown that the valence of the testing context influences DA neuron responses to the omission of the aversive US. Matsumoto and colleagues (2016) found a significant increase in DA neuron responses to the omission of aversive airpuff in a low reward, but not high reward, context. In our study, the animals were only fear conditioned and no reward learning happened prior to or during fear conditioning. Therefore, the context was exclusively aversive. In addition, the CS-US contingency during fear conditioning was higher in our study than in partial reinforcement tasks used in previous studies. The CS predicted the shock with 100% probability at the end of fear conditioning and therefore, the omission of the US was fully unexpected at the beginning of fear extinction in our study. On the other hand, in the studies using partial reinforcement tasks the CS-US contingency was 25–90% and the omission of the US was performed intermittently (Matsumoto and Hikosaka, 2009; Matsumoto et al., 2016; Tian and Uchida, 2015). The US omission was therefore arguably less unexpected. Together, these factors might therefore account for the differences in DA neuron responses between these studies and ours. Overall, our findings suggest that DA neurons not only signal better than expected rewards (Bayer and Glimcher, 2005; Eshel et al., 2016; Roesch et al., 2007; Schultz et al., 1997) but also better than expected outcomes more generally, such as the omission of an aversive event.

Detecting the discrepancy between expected and actual outcomes is critical for new learning (Rescorla and Wagner, 1972). During fear extinction, the omission of the aversive US is an unexpected outcome which initiates new learning about the CS, specifically that it no longer predicts danger. However, how this learning is initiated at the neuronal level has remained unknown. Here, by using bidirectional optogenetic manipulations, we demonstrated that the DA signal during the omission of the aversive US drives normal fear extinction learning: inhibiting this DA signal prevented, while enhancing it accelerated, normal extinction learning. Our results are consistent with previous findings establishing the causal role of DA neurons in reinforcement learning (Chang et al., 2016; Steinberg et al., 2013; Tsai et al., 2009) and further extend their role to safety learning. Consistent with our findings, a recent study inhibited VTA DA neurons during US omission in rats and also found reduced fear extinction learning (Luo et al., 2018). Our study replicates this finding in mice and further extends it by showing that enhancing DA neuron activity at the time of the US omission is also sufficient to accelerate normal extinction learning. Consistent with our results, it has been shown that enhancement of DA signaling by L-DOPA administration during extinction was sufficient to initiate fear extinction learning in a mouse model of impaired extinction learning (Whittle et al., 2016). Our results therefore suggest that enhancement of DA signaling during extinction could be a potential strategy for the treatment of anxiety disorders.

Fear extinction is mediated by a network of brain structures consisting mainly of the amygdala and the infra-limbic (IL) subregion of the medial prefrontal cortex (Duvarci and Pare, 2014; Maren et al., 2013; Pape and Pare, 2010; Sotres-Bayon and Quirk, 2010; Tovote et al., 2015). Although neuronal activation has been observed in these structures during different stages of fear extinction, they occur during the CS and later than the DA signal we observed in our study. In the basolateral amygdala, a subpopulation of neurons termed 'extinction neurons' increases their firing to the CS during extinction learning (Amano et al., 2011; Herry et al., 2008). However, these neurons become CS responsive late in the extinction session right before the animals show a decrease in fear responses (Herry et al., 2008) suggesting that these neurons likely mediate inhibition of fear responses. In the IL, a structure necessary for consolidation of extinction memories (Sotres-Bayon and Quirk, 2010), increased firing to the CS is observed during extinction recall (Milad and Quirk, 2002). In contrast, the DA signal that we demonstrated in our study occurs at the early trials of the extinction session, supporting our conclusion that this signal initiates extinction learning.

Plasticity in the amygdala and IL underlie acquisition and consolidation of fear extinction memories (Duvarci and Pare, 2014; Maren et al., 2013; Pape and Pare, 2010; Sotres-Bayon and Quirk, 2010; Tovote et al., 2015). Furthermore, DA signaling in the amygdala and IL has been shown to play an important role in fear extinction (Abraham et al., 2014; Haaker et al., 2013; Hikind and Maroun, 2008; Mueller et al., 2010; Shi et al., 2017; Whittle et al., 2016). Interestingly, optogenetic inhibition of DA neurons during US omission has been found to prevent extinction-related plasticity in the amygdala and IL (Luo et al., 2018). However, it is unclear how this DA signal induces plasticity in the amygdala and IL to underlie extinction memory. This can be mediated through direct DA projections to these structures or indirectly through a multi-synaptic circuit mechanism. The first step in addressing this issue will be to identify the projection target of these DA neurons that signal the omission of the US during fear extinction.

Recent studies have shown that midbrain DA neurons form functionally distinct and mostly non-overlapping subpopulations based on their projection targets (Beier et al., 2015; Lammel et al., 2008; Lammel et al., 2011; Lerner et al., 2015; Lynd-Balta and Haber, 1994; Menegas et al., 2015; Menegas et al., 2017; Parker et al., 2016; Roeper, 2013). Therefore, an important question is which subpopulation of VTA DA neurons generates the response to the US omission. One possible candidate is the subpopulation projecting to the NAc. Supporting this possibility, an increase in DA release around the time of the CS offset during fear extinction has been observed in the NAc (Badrinarayan et al., 2012). In particular, this was observed only during the early trials of extinction, consistent with our results. DA signaling in the NAc has also been shown to be important for relief learning (Mayer et al., 2018) and avoidance behavior (Gentry et al., 2016; Oleson et al., 2012). Whether fear extinction learning and relief and avoidance learning share related mechanisms and involve overlapping subpopulations of DA neurons projecting to NAc is not known and will be an important question for future research. Notably, pharmacological blockade of DA receptors in the NAc have been found to impair fear extinction learning (Holtzman-Assif et al., 2010). Furthermore, we have observed that fear extinction learning in humans is accompanied by a prediction error-like activation in the ventral striatum (Raczka et al., 2011). However, at odds with these findings, inhibition of DA terminals in NAc or DA neurons projecting to the medial shell of NAc during US omission did not affect fear extinction learning, although it did impair consolidation of extinction memory, in a recent study (Luo et al., 2018). Our single unit results demonstrate that a small subpopulation of DA neurons mediate this DA signal to drive extinction learning. Therefore, it is possible that this subpopulation of DA neurons projects to a specific subregion of NAc that was not targeted by Luo et al. (2018). In addition to NAc, other possible candidates include the DA neurons that project to the amygdala and/or IL. Identifying which projection-defined subpopulation of DA neurons signals the omission of the US to initiate fear extinction learning will be an important question for future research.

In conclusion, our study identifies a prediction error-like signal encoded by DA neurons that is necessary to initiate fear extinction learning. Furthermore, we found that enhancing this DA signal is sufficient to accelerate extinction learning and strengthen extinction memory consolidation. Deficits in fear extinction learning are thought to underlie anxiety disorders (Craske et al., 2017; Graham and Milad, 2011; Mahan and Ressler, 2012; Milad and Quirk, 2012; Pitman et al., 2012). Our study therefore has therapeutic implications for anxiety disorders by identifying DA neuron activity as a potential target for novel treatments.

Materials and methods

Subjects

All procedures were conducted in accordance with the guidelines of the German Animal Protection Act and were approved by the local authorities (Regierungsprasidium Darmstadt; protocol number 1038). Male C57BL/6N mice (Charles River), aged 3 months at the start of experiments, were used in the in vivo electrophysiology experiment. Male heterozygous DAT-Cre mice (Zhuang et al., 2005; backcrossed with C57BL/6N) aged 3–6 months at the start of experiments were used in the photometry and optogenetics experiments. All experimental groups were matched for age. All mice were individually housed on a 12 hr light/dark cycle. All experiments were performed during the light cycle.

Viruses

AAV5-EF1a-DIO-hChR2(H134R)-eYFP, AAV5-EF1a-DIO-eNpHR3.0-eYFP, AAV5-EF1a-DIO- eYFP and AAV5-CAG-Flex-GFP were produced and packaged by the University of North Carolina Vector Core. AAV5-CAG-Flex-GCaMP6f-WPRE-SV40 and AAV5-CAG-Flex- GCaMP6s-WPRE-SV40 were produced and packaged by the University of Pennsylvania Vector Core.

Surgical procedures

Request a detailed protocol

Animals were anesthetized using isoflurane (1–2%) and placed in a stereotaxic frame. At the onset of anesthesia, all animals received subcutaneous injections of carprofen (4 mg/kg) and dexamethasone (2 mg/kg). The animal’s temperature was maintained for the duration of the surgical procedure using a heating blanket. Anesthesia levels were monitored throughout the surgery and the concentration of isoflurane adjusted so that the breathing rate never fell below 1 Hz. After exposing the skull surface, craniotomies were made overlying the VTA (3.2 mm posterior to bregma and 0.5 mm lateral to the midline).

For in vivo single-unit recordings, we used a moveable bundle of 5–8 stereotrodes made by twisting together two 0.0005 inch tungsten wires (M219350, California Fine Wire). The stereotrode bundle was attached to a custom-made microdrive that made it possible to advance the electrodes along the dorsoventral axis. On the day of implantation, the stereotrodes were gold-plated to reduce the impedance to 0.2–0.3 MΩ at 1 kHz. The stereotrode bundle was inserted through the craniotomy above the VTA to a depth of 3.9–4.0 mm below bregma. All electrode wires were connected to an electrode interface board (EIB-16; Neuralynx) for relaying electrophysiological signals to the data acquisition system. The microdrive was anchored to the skull using skull screws and dental cement (Paladur).

For the photometry experiments, DAT-cre mice were injected unilaterally with 1 μl of AAV5-CAG-Flex-GCaMP6f-WPRE-SV40 (final titer 2.7 × 1012 pp per ml) or AAV5-CAG-Flex- GCaMP6s-WPRE-SV40 (final titer 6.4 × 1012 pp per ml) or AAV5-CAG-Flex-GFP (final titer 4.5 × 1012 pp per ml). Viruses were injected in the VTA (3.2 mm posterior to bregma, 0.5 mm lateral to the midline and 4.5 mm ventral to bregma) at 50 nl/min using a 10 μl syringe with a 33-gauge needle controlled by an injection pump. The needle was left in place for an additional 10–15 min before slowly being withdrawn. Following infusion of the virus, an optical fiber (400 μm core diameter, 0.48 NA, Doric Lenses) was slowly inserted through the same craniotomy into the VTA at a depth of 4.1–4.3 mm below bregma. The optical fiber was then anchored to the skull using skull screws and dental cement (Paladur).

For optogenetic experiments, DAT-cre mice were injected bilaterally in the VTA with 1 μl of AAV5-EF1a-DIO-hChR2(H134R)-eYFP (final titer 4.3 × 1012 pp per ml), AAV5-EF1a-DIO- eNpHR3.0-eYFP (final titer 4 × 1012 pp per ml) or AAV5-EF1a-DIO-eYFP (final titer 4.4 × 1012 pp per ml) per hemisphere using the coordinates described above. Optical fibers (200 μm core diameter, 0.22 NA, Thorlabs) were implanted bilaterally above the VTA to a depth of 3.9–4.0 mm below bregma as described above.

Behavior

Request a detailed protocol

Fear conditioning and extinction took place in two different contexts (A and B). Context A consisted of a square chamber with an electrical grid floor (Med Associates) used to deliver footshock. Context B consisted of a white teflon cylindrical chamber with bedding material on the floor. The chambers were located inside a sound attenuating box and were cleaned with 50% ethanol and 1% acetic acid before and after each session. All mice were habituated to contexts A and B for 10–15 min each in a counterbalanced fashion. For electrophysiology and photometry experiments, mice received a tone habituation session followed by fear conditioning on day 1. Tone habituation started following a 2 min baseline period in context A and consisted of 10 presentations of the CS (4 kHz tone, 75 dB, 10 s) with a random intertrial interval (ITI) of 40–120 s. Fear conditioning consisted of five pairings of the CS with a US (1 s footshock, 0.55 mA, ITI: 40–120 s). The onset of the US coincided with the offset of the CS. On day 2, mice received an extinction session consisting of 25–30 and 30 presentations of the CS alone in context B in the electrophysiology and photometry experiments, respectively. For optogenetic experiments, on day 1 mice underwent fear conditioning consisting of four pairings of a CS (4 kHz tone, 75 dB, 30 s) with a US (1 s footshock) with a random ITI of 40–120 s in context A. The US intensity was 0.45 mA and 0.5 mA in the optogenetic inhibition and excitation experiments, respectively. On day 2, mice received an extinction session consisting of 25 presentations of the CS alone in context B. Twenty-four hours later, mice received an extinction recall test in context B. The extinction recall test consisted of 25 and 10 presentations of the CS alone in the optogenetic inhibition and excitation experiments, respectively.

The behavior of mice was recorded to video and scored by an experienced observer blind to the experimental condition. Behavioral freezing, defined as the absence of all bodily movements except breathing-related movement (Blanchard and Blanchard, 1972), was used as the measure of fear. Animals that showed low conditioned fear (<50% freezing) at the start of the extinction session (First CS of extinction) were excluded from the study. This criterion led to the exclusion of 1 mouse from photometry and one mouse from optogenetics experiments.

In vivo single-unit electrophysiology

Request a detailed protocol

Following one week of recovery from surgery, animals were habituated to handling and moving with the wire tether connecting the microdrive to the recording system. Before fear conditioning began, stereotrodes were slowly advanced until single-units with low baseline firing rate (<10 Hz) were observed, to increase the probability of recording putative DA neurons during the task. The spike waveforms tended to change from day 1 (tone habituation and conditioning) to day 2 (extinction) even when electrodes were not moved or in case cells were lost overnight, we advanced (40–80 μm) the microdrive to find new cells on day 2. We therefore treated the cell populations recorded on these two days as independent. On each day of the fear conditioning task, single-units were first recorded for 5–10 min while animals were in their homecage to assess their baseline firing rates.

Neural data were acquired using a 16-channel headstage (HS-18, Neuralynx) that was connected to the electrode interface board and relayed the signals to a Digital Lynx SX (Neuralynx) data acquisition system. To extract putative spikes, neural signals were bandpass filtered between 600 and 6000 Hz, and waveforms that passed the threshold (50–60 μV) were digitized at 30 kHz. One stereotrode channel that did not have any apparent units was used as reference. We also recorded the neural signals bandpass filtered between 1 and 6000 Hz to analyze the spike waveforms. In order to verify recording locations, current (50 mA, 10 s) was passed through one of the stereotrode channels to produce a lesion in the recording site at the end of the experiment.

Analysis of single-unit data

Request a detailed protocol

Spike waveforms were sorted offline into single-unit clusters using SpikeSort3D (Neuralynx). Only well-isolated single-units that displayed a clear refractory period (>1 ms) were included in subsequent analysis, performed using scripts custom-written in MATLAB (MathWorks). The VTA contains different cell types, including DA neurons and gamma-aminobutyric acid (GABA) neurons (Morales and Margolis, 2017). Midbrain DA neurons typically exhibit baseline firing rates below 10 Hz in awake animals (Jin and Costa, 2010; Li et al., 2012) and long duration action potentials (Grace and Bunney, 1980; Jin and Costa, 2010; Li et al., 2012). We therefore classified neurons whose baseline firing rate (measured in the home cage) were below 10 Hz and exhibited long duration action potentials (peak-to-peak duration >450 µs) as putative DA neurons (Figure 1—figure supplement 3). Spike waveforms were analyzed using the neural signals bandpass filtered between 1 and 6000 Hz. To analyze firing rates during the task, we constructed peri-stimulus time histograms (PSTHs) aligned to the onset of the CS (−5 s to +15 s) using 1 s bins. PSTHs were calculated separately for early extinction trials (E-Ext: average of first 10 CSs), late extinction trials (L-Ext: average of last 10 CSs) and tone habituation (Hab: average of 10 CSs). These PSTHs were then normalized with a z-score transformation by subtracting the baseline firing rate (5 s pre-CS period) from each individual 1 s bin and then dividing this difference by the standard deviation of the baseline. A neuron was classified as US omission excited if it met two criteria: 1) z-score was greater than 2 at the time of the US omission (during the 1 s bin following the offset of the CS); 2) the z-score had to increase by at least two from the last bin of the CS to the US omission bin to ensure that the increase in firing was specific to the omission of the US and not a result of sustained increase in firing to the CS. Conversely, a neuron was classified as US omission inhibited if (1) z-score was smaller than −2 at the time of the US omission (during the 1 s bin following the offset of the CS); (2) the z-score had to decrease by at least two from the last bin of the CS to the US omission bin to ensure that the decrease in firing was specific to the omission of the US and not a result of sustained decrease in firing to the CS. To quantify responses to the CS, we averaged the z-scores during the entire CS period (10 s) for Hab, E-Ext and L-Ext trials and obtained an average z-score for each neuron. The neurons were classified as excited or inhibited by the CS if they showed average z-scores greater than two or smaller than −2, respectively.

Calcium recordings using fiber photometry

Request a detailed protocol

Animals were injected with viral vectors and implanted with an optical fiber in the VTA, as described above. After a waiting period of 3–4 weeks to allow for surgical recovery and virus expression, animals were connected to a 400 μm patch cord (Doric Lenses). Fluorescence was measured by delivering 465 nm excitation light through the patch cord and separating the emission light at 525 nm with a beamsplitter (Fluorescence MiniCube FMC5, Doric Lenses). The emission light was collected using a Femtowatt Silicon Photoreceiver (Model # 2151, Newport). The voltage output of the photoreceiver was then digitized at 2 kHz (Digital Lynx SX, Neuralynx). Using the same approach we also measured the isosbestic (activity-independent) fluorescence of gCaMP6 at 430 nm using 405 nm excitation light. After animals were habituated to handling and being connected to the patch cord, they underwent the fear conditioning protocol. At the end of fear conditioning experiments, animals were tested on an operant conditioning task (Figure 3—figure supplement 2). To this end, animals were placed in an operant chamber containing a water delivery port. Each nosepoke into the delivery port that followed the previous nosepoke by a variable inter-trial interval (3–5 s) triggered the delivery of liquid reward (10% sucrose solution) with a 50% probability. This task was used to verify that reward delivery caused large increases in fluorescence in gCaMP6-expressing mice, as expected based on previous electrophysiology (Bayer and Glimcher, 2005; Eshel et al., 2015; Eshel et al., 2016; Roesch et al., 2007; Schultz et al., 1997) and fiber photometry (Menegas et al., 2017; Parker et al., 2016; Soares et al., 2016) studies, suggesting that this Ca+2 signal is indeed generated by DA neuron activity.

Analysis of fiber photometry data

Request a detailed protocol

The voltage output of the photoreceiver, representing fluctuations in fluorescence, was first low-pass filtered at 4 Hz and then downsampled to 10 Hz. The change in fluorescence evoked by the CS (dF/F) was then calculated by subtracting from each trace the baseline fluorescence (average during the 5 s before CS onset) and dividing it by the baseline fluorescence. dF/F traces were then averaged separately for each animal for early extinction trials (E-Ext: average of first 10 CSs), late extinction trials (L-Ext: average of last 10 CSs) and tone habituation (Hab: average of 10 CSs). To examine responses to US omission we further averaged dF/F values in the 5 s following CS offset. Similar results were obtained from animals expressing gCaMP6f (n = 5) and gCaMP6s (n = 5), therefore the data was pooled. To quantify responses to the CS, we averaged the dF/F values during the CS for each animal for Hab, E-Ext and L-Ext. To quantify responses to reward (Figure 3—figure supplement 2), average dF/F was calculated separately for rewarded and unrewarded nosepokes using the baseline fluorescence 3 s before noseport entry.

Optogenetic manipulations

Request a detailed protocol

For bilateral optogenetic manipulations during behavior, the implanted optical fibers (200 μm core diameter, 0.22 NA, Thorlabs) were connected to 200 μm patch cords (Thorlabs) with zirconia sleeves and the patch cords were connected to a light splitting rotary joint (FRJ 1 × 2 i, Doric Lenses) that was connected to a laser with a 200 μm patch cord (Thorlabs). For mice expressing the light-activated inhibitory opsin halorhodopsin (NpHR-eYFP) and their eYFP controls, yellow light was delivered from a DPSS 594 nm laser (Omicron). Laser power at the tip of the optic fiber was 10–15 mW. To inhibit DA neurons around the time of the US omission during the extinction session on day 2, the laser was turned on from 1 s before to 2 s after the CS offset. The laser was then turned off gradually using a 1 s ramp to avoid rebound excitation (Mahn et al., 2016Figure 5—figure supplement 2). For mice in the Unpaired-NpHR control group, the light was delivered the same way during the ITIs. For mice expressing the light activated excitatory opsin channelrhodopsin-2 (ChR2) and their eYFP controls, blue light pulses were delivered from a 473 nm laser (LuXx473, Omicron). The laser power at the tip of the optic fiber was 5–10 mW. To phasically activate DA neurons at the time of the US omission, 5 ms light pulses were delivered at 20 Hz for 1 s beginning at the offset of the CS. The mice in the Unpaired-ChR2 group received delivery of the same light pulses during the ITIs.

In vivo single unit recordings during optogenetic excitation and inhibition

Request a detailed protocol

Using the surgical procedures described above DAT-cre mice were injected in the VTA with 1 μl of AAV5-EF1a-DIO-hChR2(H134R)-eYFP (final titer 4.3 × 1012 pp per ml) or AAV5-EF1a-DIO- eNpHR3.0-eYFP (final titer 4 × 1012 pp per ml) per hemisphere using the coordinates described above. Two to three weeks after virus injection, animals were anesthetized and placed in a stereotaxic frame with the skull exposed. A stainless-steel head post (Luigs and Neumann) was then cemented to the exposed skull. The area of the skull overlying VTA was left free of cement but covered with a silicon elastomer (Kwik-Sil, World Precision Instruments). Skull screws were inserted over the frontal cortex and cerebellum to serve as reference and ground, respectively, and to provide anchoring support for the cement. Following recovery from surgery, animals were handled and habituated to being head-fixed, by inserting the head post into a matching head post holder (Luigs and Neumann). Following 2–3 days of habituation to being head-fixed, animals underwent another surgery to prepare a craniotomy over the VTA. Animals were anesthetized with isoflurane and placed in a stereotaxic frame. The Kwik-Sil was removed from the skull and a small craniotomy was then made in the skull over the VTA and then sealed with Kwik-Sil. The following day, the animals were head-fixed, the Kwik-Sil removed, and a 32-channel optrode (silicon probe with an optical fiber; A1 × 32-Edge-5mm-20–177-OA32, NeuroNexus) was lowered into the VTA using a micromanipulator (SM-8, Luigs and Neumann). The optrode was then advanced to a depth of ~4700 μm below the brain surface to span the dorsal-ventral extend of the VTA. Following final placement of the optrode and a brief waiting period (∼15 min), neural activity was recorded while laser pulses (yellow light: 10–15 mW, blue light: 5–10 mW) were delivered through a patch cord (Neuronexus) connected to the optic fiber. Electrophysiological signals were filtered between 1 and 6000 Hz, digitized at 30 kHz using a digitizing headstage (RHD2132 Amplifier Board, Intan Technologies), and acquired using a USB interface board (RHD2000, Intan Technologies). Once the recording was over, optrode was removed from the brain and then was penetrated to another location in the VTA. Multiple penetrations and recordings were performed during a session. Before the final penetration, the silicone probe was coated with a fluorescent dye (DiI, Life Technologies) to assist with the identification of the recording location.

Histology

Request a detailed protocol

At the end of the experiments, mice were deeply anesthetized with sodium pentobarbital and were transcardially perfused with 4% paraformaldehyde and 15% picric acid in phosphate-buffered saline (PBS). Brains were removed, post-fixed overnight and coronal brain slices (60 µm) were sectioned using a vibratome (VT1000S, Leica). Standard immunohistochemical procedures were performed on free-floating brain slices. Briefly, sections were rinsed with PBS and then incubated in a blocking solution (10% horse serum, 0.5% Triton X-100% and 0.2% BSA in PBS) for 1 hr at room temperature. Slices were then incubated in a carrier solution (1% horse serum, 0.5% Triton X-100% and 0.2% BSA in PBS) containing the primary antibody overnight at room temperature. The next day, the sections were washed in PBS and then incubated in the same carrier solution containing the secondary antibody overnight at room temperature. The following primary antibodies were used: polyclonal rabbit anti-tyrosine hydroxylase (TH, catalog # 657012, 1:1000, Calbiochem), monoclonal mouse anti-TH (catalog # MAB318, 1:1000, Millipore), polyclonal rabbit anti-GFP (catalog # A11122, 1:1000, Life Technology). The following secondary antibodies were used: Alexa Fluor 568 goat anti-rabbit (catalog # A11011, 1:1000, Thermo Fisher Scientific, Invitrogen), Alexa Fluor 568 goat anti-mouse (catalog # A11004, Thermo Fisher Scientific, Invitrogen), and Alexa Fluor 488 goat anti-rabbit (catalog # A11008, 1:1000, Thermo Fisher Scientific, Invitrogen). Finally, sections were washed with PBS, mounted on slides and coverslipped with a 4’,6-diamidin-2-phenylindol (DAPI) containing medium (VECTASHIELD, Vector Laboratories) or incubated for 10 min in 0.1 M PBS containing 0.02% DAPI (catalog # D1306, Molecular Probes, Invitrogen), washed for 10 min in PBS, mounted on slides and coverslipped. Animals with incorrect electrode/optical fiber placements were excluded from data analysis. A total of four mice were excluded from the study due to incorrect placement of electrodes/optical fibers. We excluded one animal from the electrophysiology experiment because the electrodes were placed outside (posterior to) the VTA. Three animals were excluded from the fiber photometry experiment. In one animal the optical fiber was placed too dorsal and in the other two animals too ventral in the VTA. In these animals, the neuronal activity dependent fluorescence signal was undetectable or too low. No animals were excluded from optogenetic experiments.

Statistics

Data were statistically analyzed using GraphPad Prism (GraphPad Software) and MATLAB (Mathworks). All statistical tests were two-tailed and had an α level of 0.05. All error bars show s.e.m. All ANOVAs were followed by Bonferroni post hoc tests if significant main or interaction effects were detected. No statistical methods were used to predetermine sample size, but our sample sizes were similar to those generally used in the fear conditioning field. Animals were randomly assigned to experimental groups before the start of each experiment after ensuring that all experimental groups were matched for age. All results were obtained using groups of mice that were run in several cohorts.

References

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
  54. 54
  55. 55
  56. 56
  57. 57
    A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement
    1. RA Rescorla
    2. AR Wagner
    (1972)
    In: A. H Black, W. F Prokasy, editors.  In Classical Conditioning II: Current Research and Theory. Appleton-Century Crofts. pp. 64–99.
  58. 58
  59. 59
  60. 60
  61. 61
  62. 62
  63. 63
  64. 64
  65. 65
  66. 66
  67. 67
  68. 68
  69. 69
  70. 70
  71. 71

Decision letter

  1. Rui M Costa
    Reviewing Editor; Columbia University in the City of New York, United States
  2. Catherine Dulac
    Senior Editor; Harvard University, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Dopamine Neurons Drive Fear Extinction Learning by Signaling the Omission of Expected Aversive Outcomes" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Stephan Lammel (Reviewer #1); Naoshige Uchida (Reviewer #2); Nicolas Singewald (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

The reviewers see value in your work, and think it is interesting, but also think that more attention needs to be placed in describing what is novel in this study versus other recent studies.

In addition, the reviewers request more details and potential experiments, especially regarding the electrophysiological recordings. They suggest further characterization of the electrophysiological recordings (with potentially an increase of N of neurons, which is quite low), more details about the classification of the neurons (and photoId if available), characterization of the responses of putative non-DA neurons, and if possible the characterization of the effects of the optogenetic manipulations.

Below you find the point-by-point comments that can help you prepare your revision.

Reviewer #1:

In this study, Salinas-Hernandez et al. investigate DA neurons during fear extinction learning and describe a role of these cells for "safety learning". Major findings of this study include: i) Midbrain DA neurons are activated by the omission of an aversive stimuli during fear extinction and ii) optogenetic inhibition or stimulation of DA neurons at the time when an expected aversive stimulus is omitted impairs or accelerates fear extinction learning, respectively.

The results in this manuscript are convincing, the experiments are well done and I do not have any major technical concerns. However, a major concern is that, at least in its current form, the study provides relatively little novel insights and no substantial conceptual advance. The concept that omission of an expected aversive outcome activates DA neurons and increases DA release in the NAc during extinction is well established. This has been shown using various methodological approaches including in vivo electrophysiology and fast scan voltammetry (Badrinarayan et al., 2012; Matsumoto and Hikosaka, 2009; Oleson et al., 2012). This concept is consistent with previous studies reporting that the increase in DA firing or DA release caused by an aversive stimulus may reflect the rewarding effects after its termination or the relief caused when it is omitted – a process which is probably very similar to extinction (Brischoux et al., 2009; Budygin et al., 2012; Mirenowicz and Schultz, 1996). The finding that DA activity during the omission of an expected aversive outcome is necessary for fear extinction learning seemed to be the most exciting finding of this paper. However, I noticed that the same finding has been published just recently by another group (Luo et al., 2018). In addition, the Johansen group presented evidence that this effect is specifically mediated by NAc-projecting DA neurons – a finding that I felt was missing in the present work. So, it's nice to see replication of this previous work, but the reviewing editors will need to decide how to weigh reproducibility vs. novelty in this case.

What was surprising to me is that the authors only found a very small number of putative DA neurons (8/30) that exhibit an increase in firing rate to the omission of the aversive US during the early extinction trials (subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning”, first paragraph and Figure 1). Let's assume the recorded cells are indeed all dopaminergic, then it means that only a very small subpopulation of DA neuron mediates this effect. A limitation of the fiber photometry approach is that there is little information about how uniform the activity is across neurons and changes in net activity may be the dominant pattern of activity of a small subset of neurons. Additionally, the data in Figure 2F suggest that at least some animals also showed an increase in calcium activity during the CS presentation.

It is also possible that the non-responsive cells in Figure 1 are non-dopaminergic (i.e., putative GABAergic or glutamatergic neurons). Thus, another limitation of the present work is that the authors did not unambiguously identify DA neurons. Their conclusions rely upon indirect methods such as spike waveforms and response to rewarding stimuli. These identification methods have been called into question and may not be appropriate for the identification of DA neurons (Margolis et al., 2006, 2010; Ungless and Grace, 2012). Thus, the authors should consider alternative approaches such as an optogenetic tagging method to unambiguously identify DA neurons (and other VTA cell populations) in highly heterogeneous brain regions such as the VTA – they clearly have the technical expertise to do this.

References:

Brischoux, F., Chakraborty, S., Brierley, D.I., and Ungless, M.A. (2009). Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. 106, 4894-4899.

Budygin, E.A., Park, J., Bass, C.E., Grinevich, V.P., Bonin, K.D., and Wightman, R.M. (2012). Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience 201, 331-337.

Margolis, E.B., Lock, H., Hjelmstad, G.O., and Fields, H.L. (2006). The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J Physiol 577, 907-924.

Margolis, E.B., Coker, A.R., Driscoll, J.R., Lemaître, A.-I., and Fields, H.L. (2010). Reliability in the identification of midbrain dopamine neurons. PloS One 5, e15222.

Mirenowicz, J., and Schultz, W. (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449-451.

Ungless, M.A., and Grace, A.A. (2012). Are you or aren't you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35, 422-430.

Reviewer #2:

It has been proposed that fear extinction induces new learning, not merely the erasure of the previous fear memory. The mechanism underlying new learning during fear extinction remains elusive. In this study, Salinas-Hernandez and colleagues examined the role of VTA dopamine neurons in this process. The authors first characterized the activity of VTA dopamine neurons using electrophysiological recording and fiber photometry during fear extinction. The authors show that 26.6% of putative dopamine neurons were excited by the omission of electrical shock during the early extinction period, while a smaller fraction of neurons (10%) were excited during the late extinction period. Furthermore, the authors show that optogenetic inhibition of VTA dopamine neurons at the time of omission of electric shock impaired fear extinction as measured by the decrease of freezing. The authors also show that optogenetic activation of VTA dopamine neurons at the time of omission of electric shock facilitated the decrease of freezing.

Previous studies, using cyclic voltammetry and pharmacology/lesions, have indicated the role of dopamine in fear extinction. The present study extends these studies significantly by reporting single dopamine neuron activities, and by examining the effect of manipulating the activity of VTA dopamine neurons precisely at the time of shock omission using optogenetics. The results of these experiments are convincing and somewhat surprising. This study provides important insights into the mechanism of fear extinction learning.

There are some recent publications that addressed similar questions independently (Luo et al., 2018; Mayer et al., 2018), which reached overlapping conclusions. The present study complements these studies by providing additional information which were not reported in other studies (e.g. single neuron recording and the effect of optogenetic stimulation). Overall, these studies together make important contributions to the fear extinction literature.

1) One of the important and novel aspects of this study is single neuron recording during fear extinction. Although the authors report the minimum results to support their conclusions, this study can be further strengthened if the authors report a little more detail of the recorded neuron population. The authors first select those that were activated during shock omission and report the average firing pattern of these neurons. This represents only a fraction of all putative dopamine neurons (8 out of 30 putative dopamine neurons). It is important to know how other putative dopamine neurons responded (no response or were some of them inhibited?). Please show the distribution of the responses of all putative dopamine neurons during shock omission. Is the average response of all putative dopamine neurons significantly positive?

Furthermore, the authors have recorded the activity of unidentified dopamine neurons. Although this is not essential for the authors' conclusion, it would be useful to report how these neurons responded. Are there narrow-spiking neurons that were activated during shock omission? This may have already been done in Figure 1—figure supplement 2: does the red circle represent a putative non-dopaminergic neuron that was activated during shock omission? Do the black circles actually represent putative non-dopaminergic neurons that were not activated during shock omission? Please clarify. Also, it would be useful to describe these results in the main text.

2) Please report the result of quantification for Figure 2B, 3B, C (TH staining compared with the expression of NpHR or ChR2).

3) Figure 2G (the correlation between 'Percent freezing' and 'Average dF/F') is very interesting. However, additional information is needed to interpret this result. First, it is better to use the reduction in freezing rather than the absolute value of freezing, in order to represent learning. Does the correlation hold even if the authors use the difference of freezing between before and during/after extinction? Second, it is a little unclear what is causing the variation in the photometry signal. One possibility, which the authors would like to exclude, is the variation in recording location. To address this, can the authors examine the correlation between freezing (ideally the change in freezing) and reward response? Not obtaining significant correlation in this analysis would strengthen the authors' conclusion.

4) Please cite and discuss Luo et al., 2018 and Mayer et al., 2018.

Reviewer #3:

Combining sophisticated electrophysiological, photometric and optogenetic techniques the authors studied the question of how dopaminergic neurons are involved in extinction learning. In their experiments they were able to demonstrate the importance of activity of the midbrain ventral tegmental area (VTA) dopamine system in the initiation and facilitation of cued fear extinction. Activation of dopaminergic neurons is observed when an expected aversive event does not occur, which is a crucial phase in the onset of extinction learning. The study is exciting and in large parts novel. It also provides further information on the neuronal basis of findings of a very recently published study by Luo et al. reporting a role for the VTA-nucleus accumbens dopamine system in extinction learning. Methods are state-of-the art, experiments were carried out with care and the manuscript is straightforward to read. However, there are points that need to be addressed.

1) As mentioned above, in a paper published a few weeks ago Luo and co-workers demonstrate that the optogenetic inactivation of VTA-dopamine cells during the US omission period reduced extinction learning in rats. Here, part of this finding is replicated in another species and extended as, for example: a) the optogenetic stimulation of VTA dopaminergic activity facilitated fear extinction in mice (Figure 4); b) (Re)extinction is possible in the paired-NpHR mice during extinction recall when the laser is switched off (Figure 3). This finding may also suggests that silencing of the VTA-dopamine activity did not reconsolidate the fear memory. It is necessary to cite the Luo paper and emphasize the overlap and differences (some of which mentioned above) with the present manuscript.

2) Please provide evidence that optical silencing and stimulation using NpHR and ChR2, respectively, is a valid method to modulate dopaminergic activity in the VTA of mice.

3) The dopamine system has been suggested to be involved in both the learning and consolidation of fear extinction. The authors nicely demonstrate its role in the initiation of fear extinction learning, but unfortunately do not further discuss this finding in relation to published work. For example, it has been shown that administration of the dopaminergic drug L-DOPA before extinction training was able to initiate fear extinction learning in a mouse model of impaired fear extinction. This discussion needs to be added. Also the consolidation aspect is not addressed sufficiently (one mentioning that "extinction memory was strengthened"). Furthermore, the authors focus on the better-than-expected outcome reward aspect of dopamine in fear extinction in the Discussion. There is increasing evidence that dopamine activity/release may also reflect a prediction error signal that guides extinction learning. Whether and how the present data support this latter idea, is missing and should be added.

https://doi.org/10.7554/eLife.38818.023

Author response

Reviewer #1:

In this study, Salinas-Hernandez et al. investigate DA neurons during fear extinction learning and describe a role of these cells for "safety learning". Major findings of this study include: i) Midbrain DA neurons are activated by the omission of an aversive stimuli during fear extinction and ii) optogenetic inhibition or stimulation of DA neurons at the time when an expected aversive stimulus is omitted impairs or accelerates fear extinction learning, respectively.

The results in this manuscript are convincing, the experiments are well done and I do not have any major technical concerns. However, a major concern is that, at least in its current form, the study provides relatively little novel insights and no substantial conceptual advance.

We thank the reviewer for a detailed reading of our manuscript. Although the reviewer finds our results convincing, his major concern is that “the study provides relatively little novel insights and no substantial conceptual advance”. We respectfully disagree with this assessment. To begin with we would like to emphasize that the key goal of our study was to investigate the neural mechanisms underlying the initiation of fear extinction learning. As we explained in the Introduction of the manuscript, this is a key outstanding question in the fear extinction literature. Because extinction learning happens when an expected aversive US does not occur, we hypothesized that dopamine (DA) neurons would be involved since they are known to respond to better-than-expected outcomes. Supporting this hypothesis, we found that DA neurons increased their activity at the time of the US omission during fear extinction and that this response correlated with extinction learning. Furthermore, using bidirectional optogenetic manipulations, we showed that this DA signal is not only necessary for, but also sufficient to accelerate, extinction learning. Our study therefore suggests that DA neurons initiate fear extinction by signaling US omission and makes an important contribution to our understanding of this form of safety learning.

As we acknowledged in the original manuscript, previous studies have demonstrated responses of DA neurons to the omission of an aversive stimulus. It appears that the reviewer’s assessment of the novelty of our results is to a large extent based on these studies (discussed below). However, we wish to emphasize that these previous studies do not clearly demonstrate that US omission responses are involved in fear extinction learning. In fact, omission responses have mainly been observed (although not always consistently) in studies using partial reinforcement paradigms (where the US is omitted on a subset of trials), which is not the same as extinction learning. Furthermore, two studies (Badrinarayan et al., 2012; Oleson et al., 2012) that examined DA release during fear extinction do not specifically examine or make any claims about the role of US omission responses in fear extinction learning (and in fact the second of these studies does not observe US omission responses at all). Finally, we acknowledge the study of Luo et al., 2018, that was published after our original submission and which found, as we did, that inhibiting DA neuron activity during US omission impairs fear extinction. However, our study also has numerous additional and novel findings that complement those of Luo et al., 2018. Taken together, we therefore believe that our study makes a novel and important contribution to the fear extinction literature. Because the reviewer brings up these studies in his following comments, we will discuss them and their relationship to our study in more detail below.

The concept that omission of an expected aversive outcome activates DA neurons and increases DA release in the NAc during extinction is well established. This has been shown using various methodological approaches including in vivo electrophysiology and fast scan voltammetry (Badrinarayan et al., 2012; Matsumoto and Hikosaka, 2009; Oleson et al., 2012).

The reviewer states that “The concept that omission of an expected aversive outcome activates DA neurons and increases DA release in the NAc during extinction is well established”. We respectfully disagree with the reviewer. First, the relatively few studies that have investigated the firing of DA neurons during omission of an expected aversive outcome have not always yielded consistent results. For instance, whereas Matsumoto and Hikosaka, 2009, found a small increase in firing to the omission of an aversive air-puff, Tian and Uchida, 2015, did not find any change in firing to air-puff omission. A more recent study, on the other hand, found a significant yet small increase in DA firing to air-puff omission when animals were tested in a low, but not in a high, reward context (Matsumoto et al., 2016) suggesting that the valence of the testing context influences omission responses. Finally, Menegas et al., 2017, did not observe any response to air-puff omission at the DA neuron terminals in the ventral striatum. These studies and their methodological differences to ours were discussed in the original manuscript. Perhaps the most important difference is that the behavioral paradigm used in these studies was not fear extinction but rather partial reinforcement, in which the expected aversive air-puff was omitted on a subset of trials (10-75%, depending on the study). Whether there was any extinction of behavioral responses to the stimulus predicting aversive airpuff in these studies is not clear. In fact, partial reinforcement has been shown to cause learning that is resistant to extinction, a phenomenon termed as “the partial reinforcement extinction effect” (Skinner, 1938; Bouton et al., 2014; Grady et al., 2016). Thus, although the results from partial reinforcement studies are interesting and relevant, they do not address the key question of our study, namely whether US omission responses of DA neurons are involved in fear extinction learning.

More directly relevant to our study are two previous studies that examined DA release in the nucleus accumbens (NAc) during conditioned fear expression and extinction using cyclic voltammetry. Badrinarayan et al., 2012, did indeed find that DA release in the NAc was significantly elevated compared to pre-CS levels in the time period after CS offset during extinction. However, it is unclear whether this was specific to the US omission period since DA release appears to start increasing during the CS (see their Figures 3B and 3H). Indeed, the authors themselves refer to this response “as a diffuse increase because it was not nearly as robustly or reliably associated with any temporal attributes of cue presentation” (p. 15783). Although they consider the possibility that the DA signal at CS offset could be a ‘relief signal’ they also acknowledge that “there is a gradual but not significant increase to that point throughout tone presentation that would not be explained by a relief signal (Figure 3B, H)” (p. 15788). Notably, Badrinarayan et al. (2012) do not make any claims that this increase signals the US omission or that it drives extinction learning. In contrast, the second voltammetry study by Oleson et al., 2012, observed decreased DA release in the NAc during the CS presentation but no changes in DA release at CS offset (see their Figure 2D) during fear extinction. We are therefore not sure why the reviewer has mentioned this study.

Based on the above considerations we think it’s fair to say that whether DA neurons signal the US omission during fear extinction is incompletely understood. Certainly, it is not accurate to state that it is “well established”, as the reviewer does. We furthermore believe that our study provides several important novel findings that more firmly establish US omission responses of DA neurons during fear extinction and their relationship to extinction learning.

1) We clearly demonstrate that putative DA neurons increase their activity at the time of the US omission during the early trials of fear extinction. Here it is worth emphasizing that for a neuron to be classified as US omission responsive its firing rate had to be significantly above the firing rate both before the CS as well as in the last bin of the CS (see Materials and methods, subsection “Analysis of single-unit data”). Thus our US omission responses cannot simply reflect a delayed response to the CS, which is a possibility in the Badrinarayan et al., 2012 study, as already discussed.

2) We show that at the cellular level, omission responses are exclusively excitatory and are seen only in putative DA but not non-DA neurons. These new results have been added to the revised version of the manuscript in response to reviewer #2.

3) We confirmed the US omission responses of DA neurons using DA neuron-specific fiber photometry measurements.

4) We show that the US omission signal is strongly correlated with extinction learning.

5) We show using bidirectional optogenetic manipulations that this signal drives extinction learning.

In our opinion, these findings add substantially to what has been reported in previous studies and make an important contribution to our understanding of DA neuron signaling during fear extinction.

This concept is consistent with previous studies reporting that the increase in DA firing or DA release caused by an aversive stimulus may reflect the rewarding effects after its termination or the relief caused when it is omitted – a process which is probably very similar to extinction (Brischoux et al., 2009; Budygin et al., 2012; Mirenowicz and Schultz, 1996).

The reviewer raises an interesting point. However, we think it is very much an open question whether the termination of an aversive stimulus is “a process which is probably very similar to extinction”. We are not aware of any study supporting this claim. It should be emphasized that extinction learning begins when an expected aversive stimulus does not occur. This is not the same as when an aversive stimulus is terminated. In addition, both the Brischoux et al., 2009, and Budygin et al., 2012, studies were performed in anesthetized animals. It is therefore not clear whether termination of the aversive stimulus was experienced as rewarding and caused any relief. It is also not clear whether similar responses are seen in awake behaving animals. Although the study by Mirenowicz and Schultz, 1996, mentioned by the reviewer was performed in awake animals they do not report any responses to the termination of an aversive event (air puff) so we are not sure how this study is relevant (in fact, the main conclusion of this paper is that DA neurons do not respond to aversive air puffs). It is also worth emphasizing that none of the studies mentioned examine responses to aversive US omission (the reviewer says ‘omitted’ but we assume he means ‘terminated’). Thus, to what extent DA neurons respond to aversive US termination, and whether this is in any way related to the US omission responses we see, is a matter of speculation. Certainly, the results of these studies cannot be viewed as diminishing the novelty of our results.

The finding that DA activity during the omission of an expected aversive outcome is necessary for fear extinction learning seemed to be the most exciting finding of this paper. However, I noticed that the same finding has been published just recently by another group (Luo et al., 2018). In addition, the Johansen group presented evidence that this effect is specifically mediated by NAc-projecting DA neurons – a finding that I felt was missing in the present work. So, it's nice to see replication of this previous work, but the reviewing editors will need to decide how to weigh reproducibility vs. novelty in this case.

The reviewer is indeed correct that Luo et al., 2018, also show that optogenetic inhibition of DA neurons in the VTA impairs fear extinction learning. However, we believe it is not fair/accurate to describe our study simply as a “replication of this previous work”. Our study also presents a number of additional novel findings supporting the role of DA neurons in fear extinction that is not in Luo et al., 2018. Most importantly, we show using both electrophysiology and fiber photometry, that DA neurons are in fact activated by US omission during fear extinction and that this omission response correlates with extinction learning. Such a demonstration is absent in Luo et al., 2018, but is critical for interpreting the effects of optogenetic manipulations in their study and ours. Also, in addition to inhibition of DA neurons, we show that stimulating them at the time of US omission enhances extinction learning, a finding that was not in Luo et al., 2018. We therefore believe that our study and that of Luo et al., 2018, complement each other and together make a strong case for the role of DA neurons in fear extinction learning.

In addition, we do not agree with the reviewer that Luo et al., 2018, provides evidence that the effect on fear extinction learning is specifically mediated by NAc-projecting DA neurons. In fact, they found that inhibition of DA neuron terminals in NAc did not have any effect on fear extinction learning (see their Figure 3E). Instead, this manipulation resulted in impaired recall of extinction the next day suggesting an effect on the consolidation of extinction memory. The same pattern of findings was also observed when the authors inhibited the DA neurons projecting to the medial shell of NAc (Figure 4C). It is well established that learning and consolidation are separate processes (McGaugh, 2000; Johansen et al., 2011). Therefore, which DA projection mediates the effect on fear extinction learning remains an open question. This issue is discussed in the Discussion section of our revised manuscript (seventh paragraph). We agree with the reviewer that identifying the relevant DA projection will be an important goal in future research.

What was surprising to me is that the authors only found a very small number of putative DA neurons (8/30) that exhibit an increase in firing rate to the omission of the aversive US during the early extinction trials (subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning”, first paragraph and Figure 1). Let's assume the recorded cells are indeed all dopaminergic, then it means that only a very small subpopulation of DA neuron mediates this effect.

In fact, compared to previous studies (Matsumoto and Hikosaka, 2009; Matsumoto et al., 2016; Tian and Uchida, 2015), we observed much stronger DA neuron responses and a larger percentage of cells showing increased activity to the omission of an aversive US (this was already discussed in the original version of the manuscript). That being said, the reviewer is correct that only a subpopulation of DA neurons shows the US omission response and we think this is an important point. It is by now well established that DA neurons are not a homogeneous population but should rather be viewed as a collection of subpopulations that are defined in part by their projection targets and behavioral functions (Roeper, 2013). Recent studies, measuring activity in axonal terminals of DA neurons, have furthermore begun to reveal clear differences in the response properties of these DA neuron subpopulations (see for example Howe and Dombeck, 2016; Menegas et al., 2017; 2018; Parker et al., 2016). It is therefore perhaps not to be expected that the number of neurons showing US omission responses, which also may have a specific projection target, is large. It should also not be very surprising that a subpopulation of neurons can mediate an effect on fear extinction learning; indeed many studies have demonstrated behavioral effects following manipulations of DA neuron subpopulations (Lammel et al., 2012; Matthews et al., 2016; Menegas et al., 2018; Tye et al., 2013).

A limitation of the fiber photometry approach is that there is little information about how uniform the activity is across neurons and changes in net activity may be the dominant pattern of activity of a small subset of neurons. Additionally, the data in Figure 2F suggest that at least some animals also showed an increase in calcium activity during the CS presentation.

We agree with the reviewer that a limitation of fiber photometry is that it reflects the average bulk activity of all DA neurons in the vicinity of the optic fiber. If we understand the reviewer correctly, he also suggests that the US omission response we observe in the fiber photometry signal may be driven by a subset of DA neurons, a possibility which we also agree with. Indeed, our electrophysiology results suggest that only a subset of putative DA neurons show a US omission response. It is also worth pointing out that we only observed excitatory responses to US omission, whereas none of our recorded putative DA neurons displayed inhibitory responses. As a result, a significant excitatory response to US omission can be observed in the average response of the entire putative DA neuron population (now added as Figure 1J in the revised manuscript in response to reviewer #2). Our electrophysiology results thus complement the fiber photometry results by suggesting that a subpopulation of DA neurons consistently respond with excitation to US omission.

The reviewer is also correct to point out that some animals showed an increase in calcium activity during the CS. We have now added a new figure (Figure 4 in the revised manuscript) to show the responses to the CS. We find, in contrast to the uniform increase in DA neuron activity during US omission (Figure 3G), that the responses to the CS varied across animals (Figure 4A), with some animals showing increased and others decreased DA neuron activity (Figure 4AB). Accordingly, the Ca+2 signal evoked by the CS was not significantly different from the baseline when we averaged the CS responses of all animals (Figure 4A). These results are also consistent with the CS responses observed in our putative DA neurons which are now shown in Figure 1—figure supplement 4 in the revised manuscript.

It is also possible that the non-responsive cells in Figure 1 are non-dopaminergic (i.e., putative GABAergic or glutamatergic neurons). Thus, another limitation of the present work is that the authors did not unambiguously identify DA neurons. Their conclusions rely upon indirect methods such as spike waveforms and response to rewarding stimuli. These identification methods have been called into question and may not be appropriate for the identification of DA neurons (Margolis et al., 2006, 2010; Ungless and Grace, 2012). Thus, the authors should consider alternative approaches such as an optogenetic tagging method to unambiguously identify DA neurons (and other VTA cell populations) in highly heterogeneous brain regions such as the VTA – they clearly have the technical expertise to do this.

The reviewer raises an important point. It is indeed possible that some of the non-responsive neurons we classified as DAergic might actually be non-DAergic. However, it is important to note that such a misclassification would lead to an underestimation of the proportion of responsive DA neurons but would not change our conclusion that DA neurons signal the omission of an aversive US. More generally, we agree with the reviewer that the classical methods we used to identify DA neurons (i.e. spike waveform analysis) have their limitations. However, it should be emphasized that we used similar identification methods as in the majority of electrophysiology studies that have examined the activity of DA neurons in awake behaving animals. Some of these studies (performed by the labs of Schultz, Hikosaka and Schoenbaum) were the first to identify reward prediction error signaling in dopamine neurons, and similar responses were later observed using opto-tagged DA neurons (by the Uchida Lab). We therefore believe that, although not ideal, the classical waveform analysis methods are still of value. Nonetheless, because of the above mentioned limitations, we sought to confirm our electrophysiology results with DA neuron-specific fiber photometry recordings, a technique that has been used by a number of studies in the DA literature (Cui et al., 2013; Gunaydin et al., 2014; Parker et al., 2016; Matthews et al., 2016; Menegas et al., 2017; 2918; Soares et al., 2016). Importantly, we found similar results using this technique indicating that the single-units that we classified as putative DA neurons were most likely DA neurons.

We also agree with the reviewer that it would have been better to use the opto-tagging method to identify DA neurons. It is not clear to us why the reviewer thinks that we "clearly have the technical expertise to do this". We do not have any publications in which we performed optotagging of DA neurons. This is a method which we have not established sufficiently well for recording from freely behaving animals during fear extinction (it should be emphasized that for each animal there is only one extinction session and thus sampling of different neurons across different sessions is not possible, as it is for many other reward-related behaviors). On the other hand, we have been able to opto-tag DA neurons in head-fixed mice where we record acutely using opto-probes. We have used this approach to address a comment raised by Reviewer #3 (see his point #2, below).

Reviewer #2:

[…] 1) One of the important and novel aspects of this study is single neuron recording during fear extinction. Although the authors report the minimum results to support their conclusions, this study can be further strengthened if the authors report a little more detail of the recorded neuron population. The authors first select those that were activated during shock omission and report the average firing pattern of these neurons. This represents only a fraction of all putative dopamine neurons (8 out of 30 putative dopamine neurons). It is important to know how other putative dopamine neurons responded (no response or were some of them inhibited?). Please show the distribution of the responses of all putative dopamine neurons during shock omission. Is the average response of all putative dopamine neurons significantly positive?

We are grateful for the reviewer’s very careful and detailed review of our study. It is correct that we focused our analysis on the putative DA neurons that were activated during the US omission and reported the average firing pattern of only these neurons. We agree with the reviewer that presenting the response profile of other putative DA neurons will further strengthen our study. To address this we have examined whether there were any putative DA neurons in our dataset – which we have increased with the addition of data from 4 new mice – that were inhibited by the US omission. We found that none of the putative DA neurons (0 out of 40 neurons) showed a selective inhibition to the US omission during either early or late extinction. This suggests that the dominant response of putative DA neurons to the omission of the US was excitation. We next looked at the average response of all putative DA neurons, as suggested by the reviewer, and found that it was significantly positive (elevated) at the time of the US omission during early extinction. Notably, this finding is consistent with our results obtained using fiber photometry which measures the average DA neuron population activity. These results are now described in the second paragraph of the subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning” and in Figure 1J in the revised manuscript. Furthermore, we included a new figure (Figure 2A in revised manuscript) where we show the distribution of responses of all putative DA neurons during US omission and indicate the significantly excited/inhibited neurons. We believe that these new results strengthen the conclusion of our study and we thank the reviewer for his constructive comments and suggestions.

Furthermore, the authors have recorded the activity of unidentified dopamine neurons. Although this is not essential for the authors' conclusion, it would be useful to report how these neurons responded. Are there narrow-spiking neurons that were activated during shock omission? This may have already been done in Figure 1—figure supplement 2: does the red circle represent a putative non-dopaminergic neuron that was activated during shock omission? Do the black circles actually represent putative non-dopaminergic neurons that were not activated during shock omission? Please clarify. Also, it would be useful to describe these results in the main text.

The reviewer brings up a very good point. Local circuit interactions between DA and GABA neurons in the VTA play an important role in reward learning (Cohen at al., 2012; Eshel et al., 2015). Thus, how non-DA neurons in VTA respond during fear extinction is an important and interesting question. Because our main aim was to record the activity of putative DA neurons, we sampled mostly neurons with low baseline firing rate (< 10 Hz) and broad spike waveforms. We therefore had a small number of putative non-DA neurons (narrow-spiking; 18 of 48 neurons) during the extinction session in the original manuscript. To address the issue raised by the reviewer, we have now included data from four new animals and increased the number of non-DA neurons (47 of 90 neurons during tone habituation and 35 of 75 neurons during extinction) substantially in the revised manuscript. Analysis of the activity of putative nonDA neurons revealed that these neurons did not respond to US omission. The proportion of non-DA neurons which showed a significant increase or decrease during the US omission was small (excited: 2.8%, inhibited: 8.5%) and not different than the proportions observed during tone habituation. These results therefore suggest that the US omission was selectively signaled by the putative DA neurons in the VTA. These new results are now described in the third paragraph of the subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning”. Furthermore, we include a new figure (Figure 2B) in the revised manuscript that shows the distribution of responses of all putative non-DA neurons during the US omission and indicate the significantly excited/inhibited neurons in this figure. In addition, we have clarified and expanded the information shown in Figure 1—figure supplement 2 (Figure 1—figure supplement 3 in the revised manuscript), as suggested by the reviewer. We now show the neurons recorded during tone habituation and extinction on separate figures and on each figure indicate the putative DA and non-DA neurons, as well as the distribution of significantly excited and inhibited neurons. We believe that addition of these new results strengthens our study and we again thank the reviewer for his constructive suggestions.

2) Please report the result of quantification for Figure 2B, 3B, C (TH staining compared with the expression of NpHR or ChR2).

We have now quantified the DA neuron-specific expression of viral constructs by comparing the expression of gCaMP6, NpHR and ChR2 with TH staining. Consistent with previous reports (Lammel et al., 2015), we observed a high level of DA neuron selective expression in DAT-cre mice. The results of the quantification for the fiber photometry and optogenetic experiments are presented in Figure 3—figure supplement 1C (for gCaMP6 and GFP groups), Figure 5—figure supplement 1C (for NpHR and eYFP groups) and Figure 6—figure supplement 1C (for ChR2 and eYFP groups) in the revised manuscript.

3) Figure 2G (the correlation between 'Percent freezing' and 'Average dF/F') is very interesting. However, additional information is needed to interpret this result. First, it is better to use the reduction in freezing rather than the absolute value of freezing, in order to represent learning. Does the correlation hold even if the authors use the difference of freezing between before and during/after extinction? Second, it is a little unclear what is causing the variation in the photometry signal. One possibility, which the authors would like to exclude, is the variation in recording location. To address this, can the authors examine the correlation between freezing (ideally the change in freezing) and reward response? Not obtaining significant correlation in this analysis would strengthen the authors' conclusion.

We thank the reviewer for bringing these points to our attention. Because the freezing levels of all mice were comparable during the beginning of extinction, we used the absolute value of freezing during L-Ext in Figure 2G. However, we agree with the reviewer that it is better to use the change in freezing as a measure of extinction learning. We have therefore performed the correlation again, this time between the change in freezing (from E-Ext to L-Ext) and the average dF/F, and found that these two variables are significantly correlated as well. This new result is now presented in the seventh paragraph of the subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning” and in Figure 3H of the revised manuscript. The second point that the reviewer raises is a very important issue. It is indeed possible that the variation in the photometry signal can be due to the variation in the recording location rather than reflecting the relationship with extinction learning. To rule this possibility out, we examined the correlation between the change in freezing (E-Ext to L-Ext) and reward responses, as suggested by the reviewer, and did not find a significant relationship between these two variables. This suggests that the variation in the photometry signal we observed during E-Ext was unlikely due to differences in the recording location. We now describe this new result in the aforementioned paragraph and in Figure 3I of the revised manuscript. These results therefore suggest that the correlation between the magnitude of the photometry signal during E-Ext and the level of extinction learning were not due to differences in recording location.

4) Please cite and discuss Luo et al., 2018 and Mayer et al., 2018.

Because Luo et al., 2018, was published after our first submission, this study was not cited in our original manuscript. We have now cited and discussed this study and Mayer et al., 2018, in the Discussion in the revised manuscript. However, it is important to note that Mayer et al., 2018 study investigated relief and safety learning mechanisms using backward conditioning and unpaired conditioning paradigms. In backward conditioning, the order of CS and US presentations is switched such that the CS is delivered at the termination of the aversive US. For studying safety learning, the authors used an explicitly unpaired conditioning protocol in which CS and US presentations were separated in time during the conditioning session. Although these paradigms modeling relief and safety learning might be conceptually similar to fear extinction, procedurally they are quite different. We would like to emphasize that extinction is a form of learning that is initiated when an unexpected event occurs (omission of an expected aversive stimulus) and therefore we believe that the behavioral tasks used in Mayer et al., 2018, are very different than fear extinction.

In addition, we have now also cited and discussed four recent relevant studies (Felsenberg et al., 2018; Groessl et al., 2018; Jo et al., 2018; Menegas et al., 2018) in the subsection “Dopamine Neurons Signal the Unexpected Omission of the Aversive US during Fear Extinction Learning” and in the Discussion.

Reviewer #3:

Combining sophisticated electrophysiological, photometric and optogenetic techniques the authors studied the question of how dopaminergic neurons are involved in extinction learning. In their experiments they were able to demonstrate the importance of activity of the midbrain ventral tegmental area (VTA) dopamine system in the initiation and facilitation of cued fear extinction. Activation of dopaminergic neurons is observed when an expected aversive event does not occur, which is a crucial phase in the onset of extinction learning. The study is exciting and in large parts novel. It also provides further information on the neuronal basis of findings of a very recently published study by Luo et al. reporting a role for the VTA-nucleus accumbens dopamine system in extinction learning. Methods are state-of-the art, experiments were carried out with care and the manuscript is straightforward to read. However, there are points that need to be addressed.

1) As mentioned above, in a paper published a few weeks ago Luo and co-workers demonstrate that the optogenetic inactivation of VTA-dopamine cells during the US omission period reduced extinction learning in rats. Here, part of this finding is replicated in another species and extended as, for example: a) the optogenetic stimulation of VTA dopaminergic activity facilitated fear extinction in mice (Figure 4); b) (Re)extinction is possible in the paired-NpHR mice during extinction recall when the laser is switched off (Figure 3). This finding may also suggests that silencing of the VTA-dopamine activity did not reconsolidate the fear memory. It is necessary to cite the Luo paper and emphasize the overlap and differences (some of which mentioned above) with the present manuscript.

We are grateful for the reviewer’s careful and detailed reading of our study. Because Luo et al., 2018, was published after submission of our manuscript, this study was not cited in our original manuscript. It is now cited and discussed in detail in the revised version (Discussion).

We thank the reviewer for bringing the issues regarding re-extinction and reconsolidation to our attention and regret that they were not discussed sufficiently in the original manuscript. The reviewer is correct to point out that re-extinction was possible in the paired-NpHR group during extinction recall in the absence of optogenetic manipulation. To address this more directly, we have now compared the extinction learning of paired-NpHR group during extinction recall (day 3) to that of the control groups during extinction learning (day 2). This revealed that there was no significant difference in the extinction rate of these groups. This result suggests that: 1) Paired-NpHR group is capable of undergoing normal extinction learning in the absence of optogenetic manipulation the next day indicating that our manipulation did not have a nonspecific long-term effect, 2) our optogenetic manipulation likely did not have an effect on the strength of the fear memory, as suggested by the reviewer. These results are now presented and discussed in the last paragraph of the subsection “Inhibition of Dopamine Neuron Firing at the Time of the US Omission Impairs Fear Extinction Learning”.

Furthermore, we found that enhancing DA neuron firing at the time of the US omission accelerated extinction learning and enhanced extinction memory. However, the low level of freezing in Paired-ChR2 mice during extinction recall test could also be due to an effect on the fear memory itself. For instance, excitation of DA neurons could have resulted in the erasure of the fear memory by impairing the memory reconsolidation process (Nader, 2015). To rule this possibility out, we did perform a fear renewal test in these mice but didn’t include these results in the original manuscript. These results are now described in the last paragraph of the subsection “Enhancing Dopamine Neuron Firing at the Time of the US Omission Accelerates Fear Extinction Learning” and in Figure 6—figure supplement 3. We show that all three groups of mice exhibited comparable levels of fear renewal suggesting that the low level of freezing in Paired-ChR2 mice during extinction recall cannot be due to impaired memory reconsolidation, but is rather due to enhanced extinction. We believe that these new results strengthen our study and we thank the reviewer for his constructive comments and suggestions.

2) Please provide evidence that optical silencing and stimulation using NpHR and ChR2, respectively, is a valid method to modulate dopaminergic activity in the VTA of mice.

The reviewer raises an important issue. It is indeed essential to provide evidence that optical stimulation of NpHR and ChR2 are valid methods to modulate DA neuron firing in awake mice. To address this issue, we have performed single unit recordings of DA neurons in the VTA of awake DAT-cre mice expressing NpHR or ChR2 during optical stimulation. These results are now presented in Figure 5—figure supplement 2 and Figure 6—figure supplement 2, respectively. These new results confirm that optical activation of NpHR and ChR2 can inhibit and excite DA neuron firing in awake mice. These recordings were performed in awake head-fixed mice acutely using opto-probes.

3) The dopamine system has been suggested to be involved in both the learning and consolidation of fear extinction. The authors nicely demonstrate its role in the initiation of fear extinction learning, but unfortunately do not further discuss this finding in relation to published work. For example, it has been shown that administration of the dopaminergic drug L-DOPA before extinction training was able to initiate fear extinction learning in a mouse model of impaired fear extinction. This discussion needs to be added. Also the consolidation aspect is not addressed sufficiently (one mentioning that "extinction memory was strengthened"). Furthermore, the authors focus on the better-than-expected outcome reward aspect of dopamine in fear extinction in the Discussion. There is increasing evidence that dopamine activity/release may also reflect a prediction error signal that guides extinction learning. Whether and how the present data support this latter idea, is missing and should be added.

We thank the reviewer for bringing to our attention the studies on the role of L-DOPA during fear extinction and regret that they were not cited and discussed in the original manuscript. We have now discussed these findings in the Discussion section. Furthermore, because the focus of our study is on fear extinction learning, we did not discuss the memory consolidation aspect in detail in the original manuscript. However, we agree that this is an important aspect and have now included this discussion and cited the relevant literature on the role of dopamine signaling in mediating the plasticity in the neural circuit underlying fear extinction memories in the Discussion.

The reviewer is correct to point out that we did not emphasize the prediction error aspect of our findings sufficiently in the original manuscript. We agree with the reviewer that the dopamine neuron activity likely reflects a prediction error signal that drives extinction learning. Indeed, our electrophysiology and fiber photometry experiments suggest that the DA signal at the time of the US omission has the properties of a prediction error signal: it was large at the beginning of extinction when the US omission was most unexpected, and decreased significantly during the late trials of extinction when the US omission was no longer unexpected and animals showed significant extinction learning. However, this aspect of our results was only briefly discussed in the original manuscript and we have now emphasized it more in the revised version. We now clearly define prediction error signaling in the Introduction and have also extended our discussion of our results in light of prediction error signaling on in the Introduction and Discussion, as suggested by the reviewer. We believe that discussion of the prediction error and consolidation aspects of our findings in more detail strengthen our study and we thank the reviewer for his constructive comments.

References:

Bouton ME, Woods AM, Todd TP. (2014) Separation of time-based and trial-based accounts of the partial reinforcement extinction effect. Behav Processes. 101, 23-31.

Cui G, Jun SB, Jin X, Pham MD, Vogel SS, Lovinger DM, Costa RM. (2013) Concurrent activation of striatal direct and indirect pathways during action initiation. Nature. 494, 238-42

Grady AK, Bowen KH, Hyde AT, Totsch SK, Knight DC. (2016) Effect of continuous and partial reinforcement on the acquisition and extinction of human conditioned fear. Behav Neurosci. 130, 36-43.

Gunaydin LA, Grosenick L, Finkelstein JC, Kauvar IV, Fenno LE, Adhikari A, Lammel S, Mirzabekov JJ, Airan RD, Zalocusky KA, Tye KM, Anikeeva P, Malenka RC, Deisseroth K. (2014) Natural neural projection dynamics underlying social behavior. Cell. 157, 1535-51.

Howe MW, Dombeck DA. (2016) Rapid signalling in distinct dopaminergic axons during locomotion and reward. Nature. 535, 505-510.

Johansen JP, Cain CK, Ostroff LE, LeDoux JE. (2011) Molecular mechanisms of fear learning and memory. Cell. 147, 509-524.

Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature. 491, 212-7.

Matthews GA, Nieh EH, Vander Weele CM, Halbert SA, Pradhan RV, Yosafat AS, Glober GF, Izadmehr EM, Thomas RE, Lacy GD, Wildes CP, Ungless MA, Tye KM (2016) Dorsal Raphe Dopamine Neurons Represent the Experience of Social Isolation. Cell 164, 617-631.

McGaugh JL. (2000) Memory--a century of consolidation. Science. 287, 248-251.

Skinner, B. F. (1938). The Behavior of Organisms. New York: Appleton-Century-Crofts.

Tye KM, Mirzabekov JJ, Warden MR, Ferenczi EA, Tsai HC, Finkelstein J, Kim SY, Adhikari A, Thompson KR, Andalman AS, Gunaydin LA, Witten IB, Deisseroth K. (2013) Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature. 493, 537-541.

https://doi.org/10.7554/eLife.38818.024

Article and author information

Author details

  1. Ximena I Salinas-Hernández

    Institute of Neurophysiology, Neuroscience Center, Goethe University, Frankfurt, Germany
    Contribution
    Data curation, Formal analysis, Investigation, Methodology, Writing—review and editing
    Competing interests
    No competing interests declared
  2. Pascal Vogel

    Institute of Neurophysiology, Neuroscience Center, Goethe University, Frankfurt, Germany
    Contribution
    Data curation, Software, Formal analysis, Methodology
    Competing interests
    No competing interests declared
  3. Sebastian Betz

    Institute of Neurophysiology, Neuroscience Center, Goethe University, Frankfurt, Germany
    Contribution
    Software, Methodology
    Competing interests
    No competing interests declared
  4. Raffael Kalisch

    1. Deutsches Resilienz Zentrum, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
    2. Neuroimaging Center, Focus Program Translational Neuroscience, Johannes Gutenberg University, Mainz, Germany
    Contribution
    Conceptualization, Writing—review and editing
    Competing interests
    No competing interests declared
  5. Torfi Sigurdsson

    Institute of Neurophysiology, Neuroscience Center, Goethe University, Frankfurt, Germany
    Contribution
    Resources, Software, Formal analysis, Funding acquisition, Methodology, Writing—original draft, Writing—review and editing
    Competing interests
    No competing interests declared
  6. Sevil Duvarci

    Institute of Neurophysiology, Neuroscience Center, Goethe University, Frankfurt, Germany
    Contribution
    Conceptualization, Resources, Data curation, Software, Formal analysis, Supervision, Funding acquisition, Investigation, Methodology, Writing—original draft, Project administration, Writing—review and editing
    For correspondence
    duvarci@med.uni-frankfurt.de
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-4494-7741

Funding

Deutsche Forschungsgemeinschaft (SFB1193 Grant INST247/852-1)

  • Sevil Duvarci

Deutsche Forschungsgemeinschaft (SFB1193 Grant INST247/854-1)

  • Torfi Sigurdsson

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We would like to thank Jochen Roeper for helpful discussions; and Beatrice Fischer, Jasmine Sonntag and Thomas Wulf for technical assistance. We thank René Hen for generating, and Jochen Roeper and Institute of Neurophysiology for supporting the breeding of and providing DAT-cre mice. This work was supported by the Deutsche Forschungsgemeinschaft (DFG SFB1193 Grant INST247/852-1 to SD and INST247/854-1 to TS).

Ethics

Animal experimentation: All procedures were conducted in accordance with the guidelines of the German Animal Protection Act and were approved by the local authorities (Regierungsprasidium Darmstadt; protocol number 1038). Every effort was made to minimize suffering.

Senior Editor

  1. Catherine Dulac, Harvard University, United States

Reviewing Editor

  1. Rui M Costa, Columbia University in the City of New York, United States

Publication history

  1. Received: June 5, 2018
  2. Accepted: November 8, 2018
  3. Accepted Manuscript published: November 13, 2018 (version 1)
  4. Version of Record published: November 27, 2018 (version 2)

Copyright

© 2018, Salinas-Hernández et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 3,120
    Page views
  • 564
    Downloads
  • 7
    Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)