To help someone learn a new task, we might give them a reward after they have performed well. However, these rewards tend to be given several seconds or minutes after the behavior they are supposed to promote. Therefore, it is unclear how the rewards affect the brain and help accelerate the learning process.

Information is processed and sent around the brain by networks of cells called neurons. These networks are constantly remodeled because learning changes the connections—called synapses—that neighboring neurons signal across. Synapses can be strengthened so that signals are sent across them more easily in the future. Synapses can also be weakened, making it harder for the neurons to subsequently communicate.

A chemical called dopamine is often produced in the brain when a reward is received. If dopamine is present in a synapse whilst a neuron is signaling to its neighbor, it can affect how effectively this communication occurs. Brzosko et al. have now investigated whether dopamine can also change the synapses if it is applied after signaling has already happened.

The strengthening or weakening of synapses can be triggered by electrically stimulating the neurons on either side of a synapse at particular times. Brzosko et al. did this to neurons in slices of mouse brain, and then applied dopamine to the neurons. The results suggest that dopamine can reverse synaptic weakening and can even cause the synapses to strengthen. However, the dopamine had to be applied immediately after stimulation to be able to strengthen the synapse. The next challenge is to establish whether this change in synaptic strength is responsible for the change in behavior.

Spike timing-dependent plasticity (STDP) is a physiologically relevant form of Hebbian learning (Caporale and Dan, 2008). In its classic form, STDP depends on the order and precise timing of presynaptic and postsynaptic spikes: pre-before-post spike pairings induce timing-dependent long-term potentiation (t-LTP), whereas post-before-pre pairings induce timing-dependent long-term depression (t-LTD) (Markram et al., 1997; Bi and Poo, 1998). However, the quantitative rules of STDP are profoundly influenced by neuromodulators (Seol et al., 2007; Pawlak et al., 2010), including dopamine (DA) (Zhang et al., 2009; Edelmann and Lessmann, 2011; Yang and Dani, 2014). Although it is well established that reward-motivated behavior depends on the activity of DA neurons (Schultz et al., 1997; Suri and Schultz, 1999; Pan et al., 2005), the mechanisms that associate specific experiences with rewarding outcomes, which typically occur after a delay, are not well understood. This is referred to as the distal reward problem (Hull, 1943). To address this problem, here, we examined whether DA modulates STDP not only when applied during, but also—more importantly—when applied after the pairing event.

We first sought to corroborate the shape of the STDP induction curve by varying the time interval between the presynaptic and postsynaptic activity (Δt; Figure 1A,B,G). To this end, we monitored excitatory postsynaptic potentials (EPSPs) that were evoked by extracellular stimulation of the Schaffer-collateral-CA1 pathway during whole-cell recordings of CA1 pyramidal cells in mouse horizontal slices (postnatal days 12–18; ‘Materials and methods’). Plasticity was induced in current clamp mode using an induction protocol that involved 100 pairings of a single EPSP followed by a single postsynaptic spike (t-LTP; Figure 1A) or a single postsynaptic spike followed by a single EPSP (t-LTD; Figure 1B) at 0.2 Hz. Consistent with previous studies (Bi and Poo, 1998; Zhang et al., 2009; Edelmann and Lessmann, 2011), the pre-before-post pairing protocol with Δt = +10 ms induced t-LTP (182 ± 14%; t(4) = 5.6, p = 0.0049 vs 100%, n = 5; Figure 1A) and the post-before-pre pairing protocol with Δt = −20 ms induced t-LTD (61 ± 9%; t(4) = 4.4, p = 0.0121 vs 100%, n = 5; Figure 1B). Surprisingly, however, we found that the post-before-pre pairing protocol with Δt = −10 ms instead of inducing t-LTD, elicited robust t-LTP (202 ± 21%; t(6) = 5.0, p = 0.0025 vs 100%, n = 7; Figure 1C,D,G). This conflicts with previous reports from hippocampal cultures (Bi and Poo, 1998; Zhang et al., 2009) and acute slices (Edelmann and Lessmann, 2011; Yang and Dani, 2014), where post-before-pre pairing protocols never elicited synaptic potentiation in baseline conditions. Given that DA has been found to widen the time window for the induction of t-LTP (Zhang et al., 2009; Yang and Dani, 2014), we wanted to assess whether endogenous DA could be responsible for the potentiation observed with the post-before-pre pairing under our experimental conditions. Therefore, we repeated this set of experiments using the post-before-pre pairing protocol with Δt = −10 ms in the presence of DA receptor (DAR) antagonists. Indeed, combined application of the D1/D5 receptor antagonist SCH23390 (10 μM) and D2-like receptor antagonist sulpiride (50 μM) from the start of the recordings prevented t-LTP and enabled t-LTD instead (72 ± 8%; t(5) = 3.6, p = 0.0160 vs 100%, n = 6; Figure 1C,D,G), rendering the STDP induction curve similar to that observed in hippocampal cultures (Bi and Poo, 1998; Zhang et al., 2009). These results suggest a modulatory action of endogenous DA, presumably released during the pairing protocol (Frey et al., 1990; Yang and Dani, 2014), which resulted in the changed polarity of plasticity at narrow negative spike-timing intervals.

Figure 1

Download asset Open asset

Figure 1—source data 1

Source data for Figure 1.

https://doi.org/10.7554/eLife.09685.004

Download elife-09685-fig1-data1-v1.xlsx

Next, we wanted to examine whether the application of exogenous DA during pairing at negative spike-timing intervals facilitates synaptic potentiation. Indeed, the post-before-pre pairing protocol with Δt = −20 ms, which elicited robust t-LTD in control condition (74 ± 9%, t(8) = 3.0, p = 0.0165 vs 100%, n = 9; Figure 1E–G), induced significant t-LTP when exogenous DA (20 μM) was bath-applied for 10–12 min from 2 min before and during the post-before-pre pairings in interleaved experiments (144 ± 12%; t(5) = 3.7, p = 0.0148 vs 100%, n = 6; Figure 1E–G). Therefore, in accordance with previous findings (Zhang et al., 2009; Yang and Dani, 2014), the presence of DA during the coordinated spiking activity widens the spike time interval for induction of t-LTP.

A crucial aspect of reinforcement learning models is the ability of the reinforcing signal (DA) to strengthen active synapses, even when it arrives after the activity (Sutton and Barto, 1981; Izhikevich, 2007). To test this hypothesis experimentally, we applied DA after the t-LTD induction protocol. Exogenous DA (100 μM) added to the perfusion system for 10–12 min starting within 1 min after the post-before-pre pairing protocol with Δt = −20 ms converted t-LTD into t-LTP (169 ± 16%, t(5) = 4.3, p = 0.0078 vs 100%, n = 6; Figure 2A,B). This implies that DA can have a retroactive effect allowing negative spike pairings to induce t-LTP. The specificity of this DAergic conversion of STDP was assessed using DAR antagonists. Indeed, combined application of DAR antagonists, SCH23390 (10 μM) and sulpiride (50 μM), prevented DA-induced conversion of t-LTD into t-LTP, resulting in significant t-LTD instead (63 ± 12%, t(5) = 3.0, p = 0.0289 vs 100%, n = 6, Δt = −20 ms). Thus, the conversion of t-LTD into t-LTP was due to specific DAR activation. Importantly, when the test pathway was not stimulated following the pairing protocol until after DA washout (stimulation resumed 15 min after pairing), robust t-LTD was induced (61 ± 5%, t(5) = 8.1, p = 0.0005 vs 100%, n = 6; Figure 2A,B). The effect of DA was, therefore, activity dependent, demonstrating that the reinforcing signal is capable of acting specifically on the active inputs. To exclude the possibility that DA by itself could potentiate the test pathway, control experiments with ongoing synaptic stimulation over 60 min at 0.2 Hz, but without pairing with postsynaptic action potentials, were performed. Consistent with earlier reports (Otmakhova and Lisman, 1999), DA had no significant effect on the basal Schaffer-collateral transmission (110 ± 7%, t(7) = 1.4, p = 0.2169 vs 100%, n = 8; Figure 2A,B).

Figure 2

Download asset Open asset

Figure 2—source data 1

Source data for Figure 2.

https://doi.org/10.7554/eLife.09685.006

Download elife-09685-fig2-data1-v1.xlsx

Subsequently, we wanted to determine whether the observed DA-induced conversion of t-LTD into t-LTP depends on the timing of DA application following the post-before-pre protocol (Figure 3A–D). We found that delayed application of DA (10 or 30 min after t-LTD pairing protocol) failed to convert t-LTD into t-LTP. Application of DA 10 min after the post-before-pre pairing caused a reversal of t-LTD back to baseline (94 ± 9%, t(11) = 0.7, p = 0.5230 vs 100%, n = 12; Figure 3B,D), whereas application of DA 30 min after the post-before-pre pairing failed to influence t-LTD altogether (59 ± 12%, t(5) = 3.4, p = 0.0200 vs 100%, n = 6; Figure 3C,D). Whilst it has previously been reported that DA applied after the induction of low frequency stimulation-induced LTD can reduce the magnitude of synaptic depression (Mockett et al., 2007), our data demonstrate, for the first time to our knowledge, that DA can change the polarity of STDP when acting within a short time window following the induction protocol.

Figure 3

Download asset Open asset

Figure 3—source data 1

Source data for Figure 3.

https://doi.org/10.7554/eLife.09685.008

Download elife-09685-fig3-data1-v1.xlsx

Finally, we aimed to explore the possible mechanisms underlying the DA-induced conversion of t-LTD into t-LTP. Both hippocampal t-LTP and t-LTD (Edelmann and Lessmann, 2011; Yang and Dani, 2014), as well as the modulation of STDP by DA (Zhang et al., 2009), require functional NMDA receptors. We, therefore, asked whether the DA-induced conversion of t-LTD into t-LTP is also NMDA receptor dependent. Application of the NMDA receptor antagonist d-2-amino-5-phosphonopentanoic acid (d-AP5, 50 μM) after the post-before-pre pairing protocol did not by itself affect the development of t-LTD (57 ± 12%, t(5) = 3.6, p = 0.0156 vs 100%, n = 6, Δt = −20 ms; Figure 4A,D). Nevertheless, DA in the presence of d-AP5 reversed t-LTD back to baseline albeit failing to convert t-LTD into t-LTP (105 ± 12%, t(5) = 0.4, p = 0.6898 vs 100%, n = 6, Δt = −20 ms; Figure 4A,D). This suggests an important dissociation between two mechanisms involved in the DA-induced conversion of t-LTD into t-LTD, namely a reversal of synaptic depression (de-depression) and synaptic potentiation, one of which is NMDA receptor dependent.

Figure 4 with 2 supplements see all

Download asset Open asset

Figure 4—source data 1

Source data for Figure 4.

https://doi.org/10.7554/eLife.09685.010

Download elife-09685-fig4-data1-v1.xlsx

Figure 4—source data 2

Source data for Figure 4—figure supplement 1.

https://doi.org/10.7554/eLife.09685.011

Download elife-09685-fig4-data2-v1.xlsx

Figure 4—source data 3

Source data for Figure 4—figure supplement 2.

https://doi.org/10.7554/eLife.09685.012

Download elife-09685-fig4-data3-v1.xlsx

To investigate the intracellular signaling mechanisms involved, we initially set out to establish the DAR subtype associated with the DA-induced conversion of t-LTD into t-LTP. Even though combined application of D1-like and D2-like receptor antagonist completely blocked the DA effect (Figure 4—figure supplement 1A,D), application of either D1-like or D2-like receptor antagonist alone only partially prevented the conversion of t-LTD into t-LTP (SCH 23390: 131 ± 16%; t(6) = 1.4, p = 0.0113 vs DA (t-LTP); t(6) = 3.6, p = 0.2277 vs control (t-LTD); t(6) = 1.9, p = 0.0994 vs 100%; n = 7. Sulpiride: 94 ± 13%; t(6) = 3.2, p = 0.0228 vs DA (t-LTP); t(6) = 1.5, p = 0.1932 vs control (t-LTD); t(6) = 0.5, p = 0.6435 vs 100%; n = 7; Figure 4—figure supplement 1B–D). This suggests that both receptor subtypes might mediate the retroactive effect of DA on STDP. Given the astounding complexity of D2-like receptor pharmacology (Neve et al., 2004), we first wanted to evaluate the possible involvement of D1-like receptor-activated cAMP/PKA signaling cascade in the DA-induced conversion of t-LTD into t-LTP. D1/D5 receptor stimulation leads to the activation of adenylyl cyclase (AC) and subsequent increase in cyclic adenosine monophosphate (cAMP) and protein kinase A (PKA) activation (Greengard et al., 1999; Neve et al., 2004). We found that the general AC activator, forskolin (50 μM), applied for 10–12 min immediately after the pairing protocol with Δt = −20 ms resulted in robust conversion of t-LTD into t-LTP (167 ± 17%, t(6) = 3.9, p = 0.0078 vs 100%, n = 7; Figure 4B,D). Notably, forskolin (50 μM) applied in the absence of the pairing protocol had no significant effect on baseline EPSPs (83 ± 12%, t(6) = 1.4, p = 0.2046 vs 100%, n = 7; Figure 4—figure supplement 2). The forskolin-induced conversion of t-LTD into t-LTP was also NMDA receptor dependent since bath application of d-AP5 (50 μM) 1 min before forskolin treatment prevented synaptic potentiation from developing, although synaptic depression was reversed (102 ± 11%, t(5) = 0.2, p = 0.8616 vs 100%, n = 6; Figure 4B,D). This result suggests that the cAMP cascade works either upstream of or in parallel to NMDA receptor activation for the development of potentiation to occur. Downstream of cAMP, PKA is involved because the PKA inhibitor, H-89 (20 µM), blocked the DA-induced conversion of t-LTD into t-LTP, revealing significant t-LTD (76 ± 5%, t(6) = 5.0, p = 0.0025 vs 100%, n = 7; Figure 4C,D). Taken together, these results imply that the DA-induced conversion of t-LTD into t-LTP involves activation of the cAMP/PKA signaling cascade, which closely mimics the effects of DA (Figure 4A vs Figure 4B). Although, D2-like receptors are typically associated with the inhibition of AC (Neve et al., 2004), interestingly, there is also evidence that D2-like receptor stimulation can potentiate AC activity (Glass and Felder, 1997; Watts and Neve, 1997). Therefore, while the possibility that D2-like receptors contribute to the conversion of t-LTD into t-LTP via a different signaling cascade cannot be excluded, it is tempting to suggest that the DA-induced conversion of t-LTD into t-LTP is mediated primarily via the cAMP/PKA pathway. Hence, based on our results, we propose that the stimulation of postsynaptic (Figure 5ai) or presynaptic (Figure 5aii) DARs activates the cAMP/PKA pathway, which—via two cellular mechanisms (de-depression and potentiation)—leads to the retroactive conversion of t-LTD into t-LTP.

Figure 5

Download asset Open asset

The functional implications of our finding depend on the presynaptic source of DA in the hippocampus, which remains controversial because of the apparent discrepancy between DAergic terminals and receptors (Scatton et al., 1980; Gasbarri et al., 1997; Lisman and Grace, 2005). On one hand, it has been argued that noradrenergic terminals from the locus coeruleus, which mediates arousal and the optimization of behavioral performance (Aston-Jones and Cohen, 2005; Chamberlain and Robbins, 2013), may provide a major source of DA release (Smith and Greene, 2012). On the other hand, hippocampal pyramidal cell assemblies are directly affected by concurrent activity in midbrain DAergic neurons (McNamara et al., 2014), which have been linked to reward-seeking behavior (Schultz et al., 1997) and appetitive stimuli (Mirenowicz and Schultz, 1996; Fiorillo et al., 2013). This latter finding (McNamara et al., 2014) not only supports the hypothesis that hippocampal DA is relevant for reward processing, but is also consistent with our result that the activation of DAergic receptors during coordinated spiking activity changes the functional outcome of STDP (Figure 1C–G).

Previous studies examining reinforcement learning at the level of synaptic plasticity have showed that neuromodulators can affect timing-dependent plasticity in locust (Cassenaer and Laurent, 2012) and spine structural plasticity in striatal medium spiny neurons in mice (Yagishita et al., 2014) when acting within a delay time window of 1 s. While this narrow temporal detection window may be important in the striatum, it cannot account for the experimental evidence from behavioral studies of response acquisition with an extended reinforcement delay in rats and pigeons (Lattal and Gleeson, 1990; Sutphin et al., 1998), rhesus monkeys (Galuska and Woods, 2005), and humans (Okouchi, 2009). Meanwhile, our finding demonstrates that DA can modulate STDP in the CA1 with a reinforcement delay of at least 1 min (Figure 5B). Such extended reinforcement delay is likely to be particularly important in hippocampus-dependent learning during spatial exploration.

In conclusion, our work demonstrates a retroactive effect of DA on STDP—converting t-LTD into t-LTP. This effect is mediated at least in part through the activation of the cAMP/PKA cascade and requires the activation of synaptic NMDA receptors. This in turn suggests that the conversion can only occur at synapses that are re-activated following the initial pairing event. Interestingly, it has been reported that hippocampal reactivation events (sharp wave ripples) increase in frequency following reward (Singer and Frank, 2009; Atherton et al., 2015). Thus, in behaving animals, the conditions for the conversion of depression into potentiation might occur during reward-related sharp wave ripple activity. Together, these findings support the concept of a slowly decaying synaptic eligibility trace that is committed to memory by the occurrence of reward and provide a possible mechanism for associating specific experiences with behaviorally distant, rewarding outcomes in animals (Sutton and Barto, 1981; Suri and Schultz, 1999; Pan et al., 2005; Izhikevich, 2007; Harnett et al., 2009), including humans (Dunsmoor et al., 2015).

1. 1. Aston-Jones G
   2. Cohen JD

   (2005) An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance

   Annual Review of Neuroscience 28:403–450.

   https://doi.org/10.1146/annurev.neuro.28.061604.135709

   • Google Scholar
2. 1. Atherton LA
   2. Dupret D
   3. Mellor JR

   (2015) Memory trace replay: the shaping of memory consolidation by neuromodulation

   Trends in Neurosciences 38:560–570.

   https://doi.org/10.1016/j.tins.2015.07.004

   • Google Scholar
3. 1. Bi GQ
   2. Poo MM

   (1998)

   Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type

   The Journal of Neuroscience 18:10464–10472.

   • Google Scholar
4. 1. Bolshakov VY
   2. Siegelbaum SA

   (1994) Postsynaptic induction and presynaptic expression of hippocampal long-term depression

   Science 264:1148–1152.

   https://doi.org/10.1126/science.7909958

   • Google Scholar
5. 1. Caporale N
   2. Dan Y

   (2008) Spike timing-dependent plasticity: a Hebbian learning rule

   Annual Review of Neuroscience 31:25–46.

   https://doi.org/10.1146/annurev.neuro.31.060407.125639

   • Google Scholar
6. 1. Cassenaer S
   2. Laurent G

   (2012) Conditional modulation of spike-timing-dependent plasticity for olfactory learning

   Nature 482:47–52.

   https://doi.org/10.1038/nature10776

   • Google Scholar
7. 1. Chamberlain SR
   2. Robbins TW

   (2013) Noradrenergic modulation of cognition: therapeutic implications

   Journal of Psychopharmacology / British Association for Psychopharmacology 27:694–718.

   https://doi.org/10.1177/0269881113480988

   • Google Scholar
8. 1. Charton JP
   2. Herkert M
   3. Becker CM
   4. Schroder H

   (1999) Cellular and subcellular localization of the 2B-subunit of the NMDA receptor in the adult rat telencephalon

   Brain Research 816:609–617.

   https://doi.org/10.1016/S0006-8993(98)01243-8

   • Google Scholar
9. 1. Chen BS
   2. Roche KW

   (2007) Regulation of NMDA receptors by phosphorylation

   Neuropharmacology 53:362–368.

   https://doi.org/10.1016/j.neuropharm.2007.05.018

   • Google Scholar
10. 1. Dunsmoor JE
    2. Murty VP
    3. Davachi L
    4. Phelps EA

    (2015) Emotional learning selectively and retroactively strengthens memories for related events

    Nature 520:345–348.

    https://doi.org/10.1038/nature14106

    • Google Scholar
11. 1. Edelmann E
    2. Lessmann V

    (2011) Dopamine modulates spike timing-dependent plasticity and action potential properties in CA1 pyramidal neurons of acute rat hippocampal slices

    Frontiers in Synaptic Neuroscience 3:6.

    https://doi.org/10.3389/fnsyn.2011.00006

    • Google Scholar
12. 1. Fiorillo CD
    2. Song MR
    3. Yun SR

    (2013) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli

    The Journal of Neuroscience 33:4710–4725.

    https://doi.org/10.1523/JNEUROSCI.3883-12.2013

    • Google Scholar
13. 1. Frey U
    2. Schroeder H
    3. Matthies H

    (1990) Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices

    Brain Research 522:69–75.

    https://doi.org/10.1016/0006-8993(90)91578-5

    • Google Scholar
14. 1. Galuska CM
    2. Woods JH

    (2005) Acquisition of cocaine self-administration with unsignaled delayed reinforcement in rhesus monkeys

    Journal of the Experimental Analysis of Behavior 84:269–280.

    https://doi.org/10.1901/jeab.2005.99-04

    • Google Scholar
15. 1. Gasbarri A
    2. Sulli A
    3. Packard MG

    (1997) The dopaminergic mesencephalic projections to the hippocampal formation in the rat

    Progress in Neuro-psychopharmacology & Biological Psychiatry 21:1–22.

    https://doi.org/10.1016/S0278-5846(96)00157-1

    • Google Scholar
16. 1. Glass M
    2. Felder CC

    (1997) Concurrent stimulation of cannabinoid CB1 and dopamine D2 receptors augments cAMP accumulation in striatal neurons: evidence for a Gs linkage to the CB1 receptor

    The Journal of Neuroscience 17:5327–5333.

    https://doi.org/10.1124/jpet.108.145425

    • Google Scholar
17. 1. Greengard P
    2. Allen PB
    3. Nairn AC

    (1999) Beyond the dopamine receptor: the DARPP-32/protein phosphatase-1 cascade

    Neuron 23:435–447.

    https://doi.org/10.1016/S0896-6273(00)80798-9

    • Google Scholar
18. 1. Harnett MT
    2. Bernier BE
    3. Ahn KC
    4. Morikawa H

    (2009) Burst-timing-dependent plasticity of NMDA receptor-mediated transmission in midbrain dopamine neurons

    Neuron 62:826–838.

    https://doi.org/10.1016/j.neuron.2009.05.011

    • Google Scholar
19. 1. Huber KM
    2. Kayser MS
    3. Bear MF

    (2000) Role for rapid dendritic protein synthesis in hippocampal mGluR-dependent long-term depression

    Science 288:1254–1256.

    https://doi.org/10.1126/science.288.5469.1254

    • Google Scholar
20. Book

    1. Hull CL

    (1943)

    Principles of behavior: an introduction to behavior theory

    Oxford, England: Appleton Century.

    • Google Scholar
21. 1. Ingebritsen TS
    2. Cohen P

    (1983) Protein phosphatases: properties and role in cellular regulation

    Science 221:331–338.

    https://doi.org/10.1126/science.6306765

    • Google Scholar
22. 1. Izhikevich EM

    (2007) Solving the distal reward problem through linkage of STDP and dopamine signalling

    Cerebral Cortex 17:2443–2452.

    https://doi.org/10.1093/cercor/bhl152

    • Google Scholar
23. 1. Jourdain P
    2. Bergersen LH
    3. Bhaukaurally K
    4. Bezzi P
    5. Santello M
    6. Domercq M
    7. Matute C
    8. Tonello F
    9. Gundersen V
    10. Volterra A

    (2007) Glutamate exocytosis from astrocytes controls synaptic strength

    Nature Neuroscience 10:331–339.

    https://doi.org/10.1038/nn1849

    • Google Scholar
24. 1. Kemp N
    2. Bashir ZI

    (1999) Induction of LTD in the adult hippocampus by the synaptic activation of AMPA/kainate and metabotropic glutamate receptors

    Neuropharmacology 38:495–504.

    https://doi.org/10.1016/S0028-3908(98)00222-6

    • Google Scholar
25. 1. Lattal KA
    2. Gleeson S

    (1990) Response acquisition with delayed reinforcement

    Journal of Experimental Psychology 16:27–39.

    https://doi.org/10.1037/0097-7403.16.1.27

    • Google Scholar
26. 1. Lee HK
    2. Barbarosie M
    3. Kameyama K
    4. Bear MF
    5. Huganir RL

    (2000) Regulation of distinct AMPA receptor phosphorylation sites during bidirectional synaptic plasticity

    Nature 405:955–959.

    https://doi.org/10.1038/35016089

    • Google Scholar
27. 1. Lisman JE
    2. Grace AA

    (2005) The hippocampal-VTA loop: controlling the entry of information into long- term memory

    Neuron 46:703–713.

    https://doi.org/10.1016/j.neuron.2005.05.002

    • Google Scholar
28. 1. Markram H
    2. Lubke J
    3. Frotscher M
    4. Sakmann B

    (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs

    Science 275:213–215.

    https://doi.org/10.1126/science.275.5297.213

    • Google Scholar
29. 1. McNamara CG
    2. Tejero-Cantero Á
    3. Trouche S
    4. Campo-Urriza N
    5. Dupret D

    (2014) Dopaminergic neurons promote hippocampal reactivation and spatial memory persistence

    Nature Neuroscience 17:1658–1660.

    https://doi.org/10.1038/nn.3843

    • Google Scholar
30. 1. Mirenowicz J
    2. Schultz W

    (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli

    Nature 379:449–451.

    https://doi.org/10.1038/379449a0

    • Google Scholar
31. 1. Mockett BG
    2. Guévremont D
    3. Williams JM
    4. Abraham WC

    (2007) Dopamine D1/D5 receptor activation reverses NMDA receptor-dependent long-term depression in rat hippocampus

    The Journal of Neuroscience 27:2918–2926.

    https://doi.org/10.1523/JNEUROSCI.0838-06.2007

    • Google Scholar
32. 1. Neve KA
    2. Seamans JK
    3. Trantham-Davidson H

    (2004) Dopamine receptor signalling

    Journal of Receptor and Signal Transduction Research 24:165–205.

    https://doi.org/10.1081/RRS-200029981

    • Google Scholar
33. 1. Okouchi H

    (2009) Response acquisition by humans with delayed reinforcement

    Journal of the Experimental Analysis of Behavior 91:377–390.

    https://doi.org/10.1901/jeab.2009.91-377

    • Google Scholar
34. 1. Oliet SHR
    2. Malenka RC
    3. Nicoll RA

    (1997) Two distinct forms of long-term depression coexist in CA1 hippocampal pyramidal cells

    Neuron 18:1294–1297.

    https://doi.org/10.1016/S0896-6273(00)80336-0

    • Google Scholar
35. 1. Otani S
    2. Connor JA

    (1998) Requirement of rapid Ca²⁺ entry and synaptic activation of metabotropic glutamate receptors for the induction of long-term depression in adult rat hippocampus

    The Journal of Physiology 511:761–770.

    https://doi.org/10.1111/j.1469-7793.1998.761bg.x

    • Google Scholar
36. 1. Otmakhova NA
    2. Lisman JE

    (1999)

    Dopamine selectively inhibits the direct cortical pathway to the CA1 hippocampal region

    The Journal of Neuroscience 19:1437–1445.

    • Google Scholar
37. 1. Pan W-X
    2. Schmidt R
    3. Wickens JR
    4. Hyland BI

    (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network

    The Journal of Neuroscience 25:6235–6242.

    https://doi.org/10.1523/JNEUROSCI.1478-05.2005

    • Google Scholar
38. 1. Pawlak V
    2. Wickens JR
    3. Kirkwood A
    4. Kerr JND

    (2010) Timing is not everything: neuromodulation opens the STDP gate

    Frontiers in Synaptic Neuroscience 2:146.

    https://doi.org/10.3389/fnsyn.2010.00146

    • Google Scholar
39. 1. Scatton B
    2. Simon H
    3. Le Moal M
    4. Bischoff S

    (1980) Origin of dopaminergic innervation of the rat hippocampal formation

    Neuroscience Letters 18:125–131.

    https://doi.org/10.1016/0304-3940(80)90314-6

    • Google Scholar
40. 1. Schultz W
    2. Dayan P
    3. Montague RR

    (1997) A neural substrate of prediction and reward

    Science 275:1593–1599.

    https://doi.org/10.1126/science.275.5306.1593

    • Google Scholar
41. 1. Seol GH
    2. Ziburkus J
    3. Huang S
    4. Song L
    5. Kim IT
    6. Takamiya K
    7. Huganir RL
    8. Lee HK
    9. Kirkwood A

    (2007) Neuromodulators control the polarity of spike-timing-dependent synaptic plasticity

    Neuron 55:919–929.

    https://doi.org/10.1016/j.neuron.2007.08.013

    • Google Scholar
42. 1. Siegel SJ
    2. Brose N
    3. Janssen WG
    4. Gasic GP
    5. Jahn R
    6. Heinemann SF
    7. Morrison JH

    (1994) Regional, cellular, and ultrastructural distribution of N-methyl-d-aspartate receptor subunit 1 in monkey hippocampus

    Proceedings of the National Academy of Sciences of USA 91:564–568.

    https://doi.org/10.1073/pnas.91.2.564

    • Google Scholar
43. 1. Singer AS
    2. Frank LM

    (2009) Rewarded outcomes enhance reactivation of experience in the hippocampus

    Neuron 64:910–921.

    https://doi.org/10.1016/j.neuron.2009.11.016

    • Google Scholar
44. 1. Smith CC
    2. Greene RW

    (2012) CNS dopamine transmission mediated by noradrenergic innervation

    The Journal of Neuroscience 32:6072–6080.

    https://doi.org/10.1523/JNEUROSCI.6486-11.2012

    • Google Scholar
45. 1. Suri R
    2. Schultz W

    (1999) A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task

    Neuroscience 91:871–890.

    https://doi.org/10.1016/S0306-4522(98)00697-6

    • Google Scholar
46. 1. Sutphin G
    2. Byrne T
    3. Poling A

    (1998) Response acquisition with delayed reinforcement: a comparison of two-lever procedures

    Journal of the Experimental Analysis of Behavior 69:17–28.

    https://doi.org/10.1901/jeab.1998.69-17

    • Google Scholar
47. 1. Sutton RS
    2. Barto AG

    (1981) Toward a modern theory of adaptive networks: expectation and prediction

    Psychological Review 88:135–170.

    https://doi.org/10.1037/0033-295X.88.2.135

    • Google Scholar
48. 1. Watabe AM
    2. Carlisle HJ
    3. O'Dell TJ

    (2002)

    Postsynaptic induction and presynaptic expression of group 1 mGluR-dependent LTD in the hippocampal CA1 region

    Journal of Neurophysiology 87:1395–1403.

    • Google Scholar
49. 1. Watts VJ
    2. Neve KA

    (1997) Activation of type II adenylate cyclase by D2 and D4 but not D3 dopamine receptors

    Molecular Pharmacology 52:181–186.

    https://doi.org/10.1124/mol.52.2.181

    • Google Scholar
50. 1. Westphal RS
    2. Tavalin SJ
    3. Lin JW
    4. Alto NM
    5. Fraser IDC
    6. Langeberg LK
    7. Sheng M
    8. Scott JD

    (1999) Regulation of NMDA receptors by an associated phosphatase-kinase signaling complex

    Science 285:93–96.

    https://doi.org/10.1126/science.285.5424.93

    • Google Scholar
51. 1. Yagishita S
    2. Hayashi-Tkagi A
    3. Ellis-Davies GCR
    4. Urakubo H
    5. Ishii S
    6. Kasai H

    (2014) A critical time window for dopamine actions on the structural plasticity of dendritic spines

    Science 345:1616–1620.

    https://doi.org/10.1126/science.1255514

    • Google Scholar
52. 1. Yang K
    2. Dani JA

    (2014) Dopamine D1 and D5 receptors modulate spike timing-dependent plasticity at medial perforant path to dentate granule cell synapses

    The Journal of Neuroscience 34:15888–15897.

    https://doi.org/10.1523/JNEUROSCI.2400-14.2014

    • Google Scholar
53. 1. Zhang JC
    2. Lau PM
    3. Bi GQ

    (2009) Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses

    Proceedings of the National Academy of Sciences of USA 106:13028–13033.

    https://doi.org/10.1073/pnas.0900546106

    • Google Scholar

Author details

1. Zuzanna Brzosko

   Department of Physiology, Development and Neuroscience, Physiological Laboratory, University of Cambridge, Cambridge, United Kingdom

   Contribution

   ZB, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   No competing interests declared.

2. Wolfram Schultz

   Department of Physiology, Development and Neuroscience, Physiological Laboratory, University of Cambridge, Cambridge, United Kingdom

   Contribution

   WS, Conception and design, Drafting or revising the article

   Competing interests

   WS: Reviewing editor, eLife.

3. Ole Paulsen

   Department of Physiology, Development and Neuroscience, Physiological Laboratory, University of Cambridge, Cambridge, United Kingdom

   Contribution

   OP, Conception and design, Drafting or revising the article

   For correspondence

   op210@cam.ac.uk

   Competing interests

   No competing interests declared.

   "This ORCID iD identifies the author of this article:" 0000-0002-2258-5455

• 3,543

  views

• 996

  downloads

• 103

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.