Abstract
The unexpected absence of danger constitutes a pleasurable event that is critical for the learning of safety. Accumulating evidence points to similarities between the processing of absent threat and the well-established reward prediction error (PE). However, clear-cut evidence for this analogy in humans is scarce. In line with recent animal data, we showed that the unexpected omission of (painful) electrical stimulation triggers activations within key regions of the reward and salience pathways and that these activations correlate with the pleasantness of the reported relief. Furthermore, by parametrically violating participants’ probability and intensity related expectations of the upcoming stimulation, we showed for the first time in humans that omission-related activations in the VTA/SN were stronger following omissions of more probable and intense stimulations, like a positive reward PE signal. Together, our findings provide additional support for an overlap in the neural processing of absent danger and rewards in humans.
Introduction
We experience pleasurable relief when an expected threat stays away1. This relief indicates that the outcome we experienced (“nothing”) was better than we expected it to be (“threat”). Such a mismatch between expectation and outcome is generally regarded as the trigger for new learning, and is typically formalized as the prediction error (PE) that determines how much there can be learned in any given situation2. Over the last two decades, the PE elicited by the absence of expected threat (threat omission PE) has received increasing scientific interest, because it is thought to play a central role in learning of safety. Impaired safety learning is one of the core features of clinical anxiety3. A better understanding of how the threat omission PE is processed in the brain may therefore be key to optimizing therapeutic efforts to boost safety learning. Yet, despite its theoretical and clinical importance, research on how the threat omission PE is computed in the brain is only emerging.
To date, the threat omission PE has mainly been studied using fear extinction paradigms that mimic safety learning by repeatedly confronting a human or animal with a threat predicting cue (conditional stimulus, CS; e.g. a tone) in the absence of a previously associated aversive event (unconditional stimulus, US; e.g., an electrical stimulation). These (primarily non-human) studies have revealed that there are striking similarities between the PE elicited by unexpected threat omission and the PE elicited by unexpected reward. In the context of reward, it is well-established that dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN) of the midbrain increase their firing rate to unexpected rewards (positive PE), suppress their firing for unexpected reward omissions (negative PE) and show no change in firing in response to completely predicted rewards, in line with a formalized PE4,5. Likewise, in fear extinction, dopaminergic neurons in the VTA phasically increase their firing rates to early (unexpected), but not late (expected) US omissions 6–10, which consequently triggers downstream dopamine release in the nucleus accumbens (NAc) shell6,9,10. Furthermore, optogenetically blocking (or enhancing) the firing rate of these dopaminergic VTA neurons during US omissions impairs (or facilitates) subsequent fear extinction learning6–8. Notably, such dopaminergic VTA/NAc responses to threat omissions have also been observed in other experimental tasks, such as conditioned inhibition11 and avoidance12,13, confirming that these neural activations match a more general threat omission PE-signal.
In humans, reward-like PE responses to threat omissions during extinction and avoidance have mainly been reported in projection regions of dopaminergic midbrain neurons, such as the ventral striatum (more specifically, NAc and ventral putamen) and prefrontal areas (the ventromedial prefrontal cortex, vmPFC)14–19. Activations in these regions correlate with computationally modeled PE signals14,15,17 and are modulated by pharmacological manipulation of dopamine receptors17 and by genetic mutations that are known to enhance striatal phasic dopamine release14. Furthermore, connectivity analyses revealed that NAc activations during US omissions in fear extinction were functionally coupled to VTA/SN activations, and that this connectivity was enhanced by the administration of the dopamine precursor L-dopa prior to extinction17. The emerging picture is that cortico-striatal activations to unexpected omissions of threat are triggered by dopaminergic inputs from the VTA/SN, just like a positive reward PE, and that these activations play a central role in different types of threat omission-induced learning such as fear extinction and avoidance learning20,21. Still, direct observations of threat omission-related VTA/SN responses are currently lacking in humans.
As mentioned above, unexpected omissions of threat not only trigger neural activations that resemble a reward PE, they are also accompanied by a pleasurable emotional experience: relief1. Because these feelings of relief coincide with the PE at threat omission, relief has been proposed to be an emotional correlate of the threat omission PE18,22. Indeed, emerging evidence has shown that subjective experiences of relief follow the same time-course as theoretical PE during fear extinction. Participants in fear extinction experiments report high levels of relief pleasantness during early US omissions (when the omission was unexpected and the theoretical PE was high) and decreasing relief pleasantness over later omissions (when the omission was expected and the theoretical PE was low)22,23. Accordingly, preliminary fMRI evidence has shown that the pleasantness of this relief is correlated to activations in the NAc at the time of threat omission18. In that sense, studying relief may offer important insights in the mechanism driving safety learning.
However, is a correlation with the theoretical PE over time sufficient for neural activations/relief to be classified as a PE-signal? In the context of reward, Caplin and colleagues proposed three necessary and sufficient criteria all PE-signals should comply to, independent of the exact operationalizations of expectancy and reward (the so-called axiomatic approach24,25; which has also been applied to aversive PE26–28). Specifically, the magnitude of a PE signal should: (1) be positively related to the magnitude of the reward (larger rewards trigger larger PEs); (2) be negatively related to likelihood of the reward (more probable rewards trigger smaller PEs); and (3) not differentiate between fully predicted outcomes of different magnitudes (if there is no error in prediction, there should be no difference in the PE signal).
The previously discussed fear conditioning and extinction studies have been invaluable for clarifying the role of the threat omission PE within a learning context14–17,22,23. However, these studies were not tailored to create the varying intensity and probability-related conditions that are required to systematically evaluate the threat omission PE in the light of the PE axioms. First, these only included one level of aversive outcome: the electrical stimulation was either delivered or omitted; but the intensity of the stimulation was never experimentally manipulated within the same task. As a result, the magnitude-related axiom could not be tested. Second, as safety learning progressively developed over the course of extinction learning, the most informative trials to evaluate the probability axiom (i.e. the trials with the largest PE) were restricted to the first few CS+ offsets of the extinction phase, and the exact number of these informative trials likely differed across participants as a result of individually varying learning rates. This limited the experimental control and necessary variability to systematically evaluate the probability axiom. Third, because CS-US contingencies changed over the course of the task (e.g. from acquisition to extinction), there was never complete certainty about whether the US would (not) follow. This precluded a direct comparison of fully predicted outcomes. Finally, within a learning context, it remains unclear whether brain responses to the threat omission are in fact responses to the violation of expectancy itself, or whether they are the result of subsequent expectancy updating.
Based on these reasons, we recently developed the Expectancy Violation Assessment (EVA) task29 in order to study threat omission responses outside of a learning context. By providing verbal instructions on the probability and intensity of an upcoming electrical stimulation, which are then violated by not delivering the stimulation, we showed that the experienced pleasantness of the omission-relief reflects the degree of fearful expectation violation, with omissions of more intense and more probable stimulations eliciting more pleasurable feelings of relief, much like a PE-signal.
Here, we applied the EVA-task in the MRI scanner to investigate brain responses to unexpected omissions of threat in greater detail, examine their similarity to reward PE-signals, and explore the link with subjective relief. Specifically, participants received trial-by-trial instructions about the probability (0%, 25%, 50%, 75% and 100%) and intensity (weak, moderate, strong) of a potentially painful upcoming electrical stimulation, time-locked by a countdown clock (see Fig.1A). While stimulations were always delivered on 100% trials and never on 0% trials, most of the other trials (25%-75%) did not contain the expected stimulation and hence provoked an omission PE. We expected that (1) expected-but-omitted stimulation would trigger increased activity within key areas of the reward circuit (such as the VTA/SN, NAc, left ventral Putamen and vmPFC); that (2) this omission-related activity would fit the three criteria of a positive reward PE, and that (3) this activity would be related to self-reported relief. These hypotheses and the analysis approach were preregistered on Open Science Framework (OSF, https://osf.io/ugkzf). Small deviations related to the approach are reported in the supplementary material (Supplementary Note 5).
Results
Self-reported relief and omission SCR track omissions of threat in a PE-like manner
The verbal instructions were effective at raising the expectation of receiving the electrical stimulation in line with the provided probability and intensity levels. Anticipatory SCR, which we used as a proxy of fearful expectation, increased as a function of the probability and intensity instructions (see Supplemental Figure 3). Accordingly, post-experimental questions revealed that by the end of the experiment participants recollected having received more stimulations after higher probability instructions, and were willing to exert more effort to prevent stronger hypothetical stimulations (see Supplemental Figure 2).
Replicating our previous findings29, self-reported relief-pleasantness and omission SCR tracked the PE signal during threat omission (see Fig.1B/C). Overall, unexpected (non-0%) omissions of threat elicited higher levels of relief-pleasantness and omission SCR than fully expected omissions (0%), evidenced by a main effect of Probability in the 4 (Probability: 0%, 25%, 50%, 75%) x 4 (Run: 1, 2, 3, 4) LMM (For relief pleasantness (N = 31): F(3,1417) = 188.34, p < .001, ω 2 = 0.28; and for omission SCR (N = 26): F(3,1190) = 72.90, p < .001, ω 2 = 0.15, with responses to all non-0% probability levels being significantly higher than responses to 0%, p’s < .001, Bonferroni-Holm corrected). Furthermore, relief-pleasantness and omission SCR to unexpected omissions (non-0% omissions) increased as a function of instructed Probability and Intensity, in line with the first two PE axioms, evidenced by main effects of Probability (for relief-pleasantness (N = 31): F(2,1031) =30.64, p < .001, ω 2 = 0.05 , all corrected pairwise comparisons, p’s < .005; for omission SCR (N = 26): F(2,862) = 5.15 , p < .01, ω 2 = 0.01, with corrected 75% to 25% comparison, p < .01), and Intensity (for relief-pleasantness (N = 31): F(2, 1031) = 623.79, p < .001, ω 2 = 0.55, all corrected pairwise comparisons, p’s < .001; for omission SCR (N = 26): F(2,862.01) = 107.47, p < .001, ω 2 = 0.20, all corrected pairwise comparisons, p’s <.001) in a 3 (Probability: 25%, 50%, 75%) x 3 (Intensity: weak, moderate, strong) x 4 (Run: 1, 2, 3, 4) LMM. Relief-pleasantness also showed a significant Probability x Intensity interaction (F(4,1031) = 3.76, p < .005, ω 2 = 0.01), indicating that the effect of probability was most pronounced for omissions of moderate stimulation (all p’s < .05). Note that while there was a general drop in reported relief pleasantness and omission SCR over time, the effects of Probability and Intensity remained present until the last run (see Supplementary Figure 5). This further confirms that probability and intensity instructions were effective until the end of the task.
Unexpected omissions of threat trigger activations in the VTA/SN and ventral Putamen, but deactivations in the vmPFC
In line with our hypothesis and similar to relief and omission SCR, unexpected (non-0%) omissions of threat elicited on average stronger fMRI activations than fully expected (0%) omissions in the VTA/SN (t(30) = 4.48, p < .001, d = 0.81) and left ventral putamen (t(30) = 3.50, p < .005, d = 0.63) ROIs (see Fig. 2A/B). Surprisingly, NAc showed no significant change in activation (t(30) = -0.59, p = .56) (Fig. 2D), and vmPFC showed a significant deactivation (t(30) = -4.71, p < .001, d = -0.85) (Fig. 2C). This apparent deactivation could indicate that omission-related responses were lower for unexpected omissions compared to expected omissions. However, it could also have resulted from lingering safety-related vmPFC activations to the 0%-instructions (corresponding to certainty that no stimulation will follow). Such safety-related vmPFC activations are indeed commonly observed during the presentation of CS-in Pavlovian fear conditioning30 To exclude this alternative hypothesis, we examined the non0% > 0% contrast during the instruction window. We found no significant difference in vmPFC activation between 0% and non-0% trials, either as ROI average (t(30) = -1.69, punc = .1) or voxel-wise, SVC within the vmPFC mask (see Supplementary Figures 7-9 and Supplementary Tables 3-5 for full description of the anticipatory fMRI activations). This follow-up analysis suggests that the deactivation to unexpected omissions only emerged after the instruction window, and could therefore not be explained by safety-related activation that were obtained during 0% trials.
Omission-related VTA/SN, but not striatal or vmPFC activations increased in a PE-like manner
We next assessed if the omission-related (de)activations could represent reward-like PE-signals by testing the PE axioms for each ROI separately. For axiom 1 and 2, we contrasted omissions following all intensity x probability combination with 0%-omissions and extracted ROI-specific beta averages. These beta-estimates were then entered into linear mixed models that included instructed intensity and probability as regressors of interest, and averaged US-unpleasantness as regressor of no-interest, in addition to a subject-specific intercept. Axiom 3 was tested via a one-sample (two-sided) t-test over the 100%-stimulation versus 0%-omission contrast.
We found that only omission-related VTA/SN activations partially fit the profile of a positive reward PE-signal (Fig. 2A). Activations were stronger following omissions of more intense (Axiom 1, F(2,240) = 6.14, p < .005, ω 2 = .04), and at trend level of more probable threat (Axiom 2, F(2,240) = 2.94, p = .055, ω 2 = .02). However, fully predicted stimulations (100%-trials) elicited stronger activations than fully predicted omissions (0%-trials) (Vwilcoxon = 452, p < .001, r = 0.72), contradicting axiom 3 (Fig 2E).
Unlike previous findings14,15,17,18, we found no evidence for striatal reward PE-like processing of threat omissions. While ventral putamen responses were stronger following unexpected omissions compared to expected omissions, these activations did not increase with increasing intensity (axiom 1, F(2,240) = 2.29, p = .10) or probability (axiom 2, F(2,240) = 0.57, p = .57 ) (Fig 2B). Notably, we did find anecdotal evidence that fully predicted stimulations and fully predicted omissions triggered similar activations in the ventral putamen (axiom 3, t(30) = 1.23, p = .46, BF of 2.62 in favor of the null-hypothesis, Fig. 2F). This indicates that ventral putamen activations were exclusively triggered by unexpected threat omissions, and not by fully predicted outcomes, which is similar to a PE-signal.
In general, there was no evidence for omission-related NAc activations. Activations were not affected by intensity (axiom 1, F(2,240) = 1.88, p = . 16) or probability instructions (axiom 2, F(2,240) = 0.75, p = .48) (Fig 2D), and while there were no differences in activation between fully predicted stimulation and fully predicted omission (t(30) = 0.67, p = .51, BF in favor of null hypothesis = 4.24), this equivalence was most likely caused by an overall absence of activation (Fig 2G).
Finally, omission-related vmPFC deactivations were stronger for omissions of more intense threat (axiom 1, F(2,240) = 9.29, p < .001, ω 2 = 0.06), but were unaffected by probability instructions (axiom 2, F(2,240) = 1.78, p = . 17) (Fig 2C). Furthermore, responses were smaller for completely predicted stimulation compared to completely predicted omission (axiom 3, t(30) = -8.65, p < .001, d = -1.55, Fig 2H). Taken together, we found no evidence that vmPFC deactivations reflected a positive reward PE-like signal.
A potential explanation for the absent probability effects in the putamen and vmPFC might be that the effects were obscured by including participants who did not believe the probability instructions. Indeed, the provided instructions did not map exactly onto the actually experienced probabilities, but were all followed by stimulation in 25% on the trials (except for the 0% trials and the 100% trials). We therefore reran our analyses on a subset of participants who showed probability-related increases in their anticipatory SCR during the countdown clock (N = 21, larger SCR to 75% compared to 25% instructions, see Supplementary Figure 4), which we used as a post-hoc index of actual probability-related expectancy. This subgroup analysis revealed no additional effect of Probability for the ventral putamen or the vmPFC, but it rendered the effect of Intensity for the ventral Putamen significant (F(2,160) = 3.10 p < .05, ω 2 =0.03). In addition, it increased the effect of probability for the VTA/SN activation (ω 2 = .02 to .05). Likewise, a post-hoc trial-by-trial analysis of the omission-related fMRI activations confirmed that the Probability effect for the VTA/SN activations was stable over the course of the experiment (no Probability x Run interaction) and remained present when accounting for the Gambler’s fallacy (i.e., the possibility that participants start to expect a stimulation more when more time has passed since the last stimulation was experienced) (see supplemental note 6). Overall, these post-hoc analyses further confirm the PE-profile of omission-related VTA/SN responses.Anterior insula and dmPFC/aMCC clusters show increased activation for unexpected omissions of threat in a PE-like fashion.
We then explored neural threat omission processing within a wider secondary mask that combined our primary ROIs with additional regions that have previously been associated to reward, pain and PE processing (such as the wider striatum, including the caudate nucleus, and putamen; midbrain nuclei, including the periaqueductal gray (PAG), and red nucleus; medial temporal structures, including the amygdala and hippocampus; midline thalamus, habenula and cortical regions, including the anterior insula (aINS), orbitofrontal (OFC), dorsomedial prefrontal (dmPFC) and anterior cingulate (ACC) cortices). Significant omission-processing clusters (contrast non0% > 0% omissions) were extracted from the mask using a cluster-level threshold (p < .05, FWE-corrected), following a primary voxel-threshold (p < .001) and included the bilateral anterior insula, bilateral putamen, and a medial cortical cluster encompassing parts of the dorsomedial prefrontal cortex (dmPFC) and the anterior medical cingulate cortex (aMCC) (Fig. 3A-D). The left putamen cluster bordered and minimally overlapped with our predefined ventral putamen ROI (4 out of 82 voxels) which was based on the peak PE activity in previous studies14,15. Exploratory analyses within a wider whole-brain grey-matter mask identified several other omission-processing clusters (see Supplementary Figure 10, Supplementary Table 6). Probability and Intensity related activity modulations of these clusters can be found in Supplementary Figure 11.
Follow-up analysis of the bilateral putamen clusters confirmed that putamen activations did not fit the profile of a positive reward PE. Activations in neither cluster increased with increasing intensity (axiom 1, left: F(2,240) = 0.18, p = .83; right: F(2,240) = 0.06, p = .94) nor probability of threat (axiom 2, left: F(2,240) = 1.41, p = .25; right: F(2,240) = 0.87, p = .42), but like for the a priori ventral Putamen ROI, activations were comparable for fully predicted outcomes, especially in the left cluster (axiom 3, left: V = 339, p = .46, BF = 2.41, right: t(30) = 1.83, p = .46, BF = 1.20).
Positive reward PE-like responses were found in the bilateral aINS and dmPFC/aMCC, where omission-related activations were stronger following omissions of more intense (left aINS: F(2,240) = 8.95, p < .001, ω 2 = 0.06; right aINS: F(2,240) = 13.49, p < .001, ω 2 = 0.09; dmPFC/aMCC: F(2,240) = 6.59 , p < .005, ωp2 = 0.04), and at trend level of more probable threat (left aINS: F(2,240) = 1.87, p =.16 , right aINS: F(2,240) = 2.78, p = 0.06, ωp2 = 0.01; dmPFC/aMCC: F(2,240) = 2.48, p =.09, ωp2 = 0.01). Notably aINS and dmPFC/aMCC clusters extended beyond our predefined secondary mask, and including all adjacent above-threshold voxels (p < .001) rendered the effects of probability significant (see Supplementary Figure 11). However, fully predicted stimulations also elicited stronger activations than fully predicted omissions (left aINS: t(30) = 3.21, p < .05, d = 0.58, right aINS: t(30) = 5.26, p < .001, d = 0.95, mdPFC/aMCC: t(30) = 5.57, p < .001, d = 1.00).
Finally, in addition to the vmPFC deactivations (which fell entirely within our vmPFC mask), we found trend-level deactivations for unexpected omission in the right amygdala (p = .053). These deactivations were stronger for omissions of more intense (F(2,240) = 3.26, p < .05, ω 2 = 0.02), but not more probable threat (F(2,240) = 0.43, p = .65). Furthermore, fully predicted stimulations triggered larger deactivation than fully predicted omissions (t = -2.77, p = .07, BF = 0.21).
Omission-related activations are related to self-reported relief-pleasantness
We then examined whether omission-related fMRI activations were related to self-reported relief-pleasantness on a trial-by-trial basis. In a pre-registered analysis, we entered z-scored relief-pleasantness ratings as a parametric modulator to the omission regressor in a separate GLM that did not distinguish between the different probability x intensity levels. We found that the VTA/SN (t(30) = 3.26, p < .01, d = 0.59) and ventral putamen ROI (at trend level, t(30) = 2.22, p = .068, d = 0.40) were positively modulated by relief-pleasantness ratings, whereas the vmPFC ROI was negatively modulated by relief-pleasantness ratings (V = 46, p < .001, r = 0.71) (Fig. 4A-D). The positive and negative modulations indicate that stronger omission-related activations in the VTA/SN and ventral putamen, and stronger deactivations in the vmPFC were associated with more pleasant relief-reports. Likewise, the bilateral aINS (t > 6.70, p < .001, d > 1.20), dmPFC/aMCC (t(30) = 6.13, p < .001, d = 1.10), and right putamen (t(30) = 4.90, p < .001, d = 0.88), and at trend level left putamen (t(30) = 2.37, p = 0.097, d = 0.43) clusters we identified from the secondary mask were positively modulated, and right amygdala was negatively modulated by relief-pleasantness (V = 59, p < .001, r = 0.67) (Fig. 4E-H). Omission-related NAc activation was unrelated to self-reported relief-pleasantness (V = 214, p = 0.52).
A neural signature for relief-pleasantness
The (mass univariate) parametric modulation analysis showed that omission-related fMRI activity in our primary and secondary ROIs correlated with the pleasantness of the relief. However, given that each voxel/ROI is treated independently in this analysis, it remains unclear how the activations were embedded in a wider network of activation across the brain, and which regions contributed most to the prediction of relief. To overcome these limitations, we trained a (multivariate) LASSO-PCR model (Least Absolute Shrinkage and Selection Operator-Regularized Principle Component Regression) in order to identify whether a spatially distributed pattern of brain responses can predict the perceived pleasantness of the relief (or “neural signature” of relief)31. Because we used the whole-brain pattern (and not only our a priori ROIs), this analysis is completely data driven and can thus identify which clusters contribute most to the relief prediction. We trained the model using fivefold cross-validation with trial-by-trial whole-brain omission-related activation-maps as predictors, and trial-by-trial relief-pleasantness ratings as outcome.
Predicted and reported relief correlated significantly (r = 0.28, p < .001) (Fig. 5C), indicating that part of the variance in reported relief-pleasantness could be explained by the neural relief signature response (Fig. 5A). Follow-up bootstrap tests (5000 samples) identified a distributed pattern of positive and negative predictive clusters across the brain (Fig. 5B, Table 1). Increased responses in these clusters predicted increased/decreased relief-pleasantness, respectively. Notably, bootstrap tests indicated that none of our a priori regions of interest significantly contributed to the signature. This was further supported by a pre-registered virtual lesion analysis where we compared the predictive performance of our LASSO-PCR model based on whole-brain data to separate models excluding voxels from our main ROIs in each iteration (see Fig. 5D).
Discussion
We examined whether brain reactions to unexpected omissions of threat qualify as positive reward PE signals, and explored their link with subjective relief. We showed that, similar to an unexpected reward, unexpected omissions of stimulation triggered fMRI activations within key regions of the reward and salience pathways (such as the VTA/SN, putamen, dmPFC/aMCC and aINS), and that the magnitude of these activations correlated with the pleasantness of the reported relief. Moreover, omission-related activations in the VTA/SN, the primary reward PE-encoding region in animals4,5 and humans32,33, also tracked the probability and intensity of omitted stimulation, in line with the first two criteria of a positive reward-PE signal. In contrast, the NAc and the vmPFC, two other regions that have previously been implicated in reward PE, threat omission processing, and the valuation of rewards and relief16–18,34–36, showed no (NAc) or even decreased activations (vmPFC) in response to omitted threat; and no correlation (NAc) and a negative correlation (vmPFC) with subjective relief.
Overall, the observed activity pattern of the VTA/SN supports the hypothesis that unexpected omissions of threat are processed as reward PE-like signals in the human brain. However, there are two caveats to this interpretation. First, it remains unclear whether these activations reflect the activity of dopamine cells in this region. The dopamine basis of the reward PE is well established4,5, and similar VTA dopaminergic responses to threat omissions have been found during fear extinction in rodents6–8,11. Yet, the nature of fMRI measurements does not allow us to directly trace back the observed BOLD responses to the phasic firing of dopamine cells at the time of threat omission. Still, the general location of the omission-related VTA/SN activation is consistent with a dopaminergic basis37,38, as the peak activation falls within a more medial subregion of the SN, which is predominantly composed of dopamine cells (>80% of all cells)39 In addition, a recent human fear extinction study found that ingestion of the dopamine precursor L-Dopa increased functional coupling between NAc and VTA/SN at the time of threat omission17.
A second caveat to the PE-interpretation of VTA/SN activations in the light of the present results is that fully predicted stimulations (100%) triggered stronger activations than fully predicted omissions (0%), which violates the third PE axiom and therefore opposes a PE interpretation. Theoretically, the third axiom states that a pure PE-signal would not differentiate between these fully predicted outcomes, whatever the outcomes are. As such, the violation implies that the VTA/SN responses could not represent PE-signals. Yet, we argue that this axiom should not be a decisive criterion when comparing fully predicted threat to its fully predicted omission. Specifically, a wealth of studies has reported VTA/SN responses to both salient events (even aversive) and PE4,40, and showed that these responses might be functionally and anatomically distinct38,41,42. Moreover, on a smaller scale, recent electrophysiological studies suggest that dopaminergic PE-signals themselves consist of two components: an initial unselective and short-lasting component that detects any event of sufficient intensity, followed by a subsequent component that codes the prediction error4. Thus, given that we could not control for the delivery of the stimulation in the 100% > 0% contrast (the delivery of the stimulation completely overlapped with the contrast of interest), it is impossible to disentangle responses to the salience of the stimulation from those to the predictability of the outcome. A fairer evaluation of the third axiom would require outcomes that are roughly similar in terms of salience. When evaluating threat omission PE, this implies comparing fully expected threat omissions following 0% instructions to fully expected absence of stimulation at another point in the task (e.g. during a safe intertrial interval).
Beyond the midbrain, unexpected omissions of threat also elicited striatal activations that spread across the bilateral putamen (as in14,15). Although these activations did not increase with increasing probability and intensity of the omitted stimulation, they were correlated with the reported relief-pleasantness and only occurred when the omission was unexpected (not when the outcome was fully expected, 100%-stimulation or 0% omission). Yet, in contrast to our predictions, the activations did not extend to the NAc. This was surprising, because the NAc is the main striatal projection area of the VTA, and numerous rodent and human studies have attributed reward PE signaling to this region25,43. Likewise, two human studies found that self-reported relief and modeled PE-estimates at the time of threat omission covaried with NAc activations17,18.
Nevertheless, a growing body of research now indicates that human PE encoding is not confined to the NAc, but instead spreads across the striatum. Meta-analyses on reinforcement learning in humans have identified the (left) putamen, and not the NAc as the most consistent reward PE-encoding region across studies44,45. Arguably, this emerging functional divergence between rodent and human striatal responses may be linked to neuroanatomical differences at the level of the midbrain. Specifically, where the (medial) VTA and its NAc projections have revealed marked omission-processing in rodents, we found the strongest omission-related activations in the (medial) SN and the putamen, which is a SN projection target. In addition, task-related differences might have contributed to the absent NAc activations. One of the meta-analyses found that the NAc was especially involved in PE-responses when a response is required to obtain the reward/threat omission (e.g., in instrumental tasks)44. Arguably, as the EVA task does not allow active control over the omission, it might not have engaged the NAc. Together, our findings call for caution when directly extrapolating rodent findings to human fMRI results.
We found omission-related deactivations in the vmPFC that were stronger following omissions of more intense threat, and that correlated with the reported relief. This is again in contrast with our predictions, and with previous studies that showed vmPFC activations during early US omissions in the context of fear extinction16 and positive associations between vmPFC activity and subjective pleasure35,36. Instead, we found vmPFC deactivations for both omissions and stimulations. Interestingly, these deactivations were not limited to the vmPFC, but spanned across key regions of the default mode network46, such as the PCC and precuneus (see Supplementary Figure 10 and Supplementary Table 6). Furthermore, they were accompanied by widespread activations in key regions of the salience network47 (such as the VTA/SN, striatum, aINS, dmPFC/aMCC). One potential explanation is therefore that the deactivation resulted from a switch from default mode to salience network, triggered by the salience of the unexpected threat omission or by the salience of the experienced stimulation.
In addition to examining the PE-properties of neural omission responses in our a priori ROIs, we trained a LASSO-PCR model to establish a signature pattern of relief. One interesting finding that only became evident when we compared the univariate and multivariate approach was that none of our a priori ROIs appeared to be an important contributor to the multivariate neural signature, even though all of them (except NAc) were significantly modulated by relief in the univariate analysis. Instead, we identified a spatially distributed pattern of brain responses that consisted of several small clusters (all < 65 voxels) across the brain. Some of these clusters fell within other important valuation and error-processing regions in the brain (e.g., OFC, MCC, caudate nucleus). However, all were small (all < 28 voxels) and require further validation in out of sample participants. Still, these data-driven maps suggest that other regions than our ROIs might have been especially important for the emotional experience of relief and that examining these multivariate patterns can aid our understanding of emotional relief.
Finally, two limitations of the study need to be addressed. First, by aiming to provide a fine-grained analysis of the reward PE-properties of human fMRI responses to threat omission, we focused exclusively on the necessary and sufficient requirements of reward PE signaling, and thereby disregarded another core aspect of PE: its teaching property. In the EVA task expectancies are instructed and all learning is explicitly discouraged. As a result, this task assesses PE completely outside of a learning context. It therefore remains unclear how the PE-signals we observed relate to actual learning. It could for instance be that the observed responses mainly reflected the surprisingness of the outcome, independent of subsequent learning. It therefore remains important to study how the activity patterns of PE-encoding regions are related to expectancy updating in learning paradigms. Furthermore, it would be interesting to see how the neural omission-PE signals are affected in clinical populations. Second, although single unit recordings in rodents have revealed clear PE-like phasic increases in the firing of dopamine cells at the time of threat omission6–8, one could argue that fMRI measurements cannot capture the same sub-second response. Indeed, it is generally difficult to disentangle prediction error responses from the immediately preceding prediction responses in fMRI paradigms48. Nevertheless, there was no multicollinearity between anticipation and omission regressors in the first-level GLMs (Variance Inflation Factor, VIF < 4), making it unlikely that the omission responses purely represented anticipation. Still, because of the slower timescale of fMRI measurements, we cannot conclusively dismiss the alternative interpretation that we assessed (part of) expectancy instead.
In conclusion, by violating instructions about the probability and intensity of a potentially painful stimulation, we found widespread activations in the reward and salience pathways that were furthermore related to PE-like feelings of pleasurable relief. But, more importantly, we showed for the first time in humans that unexpected threat omission triggered VTA/SN activations that partially met the necessary and sufficient criteria of a positive reward PE signal. In doing so, we provided an important missing link for the human translation of the reward-like threat omission processing in rodents.
Methods
Participants
Thirty-one healthy volunteers between the ages of 18 and 25 (mean = 20.65, 19 females) were recruited to participate in our study. All were right-handed and non-smoking, and declared to be free of any serious medical or psychiatric disorder and regular medication use (see Supplementary Notes 1 & 2 and Supplementary Table 1 & 2 for an overview of the exclusion criteria we employed, the sample size rationale and the resulting study sample). Upon their enrollment, participants were furthermore asked to refrain from consuming any alcohol and/or caffeine and exerting any recreational physical exercise in the 24 hours before the scan session. The study was approved by the Ethical committee UZ/KU Leuven (S63852). All participants provided written informed consent and received either partial course credits or a monetary compensation for their participation.
Stimuli
Expectancy Violation Assessment (EVA) Task
The task was an fMRI adaptation of the previously validated EVA task29 and was programmed in affect5 software49. In this task, probability (0%, 25%, 50%, 75%, 100%) and intensity (weak, moderate, strong) information of an upcoming electrical stimulation to the wrist was presented on each trial in the upper left and right corner of the screen respectively. A countdown clock, visualized as a receding bar, indicated the exact moment of stimulation or omission. Responses to the omissions/stimulations were measured.
Electrical Stimulation
The electrical stimulation consisted of a single 2 ms electro-cutaneous pulse, generated by a Digitimer DS8R Bipolar Constant Current Stimulator (Digitimer Ltd, Welwyn Garden City, UK), and delivered via two MR-compatible EL509 electrodes (Biopac Systems, Goleta, CA, USA). Electrodes were filled with Isotonic recording gel (Gel 101; Biopac Systems, Goleta, CA, USA), and were attached to the right wrist. To match the instructions, a total of three intensities were individually selected at the start of the scanning session. The weak stimulus was calibrated to be “mildly uncomfortable”, the moderate stimulus to be “very uncomfortable, but not painful”, and the strong stimulus to be “significantly painful, but tolerable”. Selected Weak (M = 9.35, SD = 4.65), moderate (M = 16.26 mA, SD = 8.13), and strong (M = 34.26, SD = 19.53) differed significantly (Friedman test: χ2(2) = 62, p < .001, W = 1, all Bonferroni-Holm corrected pairwise comparisons, p < .001).
Experimental Procedure
Participants were invited to the lab for an intake session, during which exclusion criteria were extensively checked, experimental procedures were explained, and informed consent was obtained. All included participants then filled out the questionnaire battery (see Supplementary Table 2) and were familiarized with the task and rating scales.
At the start of the scanning session, participants were fitted with skin conductance and stimulation electrodes and stimulation intensities were calibrated using a standard workup procedure (for details, see Supplementary Note 3). Participants were then prepared for the scanner and task instructions were repeated. It was emphasized that all trials in the EVA task were independent of one another, meaning that the presence/absence of stimulations on previous trials could not predict the presence/absence of stimulation on future trials. Finally, the selected intensities were presented again and recalibrated if necessary.
In total, the EVA task comprised 72 trials (48 omission trials), divided equally over four runs of 18 trials/run (for a schematic overview of the trial numbers and types, see Supplementary Figure 1). Since we were mainly interested in how omissions of threat are processed, we wanted to maximize and balance the number of omission trials across the different probability and intensity levels, while also keeping the total number of presentations per probability and intensity instruction constant. Therefore, we crossed all non-0% probability levels (25, 50, 75, 100) with all intensity levels (weak, moderate, strong) (12 trials). The three 100% trials were always followed by the stimulation of the instructed intensity, while stimulations were omitted in the remaining nine trials. Six additional trials were intermixed in each run: Three 0% omission trials with the information that no electrical stimulation would follow (akin to 0% Probability information, but without any Intensity information as it does not apply); and three trials from the Probability x Intensity matrix that were followed by electrical stimulation (across the four runs, each Probability x Intensity combination was paired at least once, and at most twice with the electrical stimulation). Note that, based on previous research, we did not expect the inconsistency between the instructed and perceived reinforcement rate to have a negative effect on the Probability manipulation (see Supplementary Note 4). Within each run trials were presented in a pseudo-random order, with at most two trials with the same intensity or probability as the previous trial.
All trials started with the Probability and Intensity instructions for 1 second, followed by the addition of the countdown bar to the middle of the screen, counting down for 3 to 7 seconds (see Fig. 1A). Then, the screen cleared and the electrical stimulation was either delivered or omitted. Following a delay of 4 to 8 seconds (during which skin conductance and BOLD responses to the omission were measured), the rating scale appeared (probing shock-unpleasantness on shock trials and relief-pleasantness on omission trials). The scale remained on the screen for 8 seconds or until the participant responded, followed by an intertrial interval between 4 and 7 seconds during which only a fixation cross was shown. Note that all phases in the trial were jittered (duration countdown clock, duration outcome window, duration intertrial interval). After the last run, participants were asked some control questions regarding the intensity and probability instructions. Specifically, they were asked how much effort they would exert to prevent future weak/moderate/strong stimulation (from 0 “no effort” to 100 “a lot of effort”); and to estimate how many stimulations they thought they received following each probability instruction.
Subjective ratings
Relief-pleasantness and shock-unpleasantness were probed on omission-and shock-trials using Visual Analogue Scales (VAS) ranging from 0 (neutral) to 100 (very pleasant/unpleasant).
Skin Conductance Responses (SCR)
Fluctuations in skin conductance were measured between two disposable, EL509 electrodes filled with Isotonic recording gel (Gel 101; Biopac Systems, Goleta, CA, USA) that were attached to the hypothenar palm of the left hand. Data were recorded continuously at a 1000 Hz sampling rate via a Biopac MP160 System (Biopac Systems, Goleta, CA, USA), and Acqknowledge software (version 5.0).
Raw data were low-passed filtered at 5 Hz (Butterworth, zerophased) and downsampled to 100 Hz in Matlab (version R2020b), after which they were entered into a continuous decomposition analysis (CDA) with two optimization runs (Ledalab, version 3.4.950). Skin conductance responses (SCR) were scored as the time integral of the deconvoluted phasic activity (integrated SCR) within response windows of 1-4 sec after the onset (anticipatory SCR) and the offset (omission/stimulation SCR) of the countdown clock. Above-threshold responses (>0.01 µS) were square root transformed to reduce the skewness of the distribution. N = 3 participants had incomplete datasets because of a missing run (N = 2) or delayed recording (N = 1), and data of N = 5 were completely excluded for SCR analyses as a result of data loss due to technical difficulties (N = 1) or because they were identified as anticipation non-responder (i.e. participant with smaller average SCR to the clock on 100% than on 0% trials) (N = 4). This resulted in a total sample of N = 26 for all SCR analyses. For all other analyses, the full data set (N = 31) was used (see Supplementary Table 1 for detailed overview).
Statistical Analyses of Ratings and SCR
Rating and SCR data were analyzed in R 4.2.1 (R Core Team, 2022; https://www.R-project.org) via linear mixed models that were fit using the lme4 package (v1.1.2951). All models included within-subject factors of Intensity and/or Probability and their interaction as fixed effect and a subject-specific intercept as random effect. The intensity factor always contained 3 levels (weak, moderate, strong). The Probability factor depended on the outcome variable. For models of relief and omission SCR that looked at Probability as well as Intensity, Probability had 3 levels (25%, 50%, 75%); but for models of relief and omission SCR that only assessed Probability, Probability had 4 levels (0%, 25%, 50%, 75%). To account for potential changes in the effects of Probability (and Intensity) over time, models included Run (4 levels: 1, 2, 3, 4) and their interaction with Probability (and Intensity) as regressor of no-interest. In addition, we controlled for individual differences in the perceived unpleasantness of the stimulation, by calculating the average of the reported stimulation unpleasantness across the entire task, and entering the resulting (mean-centered) scores as a between-subjects covariate. Inclusion of both regressors of no-interest indeed increased model fit (lower AIC). Model assumptions were checked and influential outliers were identified. Influential outliers were defined as data points above q0.75 + 1.5 * IQR or below q0.25 -1.5*IQR, with IQR the inter quartile range and q0.25 and q0.75 corresponding to the first and third quartile, respectively; and with a cook’s distance of greater than 4/ number of data points (calculated via influence.ME package in R, v0.9-952). To reduce the influence of these data points, they were rescored to twice the standard deviation from the mean of all data points (corresponding approximately to either .05 and .95 percentile). If results did not change, we report the model including the original data points. If results changed, we report the model with adjusted data points.
Main and interaction effects were evaluated using F-tests and p-values that were computed via type III analysis of Variance using Kenward-Rogers degrees of freedom method of the lmerTest package (v3.1.353), and omega squared were reported as an unbiased estimate of effect size (calculated via effectsize package, v0.7.054). All significant effects were followed up with Bonferroni-Holm corrected pairwise comparisons of the estimated marginal means in order to assess the direction of the effect (emmeans package, v1.7.5, Length, 2022). Results related to the regressors of no-interest (Supplementary Figure 5), the anticipatory SCR (Supplementary Figure 3), the post-experimental questions (Supplementary Figure 2), and the stimulation responses (Supplementary Figure 6) are reported in the supplementary material for completeness.
fMRI Analyses
MRI Acquisition
MRI data were acquired on a 3 Tesla Philips Achieva scanner, using a 32-channel head coil, at the Department of Radiology of the University Hospitals Leuven. The four functional runs (226 volumes each) were recorded using an T2*-weighted echo planar imaging sequence (60 axial slices; FOV = 224 x 224 mm; in-plane resolution = 2 x2 mm; interslice gap = 0.2 mm; TR = 2000 ms; TE = 30 ms; MB = 2; flip angle = 90°). In addition, a high resolution T1-weighted anatomical image was acquired for each subject for co-registration and normalization of the EPI data using a MP-RAGE sequence (182 axial slices; FOV = 256 x 240 mm; in-plane resolution = 1 x 1 mm; TR = shortest, TE = 4.6 ms , flip angle = 8°); and a short reverse phase functional run (10 volumes) was acquired using the exact same imaging parameters as the functional runs, but with opposite phase encoding direction. This reverse phase run was used to estimate the B0-nonuniformity map (or fieldmap) to correct for susceptibility distortion. Functional data of one run were missing for N = 4 participants as a result of technical difficulties during scanning. Whenever available, rating and SCR data was still included in the analyses.
fMRI Preprocessing
Prior to preprocessing, image quality was visually checked via quality reports of the anatomical and functional images generated through MRIQC55. MRI data were then preprocessed using a standard preprocessing pipeline in fMRIPrep 20.2.356. A detailed overview of the preprocessing steps in fMRPrep can be found in Supplementary Methods 2. In line with our preregistration, spikes were identified and defined as volumes having a framewise displacement (FD) exceeding a threshold of 0.9 mm or DVARS exceeding a threshold of 2. No functional run had more than 15% spikes, and hence none of the runs had to be excluded from our analyses based on our preregistered criterium. Afterwards, the functional data were spatially smoothed with a 4 mm FWHM isotropic Gausian kernel within the “Statistical parametric Mapping” software (SPM12; https://www.fil.ion.ucl.ac.uk/spm/).
Subject Level Analysis
Three subject-level general linear models (GLM) were specified. In all models, we concatenated the functional runs and added run-specific intercepts to account for changes over time. The first model investigated the effects of instructed Probability and Intensity on neural omission processing and therefore included separate stick regressors (duration = 0) for omissions of each Probability x Intensity combination (10 regressors), and stimulations (2 regressors: one for non-100% stimulations, one for 100% stimulations), in addition to boxcar regressors (duration = total duration of event) for the instruction (1 regressor) and rating (1 regressor) windows. The second model assessed how omission fMRI data was related to trial-by-trial relief-pleasantness ratings, by only including a single stick regressor for all omissions (1 regressor) and shocks (1 regressor), and boxcar regressors for instructions (1 regressor) and ratings (1 regressor). Z-scored relief-pleasantness ratings were added as parametric modulator for the omission regressor. A final model estimated single-trial omission responses (48 stick regressors), in addition to a single stick regressor for stimulation and boxcar regressors for instructions (1 regressor) and ratings (1 regressor) and was used for the LASSO-PCR analyses. Regressors in all models were convolved with a canonical hemodynamic response function and a high pass filter of 180 s was applied to remove low frequency drift. Additional non-task related noise was modeled by including nuisance regressors of no-interest for global CSF signal, 24 head motion regressors (consisting of 6 translation and rotation motion parameters and their derivatives, z-scored; and their quadratic terms), and dummy spike regressors.
Group Level Univariate Analysis
Omission processing
To test whether our regions of interest (ROI) were activated by the unexpected omission of threat, we contrasted all non-0% omissions (unexpected omissions) with 0% omissions (expected omissions) at subject-level. Mean activity of each ROI was extracted from the resulting contrast map through marsbar (v0.4557), and was entered into group-level (two-sided) one-sample t-tests (per ROI) that were Bonferroni-Holm corrected for the total number of ROIs (4 in the main analyses, 10 in the secondary analyses) in R.
We then evaluated whether the observed activity qualified as a ‘Prediction Error’ by applying an axiomatic testing approach. Specifically, we tested for each ROI if the omission-related activity increased with increasing expected Intensity (axiom 1) and Probability (axiom 2) of threat, and whether completely predicted outcomes (0% omission and 100% stimulations) elicited equivalent activation (axiom 3). For axioms 1 and 2, we extracted mean activity of each ROI from separate omission contrasts that contrasted each omission type (i.e., all possible Probability x Intensity combinations) separately with 0% omissions. These were then entered into a LMM in R including Probability (3 levels: 25%, 50%, 75%) and Intensity (3 levels: weak, moderate, strong) as within-subject factors, and a between-subject covariate of average stimulation unpleasantness (mean-centered) as fixed effects, in addition to a subject-specific intercept. Note that we did not include Run as regressor of no-interest, as Run effects were already accounted for by adding run-specific intercepts to the first level models. Main and interaction effects within each model were followed up with Bonferroni-Holm corrected pairwise comparisons of the estimated marginal means in order to assess the direction of the effect. Finally, to fulfill all necessary and sufficient requirements of a prediction error signal, we contrasted completely predicted omissions (0%) with completely predicted stimulations (100%), as these should trigger equivalent activation in PE-encoding regions. Given that we would expect equivalent activation, Bayes Factors were computed using the BayesFactor package in R (v0.9.12-4.4, Morey & Rouder, 2022) to compare the evidence in favor of alternative and null hypotheses. Larger Bayes Factors indicated more evidence in favor of the null hypotheses.
Parametric modulation of relief
We then tested if the omission-related activity was correlated to self-reported relief-pleasantness on a trial-by-trial basis. To this end, we extracted the mean activity from the modulation contrast for each ROI, and entered these averages in separate one-sample (two-sided) t-test in R, again correcting for the number of ROIs (4 for main analysis, 10 for secondary analyses). In a subsequent exploratory analysis, we entered z-scored omission SCR-responses as parametric modulator to the omission regressor. Results related to this analysis are reported in Supplementary Figure 13 and Supplementary Table 8.
Neural signature of relief
A LASSO-PCR model (Least Absolute Shrinkage and Selection Operator-Regularized Principal Component Regression) as implemented in CANlab neuroimaging analysis tools (see https://canlab.github.io/) was trained using trial-by-trial whole-brain omission-related activation-maps as predictors, and trial-by-trial relief-pleasantness ratings as outcome (for other applications of this approach, see e.g.31,58–60). The added value of this machine-learning technique is that relief is predicted across a set of voxels instead of being predicted for each voxel separately (as in standard univariate regression). Specifically, while each voxel in the activation maps is considered a predictor of relief, the LASSO-PCR technique uses a combination of Principle Component Analyses (PCA) and LASSO-regression to (1) group predictive information across individual voxels into larger components (PCA), and (2) maximize the predictive weight of the most informative components by shrinking the regression weight of the least informative components to zero (LASSO regression). Here, we embedded the LASSO-PCR technique within a five-fold cross-validation loop that iteratively trained a LASSO-PCR model in each loop on a different training and validation dataset, which we then averaged across loops to obtain the final model. Model performance was then assessed by calculating the Pearson correlation between reported and model predicted relief. Important features to the signature pattern were identified using bootstrap tests (5000 samples). Furthermore, the contribution of our ROIs to the model’s performance was assessed using virtual lesion analysis that consisted of repeating the model training, but excluding voxels within the ROIs and assessing model performance. Note that we also estimated a neural signature of omission SCR, by applying a LASSO-PCR model to predict omission-related SCR responses. Results related to these analyses are reported in Supplementary Figure 14 and Supplementary Table 9.
Regions of Interest (ROI)
Our main regions of interest consisted of key regions of the reward and (relief) valuation pathways such as the Ventral Tegmental Area (VTA) /Substantia Nigra (SN), Nucleus Accumbens (NAC), ventral Putamen, and ventromedial prefrontal cortex (vmPFC). The VTA/SN mask was obtained from Esser and colleagues17, but was originally defined by Bunzeck and Düzel (2006)61. The ventral Putamen ROI was defined as a 6 mm sphere centered around the peak voxel (MNI coordinates: -32,8-6) in the left ventral putamen identified by Raczka et al. (2011)14, as in Thiele et al. (2021)15. However, we overlayed this sphere with a Putamen mask obtained from a high-resolution anatomical atlas for subcortical nuclei defined by Pauli et al. (2018)62 to assure the mask was restricted to voxels of the putamen and did not extend to the adjacent anterior Insula. Likewise, a bilateral NAc mask was obtained from the Pauli et al. (2018)62 atlas. The vmPFC mask was obtained by selecting specific parcels from the atlas vmPFC cortex of AFNI (area 14, 32, 24, bilateral). Since we consider the pleasantness of relief from omission of a threat a type of reward, the selection of parcels was made on the base of an activation likelihood estimation (ALE) meta-analysis of 87 studies (1452 subjects) comparing the brain responses to monetary, erotic and food reward outcomes63.
In addition to our main ROIs, we specified a wider secondary mask that extended our main ROIs with additional regions that have previously been associated to reward, pain and PE processing (such as the wider striatum, including the nucleus caudatus, and putamen; midbrain nuclei, including the periaqueductal gray (PAG), habenula, and red nucleus; limbic structures, including the amygdala and hippocampus; midline thalamus and cortical regions, including the anterior insula (aINS), medial orbitofrontal (OFC), dorsomedial prefrontal (dmPFC) and anterior cingulate (ACC) cortices). Masks for these regions were obtained from CANlab combined atlas (2018) (see https://canlab.github.io/), and Pauli atlas62. Whole brain voxel-wise analyses were restricted to the grey matter mask sparse (from CANlab tools), extended with midbrain voxels (including VTA/SN, RN, habenula). All masks were registered to functional space before analyses and are freely available online at OSF (https://osf.io/ywpks/).
Data availability
Raw data files are not freely accessible because one participant did not consent to publish their anonymized data online. Data are available on request by contacting ALW or BV.
Code availability
Analyses code and anatomical masks are freely available at OSF (https://osf.io/ywpks/).
Acknowledgements
This research was supported by FWO project grant G078929N (National Research Fund Flanders, Belgium) and C1 project grant C16/19/002 (KU Leuven, Belgium) awarded to BV. ALW was supported by the Internal Funds KU Leuven. LVO is a research professor funded by the KU Leuven Special Research Fund. We would like to thank Mathijs Franssen and Ronald Peeters for their technical support; and Silvia Papalini, Anraoi Rooney, Lieselotte Claes and Anamarija Banjac for their assistance during fMRI data collection.
References
- 1.How absent negativity relates to affect and motivation: an integrative relief modelFront. Psychol 6
- 2.A theory of Pavlovian conditioning: The effectiveness of reinforcement and non-reinforcementClass. Cond. Curr. Res. Theory
- 3.Understanding clinical fear and anxiety through the lens of human fear conditioningNat. Rev. Psychol 2:233–245
- 4.Dopamine reward prediction-error signalling: A two-component responseNat. Rev. Neurosci 17:183–195
- 5.Neural circuitry of reward prediction errorAnnu. Rev. Neurosci 40:373–394
- 6.A dopaminergic switch for fear to safety transitionsNat. Commun 9
- 7.Dopamine neurons drive fear extinction learning by signaling the omission of expected aversive outcomesElife 7
- 8.Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different timesElife 9
- 9.A neural circuit mechanism for encoding aversive stimuli in the mesolimbic dopamine systemNeuron 101:133–151
- 10.Aversive stimuli differentially modulate real-time dopamine transmission dynamics within the nucleus accumbens core and shellJ. Neurosci 32:15779–15790
- 11.Brain mechanisms controlling pavlovian fear conditioningJ. Exp. Psychol. Anim. Learn. Cogn 44:341–357
- 12.Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidanceJ. Neurosci 32:14804–14808
- 13.Phasic dopamine signals in the nucleus accumbens that cause active avoidance require endocannabinoid mobilization in the midbrainCurr. Biol 28:1392–1404
- 14.Empirical support for an involvement of the mesostriatal dopamine system in human fear extinctionTransl. Psychiatry 1
- 15.A ventral striatal prediction error signal in human fear extinction learningNeuroimage 229
- 16.Neural responses during extinction learning predict exposure therapy outcome in phobia: results from a randomized-controlled trialNeuropsychopharmacology 45:534–541
- 17.L-DOPA modulates activity in the vmPFC, nucleus accumbens, and VTA during threat extinction learning in humansElife 10
- 18.Relief as a reward: Hedonic and neural responses to safety from painPLoS One 6
- 19.Active avoidance: Neural mechanisms and attenuation of pavlovian conditioned respondingJ. Neurosci 37:4808–4818
- 20.Dopamine: from prediction error to psychotherapyTransl. Psychiatry 10
- 21.A dopaminergic basis for fear extinctionTrends Cogn. Sci 23:274–277
- 22.Temporal dynamics of relief in avoidance conditioning and fear extinction: Experimental validation and clinical relevanceBehav. Res. Ther 96:66–78
- 23.The role of context in persistent avoidance and the predictive value of reliefBehav. Res. Ther 138
- 24.Axiomatic methods, dopamine and reward prediction errorCurr. Opin. Neurobiol 18:197–202
- 25.Testing the reward prediction error hypothesis with an axiomatic modelJ. Neurosci 30:13525–13536
- 26.Different brain systems support learning from received and avoided pain during human pain-avoidance learningElife 11
- 27.Representation of aversive prediction errors in the human periaqueductal grayNat. Neurosci 17:1607–1612
- 28.Asymmetric representation of aversive prediction errors in Pavlovian threat conditioningNeuroimage 263
- 29.When nothing matters : Assessing markers of expectancy violation during omissions of threatBehav. Res. Ther 136
- 30.Neural signatures of human fear conditioning: An updated and extended meta-analysis of fMRI studiesMol. Psychiatry 21:500–508
- 31.An fMRI-based neurologic signature of physical painN. Engl. J. Med 368:1388–1397
- 32.BOLD responses reflecting dopaminergic signals in the human ventral tegmental areaScience 319:1264–1267
- 33.Human substantia nigra neurons encode unexpected financial rewardsScience 323:1496–1499
- 34.Dopamine-dependent prefrontal reactivations explain long-term benefit of fear extinctionNat. Commun 9
- 35.Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brainPLOS Biol 4
- 36.Pleasure systems in the brainNeuron 86:646–664
- 37.Functional imaging of the human dopaminergic midbrainTrends Neurosci 32:321–328
- 38.Anatomical and functional organization of the human substantia Nigra and its connectionsElife 6
- 39.Glutamate neurons are intermixed with midbrain dopamine neurons in nonhuman primates and humansSci. Rep 6
- 40.prediction error and beyondNeuroscientist 27:30–46
- 41.Two types of dopamine neuron distinctly convey positive and negative motivational signalsNature 459:837–841
- 42.Distinct contributions of ventromedial and dorsolateral subregions of the human substantia nigra to appetitive and aversive learningJ. Neurosci 35:14220–14233
- 43.Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error termJ. Neurosci 34:698–704
- 44.Prediction error in reinforcement learning: A meta-analysis of neuroimaging studiesNeurosci. Biobehav. Rev 37:1297–1310
- 45.Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysisCogn. Affect. Behav. Neurosci 15:435–459
- 46.The brain’s default mode networkAnnu. Rev. Neurosci 38:433–447
- 47.The salience network: A neural system for perceiving and responding to homeostatic demandsJ. Neurosci 39:9878–9882
- 48.Unconditioned responses and functional fear networks in human classical conditioningBehav. Brain Res 221:237–245
- 49.Affect 4.0: A free software package for implementing psychological and psychophysiological experimentsExp. Psychol 57:36–45
- 50.A continuous measure of phasic electrodermal activityJ. Neurosci. Methods 190:80–91
- 51.Fitting linear mixed-effects models using lme4J. Stat. Softw 67:1–48
- 52.influence.ME: Tools for detecting influential data in mixed effects modelsR J. 4:38–47
- 53.lmerTEst package: Tests in linear mixed effects modelsJ. Stat. Softw 82:1–26
- 54.effectsize: Estimation of effect size indices and standardized parametersJ. Open Source Softw 5
- 55.MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sitesPLoS One 12
- 56.fMRIPrep: a robust preprocessing pipeline for functional MRINat. Methods 16:111–116
- 57.Region of interest analysis using an SPM toolbox8th International Conference on Functional Mapping of the Human Brain
- 58.Predicting individual differences in placebo analgesia: Contributions of brain activity during anticipation and pain experienceJ. Neurosci 31:439–452
- 59.A multivariate brain signature for rewardNeuroimage 271
- 60.A distributed fMRI-based signature for the subjective experience of fearNat. Commun 12
- 61.Absolute coding of stimulus novelty in the human substantia nigra/VTANeuron 51:369–379
- 62.A high-resolution probabilistic in vivo atlas of human subcortical brain nucleiSci. Data 5
- 63.Processing of primary and secondary rewards: A quantitative meta-analysis and review of human functional neuroimaging studiesNeurosci. Biobehav. Rev 37:681–696
Article and author information
Author information
Version history
- Sent for peer review:
- Preprint posted:
- Reviewed Preprint version 1:
- Reviewed Preprint version 2:
- Reviewed Preprint version 3:
Copyright
© 2023, Willems et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
- views
- 297
- downloads
- 6
- citation
- 1
Views, downloads and citations are aggregated across all versions of this paper published by eLife.