Robust group- but limited individual-level (longitudinal) reliability and insights into cross-phases response prediction of conditioned fear

The increasing incidence (e.g., Xiong et al., 2022) and high relapse rates (Essau et al., 2018; Yonkers et al., 2003) of anxiety-related disorders call for a better understanding of anxiety- and stress-related processes which might contribute to improving existing treatments or developing more effective interventions. In the laboratory, these processes can be studied using fear conditioning paradigms (Dunsmoor et al., 2022; Fullana et al., 2020; Milad and Quirk, 2012).

In differential fear conditioning protocols (see Lonsdorf et al., 2017a) one stimulus is repetitively paired with an aversive unconditioned stimulus (US; e.g., electrotactile stimulation), and as a consequence becomes a conditioned stimulus (CS+) while another stimulus, the CS−, is never paired with the US. After this acquisition training phase, CSs are presented without the US (extinction training) leading to a gradual waning of the conditioned response. Critically, the fear memory (CS+/US association) is not erased, but a competing inhibitory extinction memory (CS+/no US association) is assumed to be formed during extinction training (Milad and Quirk, 2012; Myers and Davis, 2007). Subsequently, return of fear (RoF) can be induced by procedural manipulations such as a time delay (spontaneous recovery), a contextual change (renewal, Vervliet et al., 2013a), or a (re-)presentation of an aversive event (reinstatement, Haaker et al., 2014). Conditioned responding can be subsequently probed in an RoF test phase during which either the absence (i.e., extinction retention) or the return of conditioned responding (i.e., RoF) can be observed (Bouton, 2004; Lonsdorf et al., 2017a).

Findings from studies employing fear conditioning paradigms hold strong potential for translating neuroscientific findings into clinical applications (Anderson and Insel, 2006; Cooper et al., 2022a; Fullana et al., 2020; Milad and Quirk, 2012). More precisely, extinction learning is assumed to be the active component of exposure-based treatment (Graham and Milad, 2011; Milad and Quirk, 2012; Rachman, 1989; Vervliet et al., 2013b) and experimental RoF manipulations have been suggested to serve as a model of clinical relapse (Scharfenort et al., 2016; Vervliet et al., 2013a). Important findings in the fear conditioning field include the deficient learning of the safety signal (CS−) during acquisition training, impaired extinction learning (Duits et al., 2015) and the tendency of fear generalization to innocuous stimuli (Cooper et al., 2022a) in patients suffering from anxiety-related disorders as compared to healthy controls.

To date, both clinical and experimental research using the fear conditioning paradigm have primarily focused on group-level, basic, general mechanisms such as the effect of experimental manipulations – which is important to investigate (Lonsdorf and Merz, 2017b). Successful clinical translation (e.g., ‘Why do some individuals develop pathological anxiety while others do not?’) and particularly treatment outcome prediction (e.g., ‘Why do some patients benefit from treatment while others relapse?’), however, requires that both the experimental paradigm and the measures employed allow for individual-level predictions over and above prediction of group averages (Fröhner et al., 2019; Hedge et al., 2018; Lonsdorf and Merz, 2017b). A prerequisite for this is that the measures show stability within and reliable differences between individuals over time. Hence, tackling clinical questions regarding individual-level predictions of symptom development or treatment outcome requires a shift toward and a validation of research methods tailored to individual differences – such as a focus on measurement reliability (Zuo et al., 2019). This is a necessary prerequisite for the long-term goal of developing individualized intervention and prevention programs. This further relates to the pronounced heterogeneity in symptom manifestation among individuals diagnosed with the same disorders (e.g., post-traumatic stress disorder, PTSD, Galatzer-Levy and Bryant, 2013b) which cannot be captured in binary clinical diagnoses as two patients with for example a PTSD diagnosis may not share a single symptom (Galatzer-Levy and Bryant, 2013b).

Measurement reliability has only recently gained momentum in experimental cognitive research (Fröhner et al., 2019; Hedge et al., 2018; Zuo et al., 2019) and can be assessed through test–retest and longitudinal reliability (i.e., test–retest reliability over longer time intervals, typically assessed through e.g., intraclass correlation coefficients, ICCs, see Table 1). Importantly, longitudinal reliability (for definitions and terminology, see Table 1) also has implications for the precision with which associations of one variable (e.g., conditioned responding) with another (individual difference) variable can be measured because the correlation between those two variables cannot exceed the correlations within, that is the reliability, of these two variables (Spearman, 1910).

Yet, in fear conditioning research, surprisingly little is known about longitudinal reliability at the individual level with time intervals ranging from 9 days to 8 months in prior work (Supplementary file 1; Cooper et al., 2022b; Fredrikson et al., 1993; Ridderbusch et al., 2021; Torrents-Rodas et al., 2014; Zeidan et al., 2012). Generally (details in Supplementary file 1), individual-level longitudinal reliability of risk ratings, skin conductance responses (SCRs), and fear potentiated startle (FPS) was within the same range (Cooper et al., 2022b; Torrents-Rodas et al., 2014) whereas it was numerically somewhat lower for the BOLD response as compared to different rating types (Ridderbusch et al., 2021). Longitudinal reliability at the individual level appeared higher for acquisition training than for extinction training (SCRs: Fredrikson et al., 1993; Zeidan et al., 2012), but comparable to generalization (Cooper et al., 2022b; Torrents-Rodas et al., 2014). Moreover, it appeared higher for extinction training than for reinstatement-test (for BOLD fMRI but not ratings: Ridderbusch et al., 2021) and higher for CS+ than CS− responses (SCRs: Fredrikson et al., 1993) and CS discrimination (ratings and BOLD fMRI: Ridderbusch et al., 2021; SCRs: Zeidan et al., 2012).

However, it is difficult to extract a comprehensive picture from these five studies as they differ substantially in sample size (N = 18–100), paradigm specifications, experimental phases reported, outcome measures, time intervals, and employed reliability measures (see Supplementary file 1).

Given that the predominance of research on group-level generic mechanisms in fear conditioning research, it is even more surprising that, to our knowledge, no study to date has investigated longitudinal reliability at the group level and only few studies have (Fredrikson et al., 1993) targeted internal consistency (i.e., the degree to which all test items capture the same construct, see Table 1). More precisely, longitudinal reliability at the group level indicates the extent to which responses averaged across the group as a whole are stable over time, which is important to establish when investigating basic, generic principles such as the impact of experimental manipulations. Even though it has to be acknowledged that the group average is not necessarily representative of any individual in the group and the same group average may arise from different and even opposite individual responses at both time points in the same group, group-level reliability is important to establish in addition to individual-level reliability. Group-level reliability is relevant not only to work focusing on the understanding of general, generic processes but also for questions about differences between two groups of individuals such as patients vs. controls (e.g., see meta-analyses of Cooper et al., 2022a; Duits et al., 2015). Of note, many fear conditioning paradigms were initially developed to study general group-level processes and to elicit robust group effects (Lonsdorf and Merz, 2017b). Hence, it is important to investigate both group- and individual-level reliability given the challenges of attempts to employ cognitive tasks that were originally designed to produce robust group effects in individual difference research (Elliott et al., 2020; Hedge et al., 2018; Parsons, 2020; Parsons et al., 2019).

As pointed out above, individual-level reliability is a prerequisite for individual-level predictions such as treatment outcomes. Since the different experimental phases of fear conditioning paradigms serve as experimental models for the development, treatment, and relapse of anxiety- and stress-related disorders, it is also an important question whether responding across phases can be reliably predicted at the individual level. Interestingly, it is often implicitly assumed that responding in one experimental phase reliably predicts responding in a subsequent phase (e.g., see Milad et al., 2009; critically discussed in Lonsdorf et al., 2019a) even though empirical evidence is lacking. As a result it has been suggested to routinely ‘correct for responding’ during fear acquisition training when studying performance in later experimental phases such as extinction training or retention/RoF test (critically discussed in Lonsdorf et al., 2019a). However, empirical evidence on this cross-phases predictability (for definition and terminology, see Table 1) is scarce to date.

Evidence from experimental work on cross-phase predictability in rodents and humans is mixed. In rodents, freezing during acquisition training and 24-hrs-delayed extinction training were uncorrelated (Plendl and Wotjak, 2010) and responding during extinction training did not predict extinction retention (i.e., lever-pressing suppression: Bouton et al., 2006; or freezing behavior: Shumake et al., 2014). Similarly, in humans, extinction performance (FPS, SCRs, and US expectancy ratings) did not predict performance at 24-hrs-retention test (Prenoveau et al., 2013). Yet, a computational modeling approach suggests that the mechanism of extinction learning (i.e., the formation of a new extinction memory trace in comparison to an update of the original fear memory trace) predicts the extent of spontaneous recovery in SCRs (Gershman and Hartley, 2015).

Also evidence from work in patient samples is mixed (for a review, see Craske et al., 2008). The extent of fear reduction within therapeutic sessions was unrelated to overall treatment outcome in some studies (Kozak et al., 1988; Pitman et al., 1996; Riley et al., 1995), while others observed an association (Foa et al., 1983). Similarly, significant correlations of fear reduction between therapeutic sessions with treatment outcome were observed for reported distress (Rauch et al., 2004) and heart rate, but not for SCR (Kozak et al., 1988; Lang et al., 1970) and for self-reported fear post treatment, but not at follow-up (Foa et al., 1983). In addition, evidence that responding in different phases is related comes from pharmacological manipulations with the cognitive enhancer D-cycloserine which facilitates learning and/or consolidation. D-cycloserine promoted long-term extinction retention (Rothbaum et al., 2014; Smits et al., 2013a; Smits et al., 2013b) only if within-session learning was achieved.

With this pre-registered study, we follow the call for a stronger appreciation and more systematic investigations of measurement reliability (Zuo et al., 2019). We address longitudinal reliability and internal consistency as well as predictability of cross-phase responding in SCRs, fear ratings, and the BOLD response. For this purpose, we reanalyzed data from 120 participants that underwent a differential fear conditioning paradigm twice (at time points T0 and T1, 6 months apart) – with habituation and acquisition training on day 1 and extinction, reinstatement and reinstatement-test on day 2 to allow for fear memory consolidation prior to extinction. Part of the data have been used previously in method focused work (Kuhn et al., 2022; Lonsdorf et al., 2022; Lonsdorf et al., 2019a; Sjouwerman et al., 2022) and work investigating the association of conditioned responding with brain morphological measures (Ehlers et al., 2020).

Specifically, we (1) estimated internal consistency of SCRs at both time points and (2) systematically assessed longitudinal reliability of SCRs, fear ratings and BOLD fMRI at the individual level by calculating ICCs. This was complemented by investigations of response similarity (SCR and BOLD fMRI) and the degree of overlap of activated voxels at both time points (BOLD fMRI) as additional measurements of longitudinal reliability at the individual level that allow for a more detailed picture than the coarser ICCs (see Table 1 for terminology and definitions). We also (3) assessed whether SCR and BOLD fMRI show longitudinal reliability at the group level. Finally, we (4) investigated if individual level responding during an experimental phase is predictive of individual-level responding during subsequent experimental phases. All hypotheses are tested across different pre-registered data specifications to account for procedural heterogeneity in the literature (see Supplementary file 1): More precisely, we follow a pre-registered multiverse-inspired approach and include (1) responses to the CS+, CS−, US, and CS discrimination, (2) different phase operationalizations, (3) different data transformations none, log-transformed, log-transformed and range-corrected, and (4) ordinally ranked vs. non-ranked data (for justification of these choices, see Table 1). We acknowledge that the specifications used here are not intended to cover all potentially meaningful combinations as in a full multiverse study (Lonsdorf et al., 2022; Sjouwerman et al., 2022; Steegen et al., 2016) but can be viewed as a manyverse (Kuhn et al., 2022) in which we a priori pre-registered a number of meaningful combinations.

For a comprehensive overview of the different reliability measures used here and of the analyses conducted, see Table 1.

Satisfactory internal consistency

To assess internal consistency of SCRs, trials were split into odd and even trials (i.e., odd–even approach), averaged for each individual subject and then correlated (Pearson’s correlation coefficient). This was done separately for each time point and experimental phase. Internal consistency at T0 (see Figure 1A) and T1 (see Figure 1B) of raw SCRs to the CS+ and CS− ranged from 0.54 to 0.85 and for raw SCRs to the US from 0.91 to 0.94 for all phases. In comparison, internal consistency was lower for CS discrimination with values ranging from −0.01 to 0.60. Log-transformation did not impact internal consistency but log-transformation in combination with range correction largely resulted in reduced reliability (see Figure 1—figure supplement 1).

Figure 1 with 8 supplements see all

Download asset Open asset

Higher within- than between-subject similarity in BOLD fMRI but not SCRs

While ICCs provide information on the absolute quantity of longitudinal reliability at the individual level, comparison of within- and between-subject similarity as a complementary measure of longitudinal reliability at the individual level (see Table 1) reflects the extent to which responses in SCR and BOLD activation of one individual at T0 were more similar to themselves at T1 than to other individuals at T1 (see Figures 2 and 3).

Figure 2

Download asset Open asset

Figure 3

Download asset Open asset

SCR

For SCRs, within-subject similarity (i.e., within-subject correlation of trial-by-trial SCR across time points) and between-subject similarity (i.e., correlation of trial-by-trial SCR between one individual at T0 and all other individuals at T1; see Figure 2) did not differ significantly for most data specifications. This was true for CS discrimination (t(64) = 1.78, p = 0.079, d = 0.22) as well as for SCRs to the CS+ (t(61) = 0.84, p = 0.407, d = 0.11) and CS− (t(55) = 1.50, p = 0.138, d = 0.20) during acquisition training and for CS discrimination (t(44) = −0.23, p = 0.823, d = −0.03) and SCRs to the CS+ (t(39) = 0.25, p = 0.801, d = 0.04) during extinction training. This indicates that SCRs of one particular individual at T0 were mostly not more similar to their own SCRs than to those of other individuals at T1. The only exceptions where within-subject similarities were significantly higher than between-subject similarity were SCRs to the US during acquisition training (t(70) = 2.54, p = 0.013, d = 0.30) and to the CS− during extinction training (t(31) = 2.05, p = 0.049, d = 0.36). Note, however, that within-subject similarity had a very wide spread pointing to substantial individual differences (while this variance is removed in calculations of between-subject similarity).

fMRI data

In contrast to what was observed for SCRs, within-subject similarity was significantly higher than between-subject similarity in the whole brain (p < 0.001) and most of the ROIs for fear acquisition training (see Figure 3A and Supplementary file 6). This suggests that while absolute values for similarity might be low, individual brain activation patterns during fear acquisition training at T0 were – at large – still more similar to the same subject’s activation pattern at T1 than to any others at T1. For extinction training, however, no significant differences between within- and between-subject similarity were found for any ROI or the whole brain (all p’s > 0.306; see Figure 3B and Supplementary file 6).

Robust longitudinal reliability at the group level

While longitudinal reliability at the individual level relies on (mean) individual subject responding at both time points, longitudinal reliability at the group level relies on the percentage of explained variance of group averaged trials at T1 by group averaged trials at T0 (i.e., R squared for SCR) or the degree of group level overlap of significant voxels expressed as Dice and Jaccard indices (i.e., BOLD fMRI).

SCR

For acquisition training (see Figure 4A), 40.66% ($F\left(\mathrm{1,11}\right)=7.54$, ${\displaystyle \mathrm{p}=0.019}$), 63.59% ($F\left(\mathrm{1,11}\right)=19.21$, ${\displaystyle \mathrm{p}=0.001}$) and 75.67% ($F\left(\mathrm{1,11}\right)=34.20$, ${\displaystyle \mathrm{p}<0.001}$) of the variance of SCRs at T1 could be explained by SCRs at T0 for CS discrimination, CS+ and CS−, respectively, indicating robust longitudinal reliability of SCRs at the group level for CS responding during acquisition. Interestingly, only 19.53% ($F\left(\mathrm{1,12}\right)=2.91$, ${\displaystyle \mathrm{p}=0.114}$) of the variance of SCRs to the US could be explained. For extinction training, in contrast, only 19.58% ($F\left(\mathrm{1,11}\right)=2.68$, ${\displaystyle \mathrm{p}=0.130}$) and 21.70% ($F\left(\mathrm{1,11}\right)=3.05$, ${\displaystyle \mathrm{p}=0.109}$) of the SCR variance at T1 could be explained by SCRs at T0 for CS discrimination and CS+, respectively, indicating only limited longitudinal reliability at the group level. However, with 67.35% ($F\left(\mathrm{1,11}\right)=22.69$, ${\displaystyle \mathrm{p}=0.001}$) explained variance at T1, longitudinal reliability of SCRs to the CS− appeared to be more robust as compared to CS discrimination and responses to the CS+ (see Figure 4B).

Figure 4 with 1 supplement see all

Download asset Open asset

BOLD fMRI

In stark contrast to the low overlap of individual-level activation (see Table 2A), the overlap at the group level was rather high with 62.00% for the whole brain and up to 89.80% for individual ROIs (i.e., dACC and dlPFC; Jaccard) for CS discrimination during acquisition training (see Table 2B). Similar to what was observed for overlap at the individual level, a much lower overlap for extinction training as compared to acquisition training was observed for the whole brain (5.70% overlap) and all ROIs (all close to zero).

Cross-phases predictability of conditioned responding

Finally, we investigated if responding in any given experimental phase predicted responding in subsequent experimental phases. To this end, simple linear regressions with robust standard errors were computed for both SCRs and fear ratings and all data specifications (see Figure 5 and Supplementary file 7, Supplementary file 8). To approximate these analyses, correlations of patterns of BOLD brain activation between experimental phases were calculated (see Figure 6).

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 6

Download asset Open asset

SCR

Stronger CS discrimination in SCRs during (delayed) fear recall (i.e., first trial of extinction training) was significantly predicted by both average and end-point performance (i.e., last two trials) during acquisition training for most data specifications (Figure 5A, columns 1 and 2). In contrast, average CS discrimination during extinction training was significantly predicted by acquisition training performance only if data were ordinally ranked (columns 3 and 4). Strikingly, all predictions of extinction end-point performance (columns 5 and 6) as well as performance at reinstatement-test (columns 7–11) were non-significant – irrespective of phase operationalizations and data transformation.

The majority of predictions of SCRs to the CS+ and CS− were significant with few exceptions (see white cells in Figure 5A) – irrespective of experimental phases, their operationalization and data transformation. Most non-significant regressions included log-transformed and range corrected data. Strikingly, extinction end-point performance never predicted performance at reinstatement-test – irrespective of data transformation (column 11).

Fear ratings

Higher ratings for the CS+ as well as higher CS discrimination during acquisition training predicted higher CS+ ratings and CS discrimination at fear recall (Figure 5B, columns 1 and 2), extinction training (columns 3 and 4), and at reinstatement-test (columns 7 and 8). Higher responding to the CS+ and higher CS discrimination at fear recall predicted higher responding at reinstatement-test (column 9) – irrespective of data transformations. In contrast, predictions of CS discrimination and CS+ ratings after extinction training were mostly non-significant (columns 5 and 6). Higher CS+ ratings during extinction training significantly predicted higher ratings at reinstatement-test which was not true for CS discrimination (columns 10 and 11).

Higher CS− ratings after acquisition training predicted higher CS− ratings at fear recall as well as after extinction training and CS− ratings after extinction training predicted the performance at reinstatement-test – irrespective of ranking of the data (columns 2, 6, and 11). Furthermore, when based on ordinally ranked data, the difference between ratings prior to and after acquisition predicted CS− ratings at fear recall and CS− ratings after acquisition training predicted the difference between CS− ratings prior to and after extinction training (columns 1 and 4). All other predictions were non-significant.

In sum, all significant predictions observed were positive with weak to moderate associations and indicate that higher responding in preceding phases predicted higher responding in subsequent phases for both SCRs and fear ratings.

BOLD fMRI

In short, all but one association (CS discrimination in the NAcc) was positive, showing that higher BOLD response during acquisition was associated with higher BOLD responding during extinction training (see Figure 6). However, the standardized beta coefficients are mostly below or around 0.3 except for CS+ associations in the dACC, indicating non-substantial associations for all ROIs and CS specifications that were near absent for CS discrimination. Analysis of CS+ and CS− data was included here as the analysis is based on beta maps and not T-maps (as in previous analyses) where a contrast against baseline is not optimal.

In fear conditioning research, little is known about longitudinal reliability (in the literature often referred to as test–retest reliability) for common outcome measures and almost nothing is known about their internal consistency and to what extent predictability across experimental phases is possible.

Here, we aimed to fill this gap and complement traditionally used approaches focusing on ICCs (summarized in Supplementary file 1) with (1) analyses of response similarity, (2) the degree of overlap of individual-level brain activation patterns as well as (3) by exploring longitudinal reliability at the group level in addition to (4) internal consistency across outcome measures.

Moreover, we also directly investigated predictability of responding from one experimental phase to subsequent experimental phases. For all analyses, we followed a multiverse-inspired approach (Parsons, 2020) by taking into account different data specifications.

Overall, longitudinal group-level reliability was robust for SCRs (see Figure 4) and the BOLD response (see Table 2B) while longitudinal individual-level reliability as assessed by ICCs (see Figure 1C–F), and individual-level BOLD activation overlap (see Table 2A) was more limited across outcome measures and data specifications – particularly during extinction training. This is in line with previous work in fear conditioning (Cooper et al., 2022b; Fredrikson et al., 1993; Ridderbusch et al., 2021; Torrents-Rodas et al., 2014; Zeidan et al., 2012) reporting figures for longitudinal individual-level reliability comparable to ours across outcome measures (SCRs, fear ratings, BOLD fMRI) and experimental phases. Importantly, however, it remains a challenge to interpret the results as benchmarks for ICCs are derived from psychometric work on trait self-report measures and it is plausible that what is interpreted as ‘low’ and ‘high’ reliability in experimental work should be substantially lower (Parsons et al., 2019).

Our complementary analyses beyond traditional ICCs indicate that SCRs of one individual at T0 were not more similar to responses of the same individual at T1 than compared to others at T1 (see Figure 2). For BOLD fMRI, however, acquisition-related individual BOLD activation patterns at T0 were more similar to their own activation patterns at T1 than to other individuals’ activation patterns (see Figure 3). This was, however, not the case for extinction. Hence, this may suggest that BOLD fMRI might be more sensitive to detect similarity at individual-level responses within participants than SCRs in our data – maybe due to the dependence on spatial (i.e., voxel-by-voxel) rather than temporal (i.e., trial-by-trial) patterns.

Furthermore, we observed a few differences in longitudinal reliability at the individual level depending on data processing specifications (see also Parsons, 2020). For most data specifications, reliability was slightly higher for log-transformed and range-corrected SCRs (as opposed to raw and only log-transformed data) while – in contrast to what has been shown for other paradigms and outcome measures (Baker et al., 2021; see also https://shiny.york.ac.uk/powercontours/) – an increasing number of trials included in the calculation of ICCs did not generally improve reliability (see Figure 1—figure supplements 3–8). Together, this suggests that longitudinal reliability at the individual level is relatively stable across different data transformations and paradigm specifications (e.g., number of trials within the range used here, i.e., 1 to maximum 14) which is important information facilitating the integration of previous work using different time intervals, reliability indices, and paradigms (see Supplementary file 1; Cooper et al., 2022b; Fredrikson et al., 1993; Ridderbusch et al., 2021; Torrents-Rodas et al., 2014; Zeidan et al., 2012).

In contrast, we observed quite robust longitudinal reliability at the group level for both SCRs (see Figure 4) and BOLD fMRI (see Table 2B) between both time points with substantial (i.e., up to 90%) overlap in group-level BOLD fMRI activation patterns (whole brain and ROI based) as well as substantial (i.e., up to 76%) explained variance at T1 by variance at T0 for SCRs. However, this was generally only true for acquisition but not extinction training. This pattern of higher reliability during acquisition compared to extinction training has been described in the literature (SCRs: Fredrikson et al., 1993; Zeidan et al., 2012) and was also evident in the similarity analyses of BOLD fMRI and the group-level reliability of SCRs. While this pattern did not emerge across all analyses, it appears to be particularly present when examining reliability of CS discrimination as it was the case for BOLD fMRI and as it also emerged in individual-level reliability analyses of CS discrimination in SCRs (internal consistency and ICCs) and fear ratings (ICCs). Since CS discrimination is typically lower during extinction as compared to acquisition training, this restriction of variance potentially resulted in a floor effect which might have lowered the internal consistency and longitudinal reliability of CS discrimination during extinction training.

Reports regarding this discrepancy between group- and individual-level longitudinal reliability were recently highlighted for a number of (classic) experimental paradigms (Fröhner et al., 2019; Hedge et al., 2018; Herting et al., 2018; Plichta et al., 2012; Schümann et al., 2020). Our results add fear conditioning and extinction as assessed by SCRs and BOLD fMRI to this list and have important implications for translational questions aiming for individual-level predictions – particularly since findings obtained at the group level are not necessarily representative for any individual within the group (Fisher et al., 2018).

In addition to these methods-focused insights, we observed significant associations between responding in different experimental phases for SCR (see Figure 5A), fear ratings (see Figure 5B) and BOLD fMRI (see Figure 6) revealing that higher responses in previous phases were generally modestly associated with higher responses in subsequent phases in all outcome measures. However, a remarkable amount of predictions were non-significant – which was particularly true for CS discrimination in SCRs and BOLD fMRI. This may be explained by difference scores (i.e., CS+ minus CS−) being generally less reliable (Infantolino et al., 2018; Lynam et al., 2006) due to a subtraction of meaningful variance (Moriarity and Alloy, 2021) particularly in highly correlated predictors (Thomas and Zumbo, 2012). Especially at the end of the extinction, CS discrimination is low and hence, variance limited. Therefore, floor effects may contribute to the non-significant effects for extinction end-point performance.

Mixed findings in the literature support both the independence of conditioned responding in different experimental phases (Bouton et al., 2006; Plendl and Wotjak, 2010; Prenoveau et al., 2013; Shumake et al., 2014) but also their dependence – particularly in clinical samples (Foa et al., 1983; Rauch et al., 2004; Rothbaum et al., 2014; Smits et al., 2013a; Smits et al., 2013b). These diverging findings in experimental and clinical studies might point toward a translational gap. However, our work may suggest that the strengths of associations between responding in different phases depended on the specific outcome measure and its specifications (e.g., responses specified as CS discrimination, CS+, or CS−). Yet another explanation – in particular for predictions spanning a 24 hrs delay in experimental phases − might be that individual differences in consolidation efficacy (e.g., how efficiently the fear and extinction memories are consolidated after performing acquisition and extinction training, respectively) may underlie differences in predictability. For example, the performance during a retention or RoF test phase is considered to be determined by the strength of the fear and extinction memory, respectively. Memory strength, however, is not only determined by the strength of the initially acquired memory but also by its consolidation (discussed in Lonsdorf et al., 2019b). Thus, as acquisition training preceded the extinction training and reinstatement-test by 24 hrs, it is highly likely that individual differences in consolidation efficacy also impact on performance at test. This has also implications for the common practice of correcting responses during one experimental phase for responding during preceding experimental phases (discussed in Lonsdorf et al., 2019b).

Importantly, together with our observation of robust internal consistency (see Figure 1 and also Fredrikson et al., 1993), this pattern of findings suggests that individual-level predictions at short intervals are plausible but might be more problematic for longer time periods as suggested by the limited stability over time in our data.

Yet, we would like to point out that the values we report may in fact point toward good and not limited longitudinal individual-level reliability as our interpretation is guided by benchmarks that were not developed for experimental data but from psychometric work on trait self-report measures. We acknowledge that the upper bound of maximally observable reliability may differ between both cases of application as empirical neuroscientific research inherently comes with more noise. The problem remains that predictions in fear conditioning paradigms appear to not be meaningful for longer periods of time (~6 months). Thus, a key contribution of our work is that it highlights the need to pay more attention to measurement properties in translational research in general and fear conditioning research specifically (e.g., implement reliability calculations routinely in future studies). To date, it remains an open question what ‘good reliability’ in experimental neuroscientific work actually means (Parsons et al., 2019).

Yet, before discussing implications of our results in detail, some reflections on potential (methodological) reasons for (1) limited individual-level but robust group-level reliability and (2) on the role of time interval lengths deserve attention:

First, the limited longitudinal individual-level reliability might indicate that the fear conditioning paradigm employed here – which is a rather strong paradigm with 100% reinforcement rate – may be better suited for investigations of group effects and to a lesser extent for individual difference questions – potentially due to limited variance between individuals (Hedge et al., 2018; Parsons, 2020; Parsons et al., 2019). However, high reliability appears to be possible in principle, as we can conclude from the robust internal consistency of SCRs that we observed. This speaks against a limited between-subject variance and a general impracticability of the paradigm for individual difference research. Hence, we call for caution and warn against concluding from our report that fear conditioning and our outcome measures (SCRs, BOLD fMRI) are unreliable at the individual level.

Second, limited individual-level but robust group-level longitudinal reliability might be (in part) due to different averaging procedures which impacts error variance (Kennedy et al., 2021). More precisely, compared to individual-level data, group-level data are based on highly aggregated data resulting in generally reduced error variance which increases group-level reliability.

Third, different operationalizations of the same measurement might have different reliabilities (Kragel et al., 2021). For instance, amygdala habituation has been shown to be a more reliable measure than average amygdala activation (Plichta et al., 2014) and more advanced analytical approaches such as intraindividual neural response variability (Månsson et al., 2021) and multivariate imaging techniques Kragel et al., 2021; Marek et al., 2020; Noble et al., 2021; Visser et al., 2021 have been suggested to have better (longitudinal) reliability than more traditional analyses approaches. Similarly, methodological advances (e.g., techniques to adjust the functional organization of the brain across participants, Kong et al., 2021; or hyperalignment, Feilong et al., 2021) in measurement quality and tools may ultimately result in better reliability estimates (DeYoung et al., 2022).

Fourth, as discussed above, caution is warranted as traditional benchmarks for ‘good’ reliability were not developed for experimental work but mainly from psychometric work on trait self-report measures (see above).

Finally, longitudinal reliability refers to measurements obtained under the same conditions and hence it is both plausible and well established that higher reliability is observed at short test–retest intervals (see also Noble et al., 2021; Werner et al., 2022). Longer intervals are more susceptible to true changes of the measurand – for instance due to environmental influences such as seasonality, temperature, hormonal status, or life events (see Specht et al., 2011; Vaidya et al., 2002). Indeed most longitudinal reliability studies in the fMRI field used shorter intervals (<6 months, see Elliott et al., 2020; Noble et al., 2021) than our 6-month interval and hence our results should be conceptualized as longitudinal stability rather than a genuine test–retest reliability. The satisfactory internal consistency speaks against excessive noisiness inherent to our measures as a strong noisiness would also be evident in measurements within one time point and not only emerge across our retest interval. Thus, we rather suggest a true change of the measurand during our retest interval and hence a potentially stronger state than trait dependency.

What do our findings imply? Fear conditioning research has been highlighted as a particularly promising paradigm for the translation of neuroscientific findings into the clinics (Anderson and Insel, 2006; Cooper et al., 2022a; Fullana et al., 2020; Milad and Quirk, 2012) and some of the most pressing translational questions are based on individual-level predictions such as predicting treatment success. Our results, however, suggest that measurement reliability may allow for individual-level predictions for (very) short but potentially less so for longer time intervals (such as our 6 months retest interval). Importantly, however, robust group-level reliability appears to allow for group-level predictions over longer time intervals. This applies to SCRs and BOLD fMRI in our data but note that the latter was not investigated for fear ratings. A potential solution and promising future avenue to make use of both good group-level reliability and individual-level predictions might be the use of homogenous (latent) subgroups characterized by similar response profiles (e.g., rapid, slow or no extinction, Galatzer-Levy et al., 2013a) – to exploit the fact that reliability appears to be higher for more homogenous samples (Gulliksen, 1950).

While general recommendations and helpful discussions on the link between reliability and number of trials (Baker et al., 2021), statistical power (Parsons, 2020), maximally observable correlations (Parsons, 2020), sample and effect size (Hedge et al., 2018; Parsons, 2020) considerations exist, our results highlight the need for field and subdiscipline specific considerations. Our work allows for some initial recommendations and insights. First, we highlight the value of using multiple, more nuanced measures of reliability beyond traditional ICCs (i.e,. similarity, overlap, Fröhner et al., 2019) and second, the relation between number of trials and reliability in an experiment with a learning component (i.e., no increase in reliability with an increasing number of trials). Importantly, our work can also be understood as an empirically based call for action, since more work is needed to allow for clear-cut recommendations, and as a starting point to develop and refine comprehensive guidelines in the future. We also echo the cautionary note of Parsons that ‘estimates of reliability refer to the measurement obtained – in a specific sample and under particular circumstances, including the task parameters’ (cf. Parsons, 2020). Hence, it is important to remember that reliability is a property of a measure that is not fixed and may vary depending on task specifications and samples. In other words, reliability is not a fixed property of the task itself, here fear conditioning.

We argue that we may need to take a (number of) step(s) back and develop paradigms and data processing pipelines explicitly tailored to individual difference research (i.e., correlation) or experimental (i.e., group level) research questions (e.g., Parsons, 2020) and focus more strongly on measurement reliability in experimental work – which has major consequences on effect sizes and statistical power (Elliott et al., 2020). More precisely, multiverse-type investigations (Parsons, 2020; Steegen et al., 2016) that systematically scrutinize the impact of several alternative and equally justifiable processing and analytical decisions in a single dataset (Kuhn et al., 2022; Lonsdorf et al., 2022; Sjouwerman et al., 2022) – as also done here for transformations and number of trials – may be helpful to ultimately achieve this overarching aim. This could be complemented by systematically varying design specifications (Harder, 2020) which are extensively heterogeneous in fear conditioning research (Lonsdorf et al., 2017a). Calibration approaches, as recently suggested Bach et al., 2020 follow a similar aim.

Such work on measurement questions should be included in cognitive-experimental work as a standard practice (Parsons, 2020) and can (often) be explored in a cost and resource effective way in existing data which in the best case are openly available – which, however, requires cross-lab data sharing and data management homogenization plans. Devoting resources and funds to measurement optimization is a valuable investment into the prospect of this field contributing to improved mental health (Moriarity and Alloy, 2021) and to resume the path to successful translation from neuroscience discoveries into clinical applications.

The data that support the findings of this study and the R Markdown files that generate this manuscript are openly available in Zenodo at https://doi.org/10.5281/zenodo.7323547.

1. 1. Klingelhöfer-Jens M
   2. Ehlers MR
   3. Kuhn M
   4. Keyaniyan V
   5. Lonsdorf TB

   (2022) Zenodo

   Robust group- but limited individual-level (longitudinal) reliability and insights into cross-phases response prediction of conditioned fear.

   https://doi.org/10.5281/zenodo.7323547

1. 1. Aldridge VK
   2. Dovey TM
   3. Wade A

   (2017) Assessing test-retest reliability of psychological measures

   European Psychologist 22:207–218.

   https://doi.org/10.1027/1016-9040/a000298

   • Google Scholar
2. 1. Anderson KC
   2. Insel TR

   (2006) The promise of extinction research for the prevention and treatment of anxiety disorders

   Biological Psychiatry 60:319–321.

   https://doi.org/10.1016/j.biopsych.2006.06.022

   • PubMed
   • Google Scholar
3. Software

   1. Andri mult S

   (2021) DescTools: tools for descriptive statistics

   R-Project.

   https://cran.r-project.org/package=DescTools
4. Software

   1. Auguie B

   (2017) GridExtra: miscellaneous functions for “ grid ” graphics

   R-Project.

   https://CRAN.R-project.org/package=gridExtra
5. Software

   1. Aust F
   2. Barth M

   (2020) Papaja: create APA manuscripts with R markdown, version 4.0.3

   Github.

   https://github.com/crsh/papaja
6. 1. Bach DR
   2. Melinščak F
   3. Fleming SM
   4. Voelkle MC

   (2020) Calibrating the experimental measurement of psychological attributes

   Nature Human Behaviour 4:1229–1235.

   https://doi.org/10.1038/s41562-020-00976-8

   • PubMed
   • Google Scholar
7. 1. Baker DH
   2. Vilidaite G
   3. Lygo FA
   4. Smith AK
   5. Flack TR
   6. Gouws AD
   7. Andrews TJ

   (2021) Power contours: optimising sample size and precision in experimental psychology and human neuroscience

   Psychological Methods 26:295–314.

   https://doi.org/10.1037/met0000337

   • PubMed
   • Google Scholar
8. Software

   1. Barth M

   (2022) Tinylabels: lightweight variable labels

   R-Project.

   https://cran.r-project.org/package=tinylabels
9. 1. Bechara A
   2. Tranel D
   3. Damasio H
   4. Adolphs R
   5. Rockland C
   6. Damasio AR

   (1995) Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans

   Science 269:1115–1118.

   https://doi.org/10.1126/science.7652558

   • PubMed
   • Google Scholar
10. Software

    1. Behrendt S

    (2014) Lm.beta: add standardized regression coefficients to lm-objects

    R-Project.

    https://CRAN.R-project.org/package=lm.beta
11. 1. Bernstein DP
    2. Stein JA
    3. Newcomb MD
    4. Walker E
    5. Pogge D
    6. Ahluvalia T
    7. Stokes J
    8. Handelsman L
    9. Medrano M
    10. Desmond D
    11. Zule W

    (2003) Development and validation of a brief screening version of the childhood trauma questionnaire

    Child Abuse & Neglect 27:169–190.

    https://doi.org/10.1016/s0145-2134(02)00541-0

    • PubMed
    • Google Scholar
12. 1. Boucsein W
    2. Fowles DC
    3. Grimnes S
    4. Ben-Shakhar G
    5. roth WT
    6. Dawson ME
    7. Filion DL
    8. Society for Psychophysiological Research Ad Hoc Committee on Electrodermal Measures

    (2012) Publication recommendations for electrodermal measurements

    Psychophysiology 49:1017–1034.

    https://doi.org/10.1111/j.1469-8986.2012.01384.x

    • PubMed
    • Google Scholar
13. 1. Bouton ME

    (2004) Context and behavioral processes in extinction: table 1

    Learning & Memory 11:485–494.

    https://doi.org/10.1101/lm.78804

    • PubMed
    • Google Scholar
14. 1. Bouton ME
    2. García-Gutiérrez A
    3. Zilski J
    4. Moody EW

    (2006) Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse

    Behaviour Research and Therapy 44:983–994.

    https://doi.org/10.1016/j.brat.2005.07.007

    • PubMed
    • Google Scholar
15. Preprint

    1. Constantin AE
    2. Patil I

    (2021) Ggsignif: R Package for Displaying Significance Brackets for “Ggplot2.”

    PsyArXiv.

    https://doi.org/10.31234/osf.io/7awm6

    • Google Scholar
16. 1. Cooper SE
    2. van Dis EAM
    3. Hagenaars MA
    4. Krypotos AM
    5. Nemeroff CB
    6. Lissek S
    7. Engelhard IM
    8. Dunsmoor JE

    (2022a) A meta-analysis of conditioned fear generalization in anxiety-related disorders

    Neuropsychopharmacology: Official Publication of the American College of Neuropsychopharmacology 47:1652–1661.

    https://doi.org/10.1038/s41386-022-01332-2

    • PubMed
    • Google Scholar
17. Preprint

    1. Cooper SE
    2. Dunsmoor JE
    3. Koval K
    4. Pino E
    5. Steinman S

    (2022b) Test-Retest Reliability of Human Threat Conditioning and Generalization

    PsyArXiv.

    https://doi.org/10.31234/osf.io/84uqz

    • Google Scholar
18. 1. Craske MG
    2. Kircanski K
    3. Zelikowsky M
    4. Mystkowski J
    5. Chowdhury N
    6. Baker A

    (2008) Optimizing inhibitory learning during exposure therapy

    Behaviour Research and Therapy 46:5–27.

    https://doi.org/10.1016/j.brat.2007.10.003

    • PubMed
    • Google Scholar
19. 1. Cronbach LJ
    2. Meehl PE

    (1955) Construct validity in psychological tests

    Psychological Bulletin 52:281–302.

    https://doi.org/10.1037/h0040957

    • PubMed
    • Google Scholar
20. Book

    1. Dancey CP
    2. Reidy J

    (2007)

    Statistics without Maths for Psychology: Using SPSS for Windows

    Harlow, England ; New York: Pearson/Prentice Hall.

    • Google Scholar
21. 1. Desikan RS
    2. Ségonne F
    3. Fischl B
    4. Quinn BT
    5. Dickerson BC
    6. Blacker D
    7. Buckner RL
    8. Dale AM
    9. Maguire RP
    10. Hyman BT
    11. Albert MS
    12. Killiany RJ

    (2006) An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest

    NeuroImage 31:968–980.

    https://doi.org/10.1016/j.neuroimage.2006.01.021

    • PubMed
    • Google Scholar
22. Preprint

    1. DeYoung CG
    2. Sassenberg T
    3. Abend R
    4. Allen T
    5. Beaty R
    6. Bellgrove M
    7. Blain SD
    8. Bzdok D
    9. Chavez R
    10. Engel SA
    11. Ma F
    12. Fornito A
    13. Genç E
    14. Goghari V
    15. Grazioplene R
    16. Hanson JL
    17. Haxby JV
    18. Hilger K
    19. Homan P
    20. Joyner K
    21. Kaczkurkin AN
    22. Latzman RD
    23. Martin EA
    24. Passamonti L
    25. Pickering A
    26. Safron A
    27. Servaas M
    28. Smillie LD
    29. Spreng RN
    30. Tiego J
    31. Viding E
    32. Wacker J

    (2022) Reproducible Between-Person Brain-Behavior Associations Do Not Always Require Thousands of Individuals

    PsyArXiv.

    https://doi.org/10.31234/osf.io/sfnmk

    • Google Scholar
23. 1. Duits P
    2. Cath DC
    3. Lissek S
    4. Hox JJ
    5. Hamm AO
    6. Engelhard IM
    7. van den Hout MA
    8. Baas JMP

    (2015) Updated meta-analysis of classical fear conditioning in the anxiety disorders

    Depression and Anxiety 32:239–253.

    https://doi.org/10.1002/da.22353

    • PubMed
    • Google Scholar
24. 1. Dunsmoor JE
    2. Cisler JM
    3. Fonzo GA
    4. Creech SK
    5. Nemeroff CB

    (2022) Laboratory models of post-traumatic stress disorder: the elusive bridge to translation

    Neuron 110:1754–1776.

    https://doi.org/10.1016/j.neuron.2022.03.001

    • PubMed
    • Google Scholar
25. 1. Ehlers MR
    2. Nold J
    3. Kuhn M
    4. Klingelhöfer-Jens M
    5. Lonsdorf TB

    (2020) Revisiting potential associations between brain morphology, fear acquisition and extinction through new data and a literature review

    Scientific Reports 10:19894.

    https://doi.org/10.1038/s41598-020-76683-1

    • PubMed
    • Google Scholar
26. 1. Elliott ML
    2. Knodt AR
    3. Ireland D
    4. Morris ML
    5. Poulton R
    6. Ramrakha S
    7. Sison ML
    8. Moffitt TE
    9. Caspi A
    10. Hariri AR

    (2020) What is the test-retest reliability of common task-functional MRI measures? new empirical evidence and a meta-analysis

    Psychological Science 31:792–806.

    https://doi.org/10.1177/0956797620916786

    • PubMed
    • Google Scholar
27. 1. Essau CA
    2. Lewinsohn PM
    3. Lim JX
    4. Ho MR
    5. Rohde P

    (2018) Incidence, recurrence and comorbidity of anxiety disorders in four major developmental stages

    Journal of Affective Disorders 228:248–253.

    https://doi.org/10.1016/j.jad.2017.12.014

    • PubMed
    • Google Scholar
28. 1. Feilong M
    2. Guntupalli JS
    3. Haxby JV

    (2021) The neural basis of intelligence in fine-grained cortical topographies

    eLife 10:e64058.

    https://doi.org/10.7554/eLife.64058

    • PubMed
    • Google Scholar
29. 1. Fisher AJ
    2. Medaglia JD
    3. Jeronimus BF

    (2018) Lack of group-to-individual generalizability is a threat to human subjects research

    PNAS 115:E6106–E6115.

    https://doi.org/10.1073/pnas.1711978115

    • PubMed
    • Google Scholar
30. 1. Foa EB
    2. Grayson JB
    3. Steketee GS
    4. Doppelt HG
    5. Turner RM
    6. Latimer PR

    (1983) Success and failure in the behavioral treatment of obsessive-compulsives

    Journal of Consulting and Clinical Psychology 51:287–297.

    https://doi.org/10.1037//0022-006x.51.2.287

    • PubMed
    • Google Scholar
31. Website

    1. Fox J
    2. Weisberg S

    (2019) An R companion to applied regression (Third)

    Accessed May 17, 2020.

    https://socialsciences.mcmaster.ca/jfox/Books/Companion/
32. Software

    1. Fox J
    2. Weisberg S
    3. Price B

    (2020) CarData: companion to applied regression data sets

    R-Project.

    https://CRAN.R-project.org/package=carData
33. 1. Fredrikson M
    2. Annas P
    3. Georgiades A
    4. Hursti T
    5. Tersman Z

    (1993) Internal consistency and temporal stability of classically conditioned skin conductance responses

    Biological Psychology 35:153–163.

    https://doi.org/10.1016/0301-0511(93)90011-v

    • PubMed
    • Google Scholar
34. 1. Fröhner JH
    2. Teckentrup V
    3. Smolka MN
    4. Kroemer NB

    (2019) Addressing the reliability fallacy in fMRI: similar group effects may arise from unreliable individual effects

    NeuroImage 195:174–189.

    https://doi.org/10.1016/j.neuroimage.2019.03.053

    • PubMed
    • Google Scholar
35. 1. Fullana MA
    2. Harrison BJ
    3. Soriano-Mas C
    4. Vervliet B
    5. Cardoner N
    6. Àvila-Parcet A
    7. Radua J

    (2016) Neural signatures of human fear conditioning: an updated and extended meta-analysis of fMRI studies

    Molecular Psychiatry 21:500–508.

    https://doi.org/10.1038/mp.2015.88

    • PubMed
    • Google Scholar
36. 1. Fullana MA
    2. Dunsmoor JE
    3. Schruers KRJ
    4. Savage HS
    5. Bach DR
    6. Harrison BJ

    (2020) Human fear conditioning: from neuroscience to the clinic

    Behaviour Research and Therapy 124:103528.

    https://doi.org/10.1016/j.brat.2019.103528

    • Google Scholar
37. 1. Galatzer-Levy IR
    2. Bonanno GA
    3. Bush DEA
    4. LeDoux JE

    (2013a) Heterogeneity in threat extinction learning: substantive and methodological considerations for identifying individual difference in response to stress

    Frontiers in Behavioral Neuroscience 7:55.

    https://doi.org/10.3389/fnbeh.2013.00055

    • Google Scholar
38. 1. Galatzer-Levy IR
    2. Bryant RA

    (2013b) 636,120 ways to have posttraumatic stress disorder

    Perspectives on Psychological Science 8:651–662.

    https://doi.org/10.1177/1745691613504115

    • PubMed
    • Google Scholar
39. 1. Gershman SJ
    2. Hartley CA

    (2015) Individual differences in learning predict the return of fear

    Learning & Behavior 43:243–250.

    https://doi.org/10.3758/s13420-015-0176-z

    • PubMed
    • Google Scholar
40. Software

    1. Gohel D

    (2021) Flextable: functions for tabular reporting, version 0.8.1

    R-Project.

    https://CRAN.R-project.org/package=flextable
41. Software

    1. Gohel D
    2. Ross N

    (2022) officedown: Enhanced 'R Markdown' format for 'Word' and 'PowerPoint', version 0.2.4

    CRAN.

    https://CRAN.R-project.org/package=officedown
42. 1. Graham BM
    2. Milad MR

    (2011) The study of fear extinction: implications for anxiety disorders

    American Journal of Psychiatry 168:1255–1265.

    https://doi.org/10.1176/appi.ajp.2011.11040557

    • PubMed
    • Google Scholar
43. Software

    1. Gromer D

    (2020) Apa: format outputs of statistical tests according to APA guidelines

    R-Project.

    https://CRAN.R-project.org/package=apa
44. Book

    1. Gulliksen H

    (1950)

    Effect of group heterogeneity on test reliability

    In: Gulliksen H, editors. Theory of Mental Tests. Hoboken, NJ: Wiley. pp. 108–127.

    • Google Scholar
45. 1. Haaker J
    2. Golkar A
    3. Hermans D
    4. Lonsdorf TB

    (2014) A review on human reinstatement studies: an overview and methodological challenges

    Learning & Memory 21:424–440.

    https://doi.org/10.1101/lm.036053.114

    • PubMed
    • Google Scholar
46. 1. Harder JA

    (2020) The multiverse of methods: extending the multiverse analysis to address data-collection decisions

    Perspectives on Psychological Science 15:1158–1177.

    https://doi.org/10.1177/1745691620917678

    • PubMed
    • Google Scholar
47. 1. Häuser W
    2. Schmutzer G
    3. Brähler E
    4. Glaesmer H

    (2011) Maltreatment in childhood and adolescence: results from a survey of a representative sample of the german population

    Deutsches Arzteblatt International 108:287–294.

    https://doi.org/10.3238/arztebl.2011.0287

    • PubMed
    • Google Scholar
48. 1. Hayes AF
    2. Cai L

    (2007) Using heteroskedasticity-consistent standard error estimators in OLS regression: an introduction and software implementation

    Behavior Research Methods 39:709–722.

    https://doi.org/10.3758/bf03192961

    • PubMed
    • Google Scholar
49. 1. Hedge C
    2. Powell G
    3. Sumner P

    (2018) The reliability paradox: why robust cognitive tasks do not produce reliable individual differences

    Behavior Research Methods 50:1166–1186.

    https://doi.org/10.3758/s13428-017-0935-1

    • PubMed
    • Google Scholar
50. 1. Herting MM
    2. Gautam P
    3. Chen Z
    4. Mezher A
    5. Vetter NC

    (2018) Test-Retest reliability of longitudinal task-based fMRI: implications for developmental studies

    Developmental Cognitive Neuroscience 33:17–26.

    https://doi.org/10.1016/j.dcn.2017.07.001

    • PubMed
    • Google Scholar
51. 1. Infantolino ZP
    2. Luking KR
    3. Sauder CL
    4. Curtin JJ
    5. Hajcak G

    (2018) Robust is not necessarily reliable: from within-subjects fMRI contrasts to between-subjects comparisons

    NeuroImage 173:146–152.

    https://doi.org/10.1016/j.neuroimage.2018.02.024

    • PubMed
    • Google Scholar
52. 1. Kalisch R
    2. Korenfeld E
    3. Stephan KE
    4. Weiskopf N
    5. Seymour B
    6. Dolan RJ

    (2006) Context-Dependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network

    The Journal of Neuroscience 26:9503–9511.

    https://doi.org/10.1523/JNEUROSCI.2021-06.2006

    • PubMed
    • Google Scholar
53. Software

    1. Kassambara A

    (2020) Ggpubr: ’ ggplot2 ’ based publication ready plots

    R-Project.

    https://CRAN.R-project.org/package=ggpubr
54. Preprint

    1. Kennedy JT
    2. Harms MP
    3. Korucuoglu O
    4. Astafiev SV
    5. Barch DM
    6. Thompson WK
    7. Bjork JM
    8. Anokhin AP

    (2021) Reliability and Stability Challenges in ABCD Task FMRI Data

    bioRxiv.

    https://doi.org/10.1101/2021.10.08.463750

    • Google Scholar
55. Book

    1. Kline P

    (2013) Handbook of Psychological Testing

    Routledge.

    https://doi.org/10.4324/9781315812274

    • Google Scholar
56. 1. Kong R
    2. Yang Q
    3. Gordon E
    4. Xue A
    5. Yan X
    6. Orban C
    7. Zuo XN
    8. Spreng N
    9. Ge T
    10. Holmes A
    11. Eickhoff S
    12. Yeo BTT

    (2021) Individual-Specific areal-level parcellations improve functional connectivity prediction of behavior

    Cerebral Cortex 31:4477–4500.

    https://doi.org/10.1093/cercor/bhab101

    • PubMed
    • Google Scholar
57. 1. Koo TK
    2. Li MY

    (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research

    Journal of Chiropractic Medicine 15:155–163.

    https://doi.org/10.1016/j.jcm.2016.02.012

    • PubMed
    • Google Scholar
58. 1. Kozak MJ
    2. Foa EB
    3. Steketee G

    (1988) Process and outcome of exposure treatment with obsessive-compulsives: psychophysiological indicators of emotional processing

    Behavior Therapy 19:157–169.

    https://doi.org/10.1016/S0005-7894(88)80039-X

    • Google Scholar
59. 1. Kragel PA
    2. Han X
    3. Kraynak TE
    4. Gianaros PJ
    5. Wager TD

    (2021) Functional MRI can be highly reliable, but it depends on what you measure: a commentary on Elliott et al. (2020)

    Psychological Science 32:622–626.

    https://doi.org/10.1177/0956797621989730

    • PubMed
    • Google Scholar
60. 1. Kriegeskorte N
    2. Mur M
    3. Bandettini P

    (2008) Representational similarity analysis-connecting the branches of systems neuroscience

    Frontiers in Systems Neuroscience 2:4.

    https://doi.org/10.3389/neuro.06.004.2008

    • PubMed
    • Google Scholar
61. 1. Kuhn M
    2. Gerlicher AMV
    3. Lonsdorf TB

    (2022) Navigating the manyverse of skin conductance response quantification approaches-a direct comparison of trough-to-peak, baseline correction, and model-based approaches in ledalab and pspm

    Psychophysiology 59:e14058.

    https://doi.org/10.1111/psyp.14058

    • PubMed
    • Google Scholar
62. 1. Lang PJ
    2. Melamed BG
    3. Hart J

    (1970) A psychophysiological analysis of fear modification using an automated desensitization procedure

    Journal of Abnormal Psychology 76:220–234.

    https://doi.org/10.1037/h0029875

    • PubMed
    • Google Scholar
63. 1. Levine DW
    2. Dunlap WP

    (1982) Power of the F test with skewed data: should one transform or not?

    Psychological Bulletin 92:272–280.

    https://doi.org/10.1037/0033-2909.92.1.272

    • Google Scholar
64. 1. Lonsdorf TB
    2. Haaker J
    3. Kalisch R

    (2014) Long-Term expression of human contextual fear and extinction memories involves amygdala, hippocampus and ventromedial prefrontal cortex: a reinstatement study in two independent samples

    Social Cognitive and Affective Neuroscience 9:1973–1983.

    https://doi.org/10.1093/scan/nsu018

    • PubMed
    • Google Scholar
65. 1. Lonsdorf TB
    2. Menz MM
    3. Andreatta M
    4. Fullana MA
    5. Golkar A
    6. Haaker J
    7. Heitland I
    8. Hermann A
    9. Kuhn M
    10. Kruse O
    11. Meir Drexler S
    12. Meulders A
    13. Nees F
    14. Pittig A
    15. Richter J
    16. Römer S
    17. Shiban Y
    18. Schmitz A
    19. Straube B
    20. Vervliet B
    21. Wendt J
    22. Baas JMP
    23. Merz CJ

    (2017a) Don ’ T fear “ fear conditioning ”: methodological considerations for the design and analysis of studies on human fear acquisition, extinction, and return of fear

    Neuroscience and Biobehavioral Reviews 77:247–285.

    https://doi.org/10.1016/j.neubiorev.2017.02.026

    • PubMed
    • Google Scholar
66. 1. Lonsdorf TB
    2. Merz CJ

    (2017b) More than just noise: inter-individual differences in fear acquisition, extinction and return of fear in humans-biological, experiential, temperamental factors, and methodological pitfalls

    Neuroscience and Biobehavioral Reviews 80:703–728.

    https://doi.org/10.1016/j.neubiorev.2017.07.007

    • PubMed
    • Google Scholar
67. 1. Lonsdorf TB
    2. Klingelhöfer-Jens M
    3. Andreatta M
    4. Beckers T
    5. Chalkia A
    6. Gerlicher A
    7. Jentsch VL
    8. Meir Drexler S
    9. Mertens G
    10. Richter J
    11. Sjouwerman R
    12. Wendt J
    13. Merz CJ

    (2019a) Navigating the garden of forking paths for data exclusions in fear conditioning research

    eLife 8:e52465.

    https://doi.org/10.7554/eLife.52465

    • PubMed
    • Google Scholar
68. 1. Lonsdorf TB
    2. Merz CJ
    3. Fullana MA

    (2019b) Fear extinction retention: is it what we think it is?

    Biological Psychiatry 85:1074–1082.

    https://doi.org/10.1016/j.biopsych.2019.02.011

    • PubMed
    • Google Scholar
69. 1. Lonsdorf TB
    2. Gerlicher A
    3. Klingelhöfer-Jens M
    4. Krypotos AM

    (2022) Multiverse analyses in fear conditioning research

    Behaviour Research and Therapy 153:104072.

    https://doi.org/10.1016/j.brat.2022.104072

    • PubMed
    • Google Scholar
70. 1. Lykken DT
    2. Venables PH

    (1971) Direct measurement of skin conductance: a proposal for standardization

    Psychophysiology 8:656–672.

    https://doi.org/10.1111/j.1469-8986.1971.tb00501.x

    • PubMed
    • Google Scholar
71. 1. Lykken DT

    (1972) Range correction applied to heart rate and to GSR data

    Psychophysiology 9:373–379.

    https://doi.org/10.1111/j.1469-8986.1972.tb03222.x

    • PubMed
    • Google Scholar
72. 1. Lynam DR
    2. Hoyle RH
    3. Newman JP

    (2006) The perils of partialling: cautionary tales from aggression and psychopathy

    Assessment 13:328–341.

    https://doi.org/10.1177/1073191106290562

    • PubMed
    • Google Scholar
73. 1. Maitra R

    (2010) A re-defined and generalized percent-overlap-of-activation measure for studies of fmri reproducibility and its use in identifying outlier activation maps

    NeuroImage 50:124–135.

    https://doi.org/10.1016/j.neuroimage.2009.11.070

    • PubMed
    • Google Scholar
74. 1. Månsson KNT
    2. Waschke L
    3. Manzouri A
    4. Furmark T
    5. Fischer H
    6. Garrett DD

    (2021) Moment-to-moment brain signal variability reliably predicts psychiatric treatment outcome

    Biological Psychiatry 91:658–666.

    https://doi.org/10.1016/j.biopsych.2021.09.026

    • PubMed
    • Google Scholar
75. Preprint

    1. Marek S
    2. Tervo-Clemmens B
    3. Calabro FJ
    4. Montez DF
    5. Kay BP
    6. Hatoum AS
    7. Dosenbach NUF

    (2020) Towards Reproducible Brain-Wide Association Studies

    bioRxiv.

    https://doi.org/10.1101/2020.08.21.257758

    • Google Scholar
76. Software

    1. MATLAB

    (2016) Matlab, natick, version 0.1

    The MathWorks, Inc.

    https://www.mathworks.com/company/jobs/resources/locations/us-natick.html
77. Software

    1. MATLAB

    (2019) Matlab, sherborn, version 4.1

    The MathWorks, Inc.

    https://www.indeed.com/jobs?q=The+Mathworks&l=Sherborn,+MA&redirected=1
78. 1. Milad MR
    2. Wright CI
    3. Orr SP
    4. Pitman RK
    5. Quirk GJ
    6. Rauch SL

    (2007) Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert

    Biological Psychiatry 62:446–454.

    https://doi.org/10.1016/j.biopsych.2006.10.011

    • PubMed
    • Google Scholar
79. 1. Milad MR
    2. Pitman RK
    3. Ellis CB
    4. Gold AL
    5. Shin LM
    6. Lasko NB
    7. Zeidan MA
    8. Handwerger K
    9. Orr SP
    10. Rauch SL

    (2009) Neurobiological basis of failure to recall extinction memory in posttraumatic stress disorder

    Biological Psychiatry 66:1075–1082.

    https://doi.org/10.1016/j.biopsych.2009.06.026

    • PubMed
    • Google Scholar
80. 1. Milad MR
    2. Quirk GJ

    (2012) Fear extinction as a model for translational neuroscience: ten years of progress

    Annual Review of Psychology 63:129–151.

    https://doi.org/10.1146/annurev.psych.121208.131631

    • PubMed
    • Google Scholar
81. 1. Moriarity DP
    2. Alloy LB

    (2021) Back to basics: the importance of measurement properties in biological psychiatry

    Neuroscience and Biobehavioral Reviews 123:72–82.

    https://doi.org/10.1016/j.neubiorev.2021.01.008

    • PubMed
    • Google Scholar
82. Software

    1. Müller K

    (2020) Here: a simpler way to find your files

    R-Project.

    https://CRAN.R-project.org/package=here
83. 1. Myers KM
    2. Davis M

    (2007) Mechanisms of fear extinction

    Molecular Psychiatry 12:120–150.

    https://doi.org/10.1038/sj.mp.4001939

    • PubMed
    • Google Scholar
84. 1. Ney LJ
    2. Laing PAF
    3. Steward T
    4. Zuj DV
    5. Dymond S
    6. Felmingham KL

    (2020) Inconsistent analytic strategies reduce robustness in fear extinction via skin conductance response

    Psychophysiology 57:11.

    https://doi.org/10.1111/psyp.13650

    • Google Scholar
85. 1. Ney LJ
    2. Laing PAF
    3. Steward T
    4. Zuj DV
    5. Dymond S
    6. Harrison B
    7. Graham B
    8. Felmingham KL

    (2022) Methodological implications of sample size and extinction gradient on the robustness of fear conditioning across different analytic strategies

    PLOS ONE 17:e0268814.

    https://doi.org/10.1371/journal.pone.0268814

    • PubMed
    • Google Scholar
86. 1. Nieuwenhuys R

    (2012) The insular cortex

    Progress in Brain Research 195:123–163.

    https://doi.org/10.1016/B978-0-444-53860-4.00007-6

    • Google Scholar
87. 1. Noble S
    2. Scheinost D
    3. Constable RT

    (2021) A guide to the measurement and interpretation of fMRI test-retest reliability

    Current Opinion in Behavioral Sciences 40:27–32.

    https://doi.org/10.1016/j.cobeha.2020.12.012

    • PubMed
    • Google Scholar
88. 1. Nosek BA
    2. Ebersole CR
    3. DeHaven AC
    4. Mellor DT

    (2018) The preregistration revolution

    PNAS 115:2600–2606.

    https://doi.org/10.1073/pnas.1708274114

    • PubMed
    • Google Scholar
89. 1. Parsons S
    2. Kruijt AW
    3. Fox E

    (2019) Psychological science needs a standard practice of reporting the reliability of cognitive-behavioral measurements

    Advances in Methods and Practices in Psychological Science 2:378–395.

    https://doi.org/10.1177/2515245919879695

    • Google Scholar
90. Preprint

    1. Parsons S

    (2020) Exploring Reliability Heterogeneity with Multiverse Analyses: Data Processing Decisions Unpredictably Influence Measurement Reliability

    PsyArXiv.

    https://doi.org/10.31234/osf.io/y6tcz

    • Google Scholar
91. Software

    1. Pedersen TL

    (2020) Patchwork: the composer of plots

    R-Project.

    https://CRAN.R-project.org/package=patchwork
92. 1. Pitman RK
    2. Orr SP
    3. Altman B
    4. Longpre RE
    5. Poiré RE
    6. Macklin ML
    7. Michaels MJ
    8. Steketee GS

    (1996) Emotional processing and outcome of imaginal flooding therapy in Vietnam veterans with chronic posttraumatic stress disorder

    Comprehensive Psychiatry 37:409–418.

    https://doi.org/10.1016/s0010-440x(96)90024-3

    • PubMed
    • Google Scholar
93. 1. Plendl W
    2. Wotjak CT

    (2010) Dissociation of within- and between-session extinction of conditioned fear

    The Journal of Neuroscience 30:4990–4998.

    https://doi.org/10.1523/JNEUROSCI.6038-09.2010

    • PubMed
    • Google Scholar
94. 1. Plichta MM
    2. Schwarz AJ
    3. Grimm O
    4. Morgen K
    5. Mier D
    6. Haddad L
    7. Gerdes ABM
    8. Sauer C
    9. Tost H
    10. Esslinger C
    11. Colman P
    12. Wilson F
    13. Kirsch P
    14. Meyer-Lindenberg A

    (2012) Test-Retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery

    NeuroImage 60:1746–1758.

    https://doi.org/10.1016/j.neuroimage.2012.01.129

    • PubMed
    • Google Scholar
95. 1. Plichta MM
    2. Grimm O
    3. Morgen K
    4. Mier D
    5. Sauer C
    6. Haddad L
    7. Tost H
    8. Esslinger C
    9. Kirsch P
    10. Schwarz AJ
    11. Meyer-Lindenberg A

    (2014) Amygdala habituation: a reliable fMRI phenotype

    NeuroImage 103:383–390.

    https://doi.org/10.1016/j.neuroimage.2014.09.059

    • PubMed
    • Google Scholar
96. 1. Prenoveau JM
    2. Craske MG
    3. Liao B
    4. Ornitz EM

    (2013) Human fear conditioning and extinction: timing is everything…or is it?

    Biological Psychology 92:59–68.

    https://doi.org/10.1016/j.biopsycho.2012.02.005

    • PubMed
    • Google Scholar
97. Software

    1. Presentation software

    (2010) Presentation software

    Neurobehavioral Systems, Inc.

    https://www.neurobs.com/
98. 1. Rachman S

    (1989) The return of fear: review and prospect

    Clinical Psychology Review 9:147–168.

    https://doi.org/10.1016/0272-7358(89)90025-1

    • Google Scholar
99. 1. Rauch SAM
    2. Foa EB
    3. Furr JM
    4. Filip JC

    (2004) Imagery vividness and perceived anxious arousal in prolonged exposure treatment for PTSD

    Journal of Traumatic Stress 17:461–465.

    https://doi.org/10.1007/s10960-004-5794-8

    • PubMed
    • Google Scholar
100. Software

     1. R Development Core Team

     (2020) R: a language and environment for statistical computing

     R Foundation for Statistical Computing, Vienna, Austria.

     https://www.r-project.org/index.html
101. 1. Revelle W

     (1979) Hierarchical cluster analysis and the internal structure of tests

     Multivariate Behavioral Research 14:57–74.

     https://doi.org/10.1207/s15327906mbr1401_4

     • PubMed
     • Google Scholar
102. Software

     1. Revelle W

     (2020) Psych: procedures for psychological, psychometric, and personality research

     R-Project.

     https://CRAN.R-project.org/package=psych
103. 1. Ridderbusch IC
     2. Wroblewski A
     3. Yang Y
     4. Richter J
     5. Hollandt M
     6. Hamm AO
     7. Wittchen HU
     8. Ströhle A
     9. Arolt V
     10. Margraf J
     11. Lueken U
     12. Herrmann MJ
     13. Kircher T
     14. Straube B

     (2021) Neural adaptation of cingulate and insular activity during delayed fear extinction: a replicable pattern across assessment sites and repeated measurements

     NeuroImage 237:118157.

     https://doi.org/10.1016/j.neuroimage.2021.118157

     • PubMed
     • Google Scholar
104. 1. Riley WT
     2. McCormick MGF
     3. Simon EM
     4. Stack K
     5. Pushkin Y
     6. Overstreet MM
     7. Carmona JJ
     8. Magakian C

     (1995) Effects of alprazolam dose on the induction and habituation processes during behavioral panic induction treatment

     Journal of Anxiety Disorders 9:217–227.

     https://doi.org/10.1016/0887-6185(95)00003-7

     • Google Scholar
105. 1. Rothbaum BO
     2. Price M
     3. Jovanovic T
     4. Norrholm SD
     5. Gerardi M
     6. Dunlop B
     7. Davis M
     8. Bradley B
     9. Duncan EJ
     10. Rizzo A
     11. Ressler KJ

     (2014) A randomized, double-blind evaluation of D-cycloserine or alprazolam combined with virtual reality exposure therapy for posttraumatic stress disorder in Iraq and Afghanistan war veterans

     The American Journal of Psychiatry 171:640–648.

     https://doi.org/10.1176/appi.ajp.2014.13121625

     • PubMed
     • Google Scholar
106. 1. Scharfenort R
     2. Menz M
     3. Lonsdorf TB

     (2016) Adversity-induced relapse of fear: neural mechanisms and implications for relapse prevention from a study on experimentally induced return-of-fear following fear conditioning and extinction

     Translational Psychiatry 6:e858.

     https://doi.org/10.1038/tp.2016.126

     • PubMed
     • Google Scholar
107. 1. Schümann D
     2. Joue G
     3. Jordan P
     4. Bayer J
     5. Sommer T

     (2020) Test-Retest reliability of the emotional enhancement of memory

     Memory 28:49–59.

     https://doi.org/10.1080/09658211.2019.1679837

     • PubMed
     • Google Scholar
108. 1. Seel NM

     (2012)

     Encyclopedia of the Sciences of Learning

     Rescorla-Wagner model, Encyclopedia of the Sciences of Learning, Springer US, 10.1007/978-1-4419-1428-6_2377.

     • Google Scholar
109. 1. Shrout PE
     2. Fleiss JL

     (1979) Intraclass correlations: uses in assessing rater reliability

     Psychological Bulletin 86:420–428.

     https://doi.org/10.1037//0033-2909.86.2.420

     • PubMed
     • Google Scholar
110. 1. Shumake J
     2. Furgeson-Moreira S
     3. Monfils MH

     (2014) Predictability and heritability of individual differences in fear learning

     Animal Cognition 17:1207–1221.

     https://doi.org/10.1007/s10071-014-0752-1

     • PubMed
     • Google Scholar
111. 1. Sjouwerman R
     2. Lonsdorf TB

     (2019) Latency of skin conductance responses across stimulus modalities

     Psychophysiology 56:e13307.

     https://doi.org/10.1111/psyp.13307

     • PubMed
     • Google Scholar
112. 1. Sjouwerman R
     2. Illius S
     3. Kuhn M
     4. Lonsdorf TB

     (2022) A data multiverse analysis investigating non-model based SCR quantification approaches

     Psychophysiology 1:e14130.

     https://doi.org/10.1111/psyp.14130

     • PubMed
     • Google Scholar
113. 1. Smits JAJ
     2. Rosenfield D
     3. Otto MW
     4. Marques L
     5. Davis ML
     6. Meuret AE
     7. Simon NM
     8. Pollack MH
     9. Hofmann SG

     (2013a) D-Cycloserine enhancement of exposure therapy for social anxiety disorder depends on the success of exposure sessions

     Journal of Psychiatric Research 47:1455–1461.

     https://doi.org/10.1016/j.jpsychires.2013.06.020

     • PubMed
     • Google Scholar
114. 1. Smits JAJ
     2. Rosenfield D
     3. Otto MW
     4. Powers MB
     5. Hofmann SG
     6. Telch MJ
     7. Pollack MH
     8. Tart CD

     (2013b) D-Cycloserine enhancement of fear extinction is specific to successful exposure sessions: evidence from the treatment of height phobia

     Biological Psychiatry 73:1054–1058.

     https://doi.org/10.1016/j.biopsych.2012.12.009

     • PubMed
     • Google Scholar
115. 1. Spearman C

     (1910) Correlation calculated from faulty data

     British Journal of Psychology, 1904-1920 3:271–295.

     https://doi.org/10.1111/j.2044-8295.1910.tb00206.x

     • Google Scholar
116. 1. Specht J
     2. Egloff B
     3. Schmukle SC

     (2011) Stability and change of personality across the life course: the impact of age and major life events on mean-level and rank-order stability of the big five

     Journal of Personality and Social Psychology 101:862–882.

     https://doi.org/10.1037/a0024950

     • PubMed
     • Google Scholar
117. Book

     1. Spielberger CD

     (1983)

     Manual for the State-Trait Inventory STAI

     Palo Alto, CA: Mind Garden.

     • Google Scholar
118. 1. Steegen S
     2. Tuerlinckx F
     3. Gelman A
     4. Vanpaemel W

     (2016) Increasing transparency through a multiverse analysis

     Perspectives on Psychological Science 11:702–712.

     https://doi.org/10.1177/1745691616658637

     • PubMed
     • Google Scholar
119. 1. Thomas DR
     2. Zumbo BD

     (2012) Difference scores from the point of view of reliability and repeated-measures ANOVA: in defense of difference scores for data analysis

     Educational and Psychological Measurement 72:37–43.

     https://doi.org/10.1177/0013164411409929

     • Google Scholar
120. Software

     1. Tiedemann F

     (2020) Gghalves: compose half-half plots using your favourite geoms

     R-Project.

     https://CRAN.R-project.org/package=gghalves
121. Software

     1. Torchiano M

     (2020) Effsize: efficient effect size computation

     Zenodo.

     https://doi.org/10.5281/zenodo.1480624
122. 1. Torrents-Rodas D
     2. Fullana MA
     3. Bonillo A
     4. Andión O
     5. Molinuevo B
     6. Caseras X
     7. Torrubia R

     (2014) Testing the temporal stability of individual differences in the acquisition and generalization of fear

     Psychophysiology 51:697–705.

     https://doi.org/10.1111/psyp.12213

     • PubMed
     • Google Scholar
123. Software

     1. Ushey K

     (2020) Renv: project environments

     R-Project.

     https://CRAN.R-project.org/package=renv
124. 1. Vaidya JG
     2. Gray EK
     3. Haig J
     4. Watson D

     (2002) On the temporal stability of personality: evidence for differential stability and the role of life experiences

     Journal of Personality and Social Psychology 83:1469–1484.

     https://doi.org/10.1037/0022-3514.83.6.1469

     • PubMed
     • Google Scholar
125. 1. Vervliet B
     2. Baeyens F
     3. Van den Bergh O
     4. Hermans D

     (2013a) Extinction, generalization, and return of fear: a critical review of renewal research in humans

     Biological Psychology 92:51–58.

     https://doi.org/10.1016/j.biopsycho.2012.01.006

     • PubMed
     • Google Scholar
126. 1. Vervliet B
     2. Craske MG
     3. Hermans D

     (2013b) Fear extinction and relapse: state of the art

     Annual Review of Clinical Psychology 9:215–248.

     https://doi.org/10.1146/annurev-clinpsy-050212-185542

     • PubMed
     • Google Scholar
127. 1. Visser RM
     2. de Haan MIC
     3. Beemsterboer T
     4. Haver P
     5. Kindt M
     6. Scholte HS

     (2016) Quantifying learning-dependent changes in the brain: single-trial multivoxel pattern analysis requires slow event-related fMRI

     Psychophysiology 53:1117–1127.

     https://doi.org/10.1111/psyp.12665

     • PubMed
     • Google Scholar
128. 1. Visser RM
     2. Bathelt J
     3. Scholte HS
     4. Kindt M

     (2021) Robust BOLD responses to faces but not to conditioned threat: challenging the amygdala ’ S reputation in human fear and extinction learning

     The Journal of Neuroscience 41:10278–10292.

     https://doi.org/10.1523/JNEUROSCI.0857-21.2021

     • PubMed
     • Google Scholar
129. Preprint

     1. Werner F
     2. Klingelhöfer-Jens M
     3. Schümann D
     4. Gamer M
     5. Kalisch R
     6. Sommer T
     7. Lonsdorf TB

     (2022) Limited Temporal Stability of the Spielberger State-Trait Inventory over 3.5 Years

     PsyArXiv.

     https://doi.org/10.31234/osf.io/mubgv

     • Google Scholar
130. 1. Wickham H

     (2007) Reshaping data with the reshape package

     Journal of Statistical Software 21:1–20.

     https://doi.org/10.18637/jss.v021.i12

     • Google Scholar
131. Software

     1. Wickham H

     (2016) Elegant graphics for data analysis, version 3.3.5

     Ggplot2.

     https://ggplot2.tidyverse.org
132. Software

     1. Wickham H

     (2019) Stringr: simple, consistent wrappers for common string operations, version 1.4.1

     R-Project.

     https://CRAN.R-project.org/package=stringr
133. Software

     1. Wickham H

     (2020) Tidyr: tidy messy data

     R-Project.

     https://CRAN.R-project.org/package=tidyr
134. Software

     1. Wickham H
     2. François R
     3. Henry L
     4. Müller K

     (2021) Dplyr: a grammar of data manipulation

     R-Project.

     https://CRAN.R-project.org/package=dplyr
135. Software

     1. Wilke CO

     (2020) Cowplot: streamlined plot theme and plot annotations for ’ ggplot2

     R-Project.

     https://CRAN.R-project.org/package=cowplot
136. Software

     1. Xie Y

     (2015) Dynamic documents with R and knitr

     Yihui.

     https://yihui.org/knitr/
137. 1. Xiong P
     2. Liu M
     3. Liu B
     4. Hall BJ

     (2022) Trends in the incidence and dalys of anxiety disorders at the global, regional, and national levels: estimates from the global burden of disease study 2019

     Journal of Affective Disorders 297:83–93.

     https://doi.org/10.1016/j.jad.2021.10.022

     • PubMed
     • Google Scholar
138. 1. Yonkers KA
     2. Bruce SE
     3. Dyck IR
     4. Keller MB

     (2003) Chronicity, relapse, and illness? course of panic disorder, social phobia, and generalized anxiety disorder: findings in men and women from 8 years of follow-up

     Depression and Anxiety 17:173–179.

     https://doi.org/10.1002/da.10106

     • PubMed
     • Google Scholar
139. 1. Zeidan MA
     2. Lebron‐Milad K
     3. Thompson‐Hollands J
     4. Im JJY
     5. Dougherty DD
     6. Holt DJ
     7. Orr SP
     8. Milad MR

     (2012) Test–retest reliability during fear acquisition and fear extinction in humans

     CNS Neuroscience & Therapeutics 18:313–317.

     https://doi.org/10.1111/j.1755-5949.2011.00238.x

     • PubMed
     • Google Scholar
140. 1. Zeileis A
     2. Hothorn T

     (2002)

     Diagnostic checking in regression relationships

     R News 2:7–10.

     • Google Scholar
141. 1. Zeileis A

     (2004) Econometric computing with HC and HAC covariance matrix estimators

     Journal of Statistical Software 11:1–17.

     https://doi.org/10.18637/jss.v011.i10

     • Google Scholar
142. 1. Zeileis A
     2. Grothendieck G

     (2005) Zoo: S3 infrastructure for regular and irregular time series

     Journal of Statistical Software 14:1–27.

     https://doi.org/10.18637/jss.v014.i06

     • Google Scholar
143. 1. Zeileis A

     (2006) Object-oriented computation of sandwich estimators

     Journal of Statistical Software 16:1–16.

     https://doi.org/10.18637/jss.v016.i09

     • Google Scholar
144. 1. Zeileis A
     2. Köll S
     3. Graham N

     (2020) Various versatile variances: an object-oriented implementation of clustered covariances in R

     Journal of Statistical Software 95:1–36.

     https://doi.org/10.18637/jss.v095.i01

     • Google Scholar
145. Software

     1. Zhu H

     (2020) KableExtra: construct complex table with ’ kable ’ and pipe SYNTAX

     R-Project.

     https://CRAN.R-project.org/package=kableExtra
146. 1. Zuo XN
     2. Xu T
     3. Milham MP

     (2019) Harnessing reliability for neuroscience research

     Nature Human Behaviour 3:768–771.

     https://doi.org/10.1038/s41562-019-0655-x

     • PubMed
     • Google Scholar

Author details

1. Maren Klingelhöfer-Jens

   Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Visualization, Methodology, Writing - original draft, Pre-registration of the study

   For correspondence

   m.klingelhoefer-jens@uke.de

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5393-7871

2. Mana R Ehlers

   Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

   Contribution

   Conceptualization, Formal analysis, Visualization, Methodology, Writing - original draft

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1316-3787

3. Manuel Kuhn

   1. Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
   2. Department of Psychiatry, Harvard Medical School, and Center for Depression, Anxiety and Stress Research, McLean Hospital, Belmont, United States

   Contribution

   Data curation, Software, Investigation, Writing – review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-2210-9130

4. Vincent Keyaniyan

   Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

   Contribution

   Formal analysis, Visualization, Methodology, Writing – review and editing, Pre-registration of the study

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-5674-5197

5. Tina B Lonsdorf

   Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Methodology, Writing - original draft, Pre-registration of the study

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-1501-4846

• 904

  views

• 160

  downloads

• 14

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.