An observational treatment study of metacognition in anxious-depression
eLife assessment
This valuable study advances our knowledge of the effects of anxiety/depression treatment on metacognition, demonstrating that treatment increases metacognitive confidence alongside improving symptoms. The authors provide convincing evidence for the state-dependency of metacognitive confidence, based on a large longitudinal treatment dataset. However, it is unclear to what extent this effect is truly specific to treatment, as changes in metacognitive confidence in the group receiving online therapy were not statistically different from those in the control group.
https://doi.org/10.7554/eLife.87193.3.sa0Valuable: Findings that have theoretical or practical implications for a subfield
- Landmark
- Fundamental
- Important
- Valuable
- Useful
Convincing: Appropriate and validated methodology in line with current state-of-the-art
- Exceptional
- Compelling
- Convincing
- Solid
- Incomplete
- Inadequate
During the peer-review process the editor and reviewers write an eLife Assessment that summarises the significance of the findings reported in the article (on a scale ranging from landmark to useful) and the strength of the evidence (on a scale ranging from exceptional to inadequate). Learn more about eLife Assessments
Abstract
Prior studies have found metacognitive biases are linked to a transdiagnostic dimension of anxious-depression, manifesting as reduced confidence in performance. However, previous work has been cross-sectional and so it is unclear if under-confidence is a trait-like marker of anxious-depression vulnerability, or if it resolves when anxious-depression improves. Data were collected as part of a large-scale transdiagnostic, four-week observational study of individuals initiating internet-based cognitive behavioural therapy (iCBT) or antidepressant medication. Self-reported clinical questionnaires and perceptual task performance were gathered to assess anxious-depression and metacognitive bias at baseline and 4-week follow-up. Primary analyses were conducted for individuals who received iCBT (n=649), with comparisons between smaller samples that received antidepressant medication (n=82) and a control group receiving no intervention (n=88). Prior to receiving treatment, anxious-depression severity was associated with under-confidence in performance in the iCBT arm, replicating previous work. From baseline to follow-up, levels of anxious-depression were significantly reduced, and this was accompanied by a significant increase in metacognitive confidence in the iCBT arm (β=0.17, SE=0.02, p<0.001). These changes were correlated (r(647)=-0.12, p=0.002); those with the greatest reductions in anxious-depression levels had the largest increase in confidence. While the three-way interaction effect of group and time on confidence was not significant (F(2, 1632)=0.60, p=0.550), confidence increased in the antidepressant group (β=0.31, SE = 0.08, p<0.001), but not among controls (β=0.11, SE = 0.07, p=0.103). Metacognitive biases in anxious-depression are state-dependent; when symptoms improve with treatment, so does confidence in performance. Our results suggest this is not specific to the type of intervention.
Introduction
Metacognition refers to the ability to accurately monitor and appraise one’s own cognitive experience (Fleming and Lau, 2014). Metacognition is crucial for adaptive behaviour: it allows for flexible adjustment of behavioural strategies in order to improve performance, signals when to engage or withdraw from an activity, and guides the engagement in social interactions (Fleming and Daw, 2017; Fleming et al., 2012). Metacognitive abilities vary across individuals; one can be under- or over-confident, and these biases can be associated with maladaptive thoughts, feelings and behaviours (Philipp et al., 2020). There is a growing interest in these confidence abnormalities in psychiatry, with studies implicating alterations of metacognition in depression, obsessive-compulsive disorder, and psychosis (Hoven et al., 2019). While case-control studies have mainly found patterns of reduced confidence across several disorders, newer methods that can separate transdiagnostic dimensions of mental health using large online samples have revealed specific and bi-directional effects of confidence (Wise et al., 2023). Using these methods, studies have shown that the transdiagnostic dimension ‘anxious-depression’ is linked to under-confidence in one’s own performance, while a separate dimension ‘compulsivity and intrusive thought’ is related to elevated confidence (Rouault et al., 2018; Seow and Gillan, 2020; Hoven et al., 2022; Seow et al., 2021; Benwell et al., 2022). A major gap in this area is that studies to-date only measure metacognition and transdiagnostic psychopathology at a single time point. Therefore, it is unclear if metacognitive biases are stable, fixed traits, or if they might change with treatment response. Preliminary evidence suggests metacognition may indeed be malleable; metacognitive abilities can be improved with metacognitive interventions, such as training, in unselected online samples (Carpenter et al., 2019; Engeler and Gilbert, 2020) and in clinical populations (Jelinek et al., 2017; Lysaker et al., 2018). However, it remains unknown if metacognitive changes generalise beyond its specific training context and are associated with any real-world improvement in psychiatric symptoms. In clinical studies, research has identified confidence abnormalities in at-risk populations (Eisenacher et al., 2015; Gawęda et al., 2018), suggestive of a trait-dependence. In contrast, stimulant use disorders remitters have better metacognition than active users, suggesting state-dependence (Moeller et al., 2016). Within-subject designs are needed to extend this work and understand if metacognition can improve in parallel to symptom alleviation, or if those with greater metacognitive deficits are simply the most vulnerable to illness onset and persistence. The present study aimed to address this by examining metacognition in a large cohort of individuals before and after internet-based cognitive behavioural therapy (iCBT). iCBT has emerged as an important intervention for reducing the treatment-gap in mental healthcare provision globally; it is low-cost, scalable, geographically unconstrained and flexible (Mogoașe et al., 2017; Webb et al., 2017). iCBT offers patients standardised content and records objective metrics of treatment engagement, making it particularly well-suited to treatment-oriented research in psychiatry (Lee et al., 2023). Additionally, iCBT has demonstrated clinical effectiveness in terms of symptom improvement (Andersson et al., 2019; Eilert et al., 2021; Karyotaki et al., 2021). While one study found that iCBT modified self-reported metacognitive beliefs (Newby et al., 2014), it remains unknown if metacognitive confidence in decision-making improves following successful iCBT. In the current study, we used an objective task measure of metacognition (Fleming et al., 2014), which allowed us to test if successful treatment is linked to within-person improvements in metacognition. We also tested if any changes in metacognition were iCBT-specific, by comparing data gathered from smaller samples of individuals receiving antidepressant medication and a control group receiving no intervention. Similar to iCBT, antidepressants have established transdiagnostic efficacy (Cipriani et al., 2018; Gøtzsche and Dinnage, 2020; Skapinakis et al., 2016). However, studies examining the impact of antidepressants on cognition have typically focused on cognitive capacities other than metacognition (Prado et al., 2018; Rosenblat et al., 2015; Salagre et al., 2017; Vernon et al., 2014). Accordingly, a secondary aim of this study was to compare metacognitive changes across the different intervention arms, which may shed light on differential therapeutic mechanisms and potentially augment therapeutic decision-making in the future.
Results
Cross-sectional findings at baseline: iCBT
At baseline, age was not significantly associated with mean confidence (β=0.003, SE = 0.003, p=0.401). Males had significantly higher confidence (M=3.91, SD = 0.81), than females (M=3.75, SD = 0.86) (β=0.08, SE = 0.04, p=0.044). Mean confidence was significantly lower among those with educational attainment above undergraduate level (M=3.73, SD = 0.89), when compared to participants with attainment below undergraduate (M=3.94, SD = 0.87) (β=0.21, SE = 0.10, p=0.036). Mean confidence did not significantly differ when comparing those who had educational attainment above undergraduate (M=3.73, SD = 0.89) to those who had some or had completed an undergraduate degree (M=3.74, SD = 0.83) (β=0.01, SE = 0.08, p=0.921). When comparing task measures, mean confidence was not significantly associated with mean accuracy (β=−1.29, SE = 1.42, p=0.366), mean dot difference (task difficulty) (β=0.002, SE = 0.002, p=0.406), or mean response time (β=6.47, SE = 15.13, p=0.669).
Participants with higher levels of anxious-depression had lower levels of mean confidence (β=−0.09, SE = 0.03, p=0.008; Figure 1A and B), while those with higher levels of compulsivity and intrusive thought had elevated mean confidence (β=0.11, SE = 0.03, p=0.002; Figure 1A and C), controlling for age, gender, and education. Levels of social withdrawal were not associated with mean confidence (β=−0.05, SE = 0.03, p=0.168; Figure 1A).
Treatment findings: iCBT
The transdiagnostic dimensions and psychiatric scale scores all significantly improved from baseline to four-week follow-up, except for impulsivity (Figure 2A and Figure 2—figure supplement 1). In tandem with these clinical changes, there was a small but significant increase in mean confidence from baseline (M=3.78, SD=0.85) to follow-up (M=3.95, SD=0.89), (β=0.17, SE = 0.02, p<0.001, r2=0.01) (Figure 2B). Although mean accuracy did not change from baseline (M=0.71, SD=0.02) to follow-up (M=0.71, SD = 0.02) (β=0.001, SE = 0.001, p=0.299), due to the staircasing procedure, participants’ ability to detect differences between the visual stimuli improved. This was reflected as the overall increase in task difficulty to maintain the accuracy rates from baseline (dot difference: M=41.82, SD=11.61) to follow-up (dot difference: M=39.80, SD=12.62), (β=−2.02, SE = 0.44, p<0.001, r2=0.01) (Figure 2C). Additionally, the effect of time on confidence was not dependent on how much participants engaged in iCBT, as indexed by time spent in the program (β<0.01, SE <0.01, p=0.756) and percentage of the iCBT program viewed (β=0.09, SE = 0.21, p=0.650). Confidence changes did not differ across the different types of iCBT programs, when the Space from Depression and Anxiety program was compared pairwise to Space from Depression (β=−0.09, SE = 0.17, p=0.594), Space from Anxiety (β=−0.15, SE = 0.17, p=0.376), Life Skills (β=−0.15, SE = 0.19, p=0.435) or the ‘other’ category comprising miscellaneous iCBT programs (β=−0.03, SE = 0.18, p=0.881). Change in confidence was not different among those receiving concurrent treatment versus not (β=−0.03, SE = 0.06, p=0.566). To test if changes in confidence from baseline to follow-up scaled with changes in anxious-depression, we ran a repeated measure regression analyses with per-person changes in anxious-depression as an additional independent variable. We found this was the case, evidenced by a significant interaction effect of time and change in anxious-depression on confidence (β=−0.12, SE = 0.04, p=0.002). Those with the largest decrease in anxious-depression had the greatest increase in confidence. This was similarly evident in a simple correlation between change in confidence and change in anxious-depression (r(647)=-0.12, p=0.002) (Figure 2D). This effect was specific to anxious-depression; the interaction effect of time and change in compulsivity and intrusive thought on mean confidence was not significant (β=−0.06, SE = 0.05, p=0.221). Similarly, the significant interaction effect of time and anxious-depression on mean confidence held when including change in the other transdiagnostic dimensions (compulsivity and intrusive thought and change in social withdrawal) as covariates in the model (β=−0.07, SE = 0.03, p=0.005). The interaction effect of time and change in anxious-depression on task difficulty was not significant (β=0.14, SE = 0.69, p=0.835). To test the extent to which baseline differences in mean confidence or anxious depression might drive the results, we re-ran these regression analyses with baseline measures instead of change indices. There was no significant interaction effect of time and baseline confidence (β=0.06, SE = 0.04, p=0.144) or an interaction effect of time and baseline anxious-depression (β=−0.03, SE = 0.05, p=0.611) on mean confidence. Exploratory analyses further tested the specificity of these effects to anxious-depression by examining the interaction effect of time and change in each psychiatric score on mean confidence. Changes in trait anxiety (β=−0.08, SE = 0.02, p=0.002), depression (β=−0.06, SE = 0.02 p=0.011) and alcohol misuse (β=−0.05, SE = 0.02, p=0.037) also showed an association with changes in confidence (Figure 2E and Figure 2—figure supplement 2).
Comparing iCBT, antidepressant and control groups
When comparing the three groups directly, ANOVA analysis predicting anxious-depression scores with group and time as independent variables revealed a main effect of time (F(1, 1632)=62.99, p<0.001), a main effect of group (F(2, 1632)=249.74, p<0.001), and an interaction effect of group and time (F(2, 1632)=9.23, p<0.001). Examining simple effects in the antidepressant arm, there was a significant reduction in anxious-depression from baseline to follow-up (β=−0.61, SE = 0.09, p<0.001). Among controls, levels of anxious-depression did not significantly change (β=0.10, SE = 0.06, p=0.096). Further details of transdiagnostic clinical changes for the antidepressant and controls groups are presented in Figure 3A and Figure 3—figure supplement 1. Predicting confidence scores using ANOVA analysis with group and time as independent variables revealed a main effect of time (F(1, 1632)=16.26, p<0.001), and no significant main effect of group (F(2, 1632)=2.35, p=0.096). The interaction effect of group and time on mean confidence was not significant (F(2, 1632)=0.60, p=0.550), suggesting that change in confidence did not differ across the three groups. Tests of simple effects revealed that mean confidence significantly increased from baseline (M=3.77, SD=0.88) to follow-up (M=4.07, SD=0.79) in the antidepressant arm (β=0.31, SE = 0.08, p<0.001) (Figure 3B). Among controls, there was no significant change in confidence from baseline (M=3.68, SD=0.86) to follow-up (M=3.79, SD=0.92) (β=0.11, SE = 0.07, p=0.103) (Figure 3B). With respect to task performance, there was a significant main effect of time (F(1, 1632)=15.17, p=0.001) and group (F(2, 1632)=4.56, p=0.011) on mean dot difference when the three groups were included in the model. The interaction effect of time and group on mean dot difference was not significant (F(2, 1632)=1.91, p=0.148), suggesting no differences across the groups in task difficulty changes. In the antidepressant arm, mean dot difference decreased from baseline (M=41.2, SD=13.3) to follow-up (M=35.3, SD=13.1) (β=−5.91, SE = 1.25, p<0.001), indicating increased task difficulty. There was no significant change in task difficulty among controls from baseline (M=43.0, SD=11.8) to follow-up (M=41.4, SD=13.6) (β=−1.64, SE = 1.30, p=0.210) (Figure 3C). While our sample was underpowered to examine individual differences, we conducted an exploratory analysis examining the connection between changes in anxious-depression symptoms and changes in confidence in the antidepressant and controls groups. When examining the effects of time, group and anxious-depression change on mean confidence, there was a significant interaction effect of time and anxious-depression change on mean confidence (F(1, 1626)=4.04, p=0.045), suggesting change in confidence is associated with change in anxious-depression. There was no significant three-way interaction of anxious-depression change, time and group on mean confidence when comparing the three groups (F(2, 1626)=0.08, p=0.928), indicating that the significant association between confidence change and anxious-depression change was not specific to any group. Although not significant, the association between change in confidence and change in anxious-depression was in the expected negative direction in the antidepressant arm (r(80)=-0.10, p=0.381), and among controls (r(86)=-0.17, p=0.111) (Figure 3D).
Discussion
Metacognitive biases are linked to transdiagnostic dimensions of mental health, but it is presently unclear if these biases are stable traits, or if they fluctuate alongside symptoms and change during the course of treatment (Seow et al., 2021). To answer these questions, we administered a previously validated adaptive task of metacognitive ability that controls for objective performance differences (Fleming et al., 2014) in a large sample of individuals before and after four weeks of iCBT or antidepressant medications (Lee et al., 2023). As expected, a 4-week course of iCBT or antidepressant medication led to transdiagnostic improvements in mental health (Cipriani et al., 2018; Gøtzsche and Dinnage, 2020; Skapinakis et al., 2016). Alongside this, there was a significant increase in metacognitive confidence following four weeks of iCBT or antidepressant medication. Not simply a practice effect, we found that individuals in the iCBT arm with the greatest improvements in anxious-depression had the largest increase in confidence at follow-up. This association with clinical improvements was specific to metacognitive changes, and not changes in task performance, suggesting that changes in confidence do not merely reflect greater task familiarity at follow-up. These findings suggest that metacognitive biases in anxious-depression are state-dependent. This builds on previous findings in small samples that have shown iCBT improves self-reported metacognitive self-beliefs (Newby et al., 2014) and that metacognition can be altered through adaptive training (Carpenter et al., 2019; Engeler and Gilbert, 2020; Jelinek et al., 2017; Lysaker et al., 2018). At baseline, we replicated the previously observed bi-directional associations between metacognitive bias and anxious-depression and compulsivity and intrusive thought (Rouault et al., 2018; Seow and Gillan, 2020; Hoven et al., 2022; Benwell et al., 2022). While higher levels of anxious-depression is associated with lower confidence, those with higher levels of compulsivity and intrusive thought have elevated confidence. This is a somewhat surprising dissociation, as compulsivity and anxious-depression are themselves positively correlated in the population. One way this can be reconciled is if the mechanisms underlying these opposing confidence biases are distinct. In anxious-depression, there appears to be more pervasive metacognitive biases that affect confidence in many domains and levels of a metacognitive hierarchy (spanning confidence in low level perceptual decisions to ideas of self-worth) (Hoven et al., 2022; Seow et al., 2021). In contrast, inflated confidence in compulsivity may be based on more specific biases in learning and inference (Seow and Gillan, 2020). The present study was observational and therefore did not randomly assign participants to a different treatments. To partially remediate that limitation, we included two smaller groups receiving antidepressant medication and a control group. Levels of transdiagnostic psychiatric dimensions remained stable across time among controls, while they significantly improved in the antidepressant arm. Similarly to iCBT, we found that confidence improved in the antidepressant group, but not among controls. The interaction, however, was not significant, meaning that we cannot reject the null hypothesis that confidence improved to the same degree across the three groups. As increased task difficulty among clinical groups was not significantly greater relative to controls, changes in task difficulty may simply reflect greater task familiarity at follow-up across groups, as opposed to gains in general perceptual performance among clinical arms. Examining the three groups together, the data suggests that confidence changes are unlikely to be treatment specific, rather, confidence fluctuates in tandem with anxious-depression. This was evident in an overall association between change in anxious-depression and change in confidence that was not modified by treatment arm. Additionally, levels of iCBT engagement and concurrent treatments did not bolster changes in confidence. Overall, the results indicated that metacognition fluctuates with anxious-depression state, regardless of treatment type or exposure. Future research with larger samples are required to address this definitively.
Limitations and future directions
Confidence change and anxious-depression change were significantly but weakly associated. Similarly, the relative change in confidence across treatment arms was small. Therefore, while tests of metacognitive confidence can inform theoretical models, like most cognitive tests, they are likely of limited utility in clinical practice, at least when used in isolation (Mogoașe et al., 2017). Given the complexity of mental health causes and presentations, multivariable models are needed to see practical value from such tests. We did not assess confidence or anxious-depression to treatment cessation and so the causal path and temporal dependence, if they exist, cannot be derived from these data. Future research should consider assessing metacognition and anxious-depression continuously through treatment, in order to elucidate the causal relationship between anxious-depression and metacognition with mediation analysis (Newby et al., 2014). More intensive, repeating testing in future studies may also reveal the temporal window at which metacognition has the propensity to change, which could be more momentary in nature. While this study examined changes in metacognition with iCBT generally, future research should examine if the strength of the association between confidence change and anxious-depression change is greater following iCBT modules targeting metacognition or following metacognitive intervention (Philipp et al., 2020). The iCBT programs in this study primarily targeted depression and anxiety, which may explain why changes in confidence did not scale with improvements in compulsivity. Future research is required to assess if treatments aimed at compulsive disorders decrease the over-confidence commonly observed in those high in compulsivity and intrusive thought. As the antidepressant and control groups were much smaller than the iCBT arm, we were unable to compare changes in confidence across the types of antidepressant medications individuals received and we were underpowered more generally for individual differences analyses and multi-arm comparisons. Exploratory analyses were nonetheless presented and can form the basis for future investigations.
Conclusions
Our findings replicated the cross-sectional evidence that higher levels of anxious-depression are associated with under-confidence. We demonstrate that metacognitive confidence increases following four weeks of iCBT or antidepressant treatment. Overall, we observed that the greater the improvement in anxious-depression, the more confident participants became, which did not appear to be dependent on treatment type. This suggests that metacognitive biases in anxious-depression are state-dependent and might be normalised through clinical improvements.
Methods
Participants
Participants were recruited as part of the Precision in Psychiatry (PIP) study (Lee et al., 2023), an observational, longitudinal study in which participants underwent a 4-week course of iCBT or antidepressant medication. Further details of the PIP study procedures that are not specific to this study can be found in a prior publication (Lee et al., 2023). Ethical approval for the PIP study was obtained from the Research Ethics Committee of School of Psychology, Trinity College Dublin (Approval ID: SPREC072017-01) and the Northwest-Greater Manchester West Research Ethics Committee of the National Health Service, Health Research Authority and Health and Care Research Wales (REC reference: 20/NW/0020, IRAS project ID: 270623). Informed consent, and consent to publish, was obtained from all participants prior to study participation. A power analysis was carried out using effect sizes from a previous study examining cross-sectional associations between metacognition and anxious-depression, and compulsivity and intrusive thought (Rouault et al., 2018). Sample sizes of N=454 and N=332 respectively were required to detect these associations with 80% power. The sample sizes of the antidepressant arm and control group were smaller and used for secondary and more exploratory analyses.
iCBT Arm
Individuals initiating iCBT provided by SilverCloud Health were recruited from two sites: (1) the National Health Service Berkshire Foundation in the UK and (2) Aware mental health charity in Ireland. Participants included in the study either started their iCBT intervention ≤2 days prior to signing up, or provided a treatment start date in the near future, and scored ≥10 on the Work and Social Adjustment Scale (WSAS) at baseline, which indicated significant functional impairment due to clinical symptoms (Mundt et al., 2002). Figure 4A shows the disposition of participants throughout the study. N=2404 were screened, of whom N=1,496 were eligible, N=836 completed baseline assessments and met inclusion criteria. A final N=649 completed and met inclusion criteria for follow-up assessments. While study follow-up data was collected after four weeks of treatment, iCBT could last up to 12 weeks (Lee et al., 2023). The final sample was, on average, 32.2 years old (SD=11.0), mostly female (n=501, 77.4%), living in the United Kingdom (n=546, 84.4%), and had some or completed undergraduate level education (n=342, 52.9%) (Table 1).
Antidepressant Arm
Individuals were recruited globally using advertisements placed on Google search, in addition to social media platforms, mental health websites, local pharmacies and General Practitioner waiting rooms. Participants were included if they started or planned to start treatment ≤2 days of study sign-up, scored ≥10 on the WSAS at baseline, and provided a valid photograph of an antidepressant medication prescription. N=270 individuals were screened, of whom N=174 were eligible, N=102 completed and met inclusion criteria at baseline and a final N=82 had follow-up data (Figure 4A). Participants were mostly female (n=60, 73.2%), mean age = 30.5 (10.5), were living in Ireland or the United Kingdom (n=66, 80.5%) and had some or completed undergraduate level education (n=49, 59.8%) (Table 1).
Control group
Participants in the no treatment control group were recruited through university mailings lists and advertisements posted online and around Trinity College Dublin. Participants included in this arm scored <10 on an adapted version of the WSAS (where they rated functional impairment from their general problems rather than mental health problems) and self-reported that they had no current mental health problems and were not undergoing treatment for any mental health problems at the time of screening. N=444 individuals were screened, of whom N=245 were eligible, N=113 had baseline data and a final N=88 completed follow-up assessments and met inclusion criteria for the study (Figure 4A). Participants in the control group were matched for sociodemographic characteristics in the antidepressant arm, with no significant differences across the two groups in gender (Fisher’s exact test p=0.498), or age (Welch’s t(166.66)=−0.81, p=0.420), or levels of educational attainment (χ2(2)=0.66, p=0.718). Participants in the control group came from the United Kingdom and Ireland (>93%), whereas the antidepressant arm was more international, with 20% coming from other countries (χ2(2)=13.59, p=0.001).
When comparing sociodemographic characteristics across the three study groups, there were no significant differences between groups in gender proportions (Fisher’s exact test p=0.414), or levels of educational attainment (χ2(4)=7.11, p=0.130), as reported in Table 1. Age differed across the three arms (F(2, 813)=3.68, p=0.026), with post hoc Tukey tests indicating that mean age was higher in the iCBT arm (M=32.2, SD=11.0) compared to the controls (M=29.1, SD=12.0) (padj = 0.034), but not when compared to the antidepressant arm (M=30.5, SD=10.5) (padj = 0.376) (Table 1). The countries participants were living in varied across the study arms (χ2(4)=211.73, p<0.001), as there was a higher proportion of individuals in the iCBT arm living in the UK (n=546, 84.4%) when compared to the antidepressant arm (n=34, 41.5%) and the control group (n=24, 27.3%) (Table 1).
Procedure
Figure 4B shows an overview of the study design, including the assessments involved at each timepoint. For the purposes of this study, we focused on a select set of sociodemographic characteristics (gender, age, country of residence, level of educational attainment), self-reported psychiatric questionnaires, metacognitive task performance and treatment data from the PIP study (Lee et al., 2023).
Self-reported psychiatric questionnaires
Participants completed the WSAS (Mundt et al., 2002) at baseline and follow-up. Individuals in the iCBT and antidepressant arms received the original WSAS, which asks them to rate their level of functional impairment from mental health problems. Individuals in the control group received an adapted version of the WSAS, which asks them to rate their level of impairment from ‘their problems’ more generally. Each WSAS item was scored from 0 ‘not at all’ to 8 ‘very severely’, with overall scores ranging from 0 to 40. Higher WSAS scores indicated higher levels of functional impairment (Mundt et al., 2002). Participants completed nine standard self-report clinical questionnaires at baseline and four-week follow-up that assess a variety of psychiatric symptoms, including depression (Zung Self-Rating Depression Scale; Zung, 1965), trait anxiety (State Trait Anxiety Inventory) (Spielberger et al., 1983), schizotypy (Short Scales for Measuring Schizotypy) (Mason et al., 2005), impulsivity (Barratt Impulsiveness Scale 11) (Patton et al., 1995), obsessive-compulsive disorder (OCD) (Obsessive-Compulsive Inventory-Revised, OCI-R) (Foa et al., 2002), social anxiety (Liebowitz Social Anxiety Scale; Liebowitz, 1987), eating disorders (Eating Attitudes Test) (Garner et al., 1982), apathy (Apathy Evaluation Scale) (Marin et al., 1991), and alcohol misuse (Alcohol Use Disorders Identification Test) (Saunders et al., 1993).
Metacognitive task
Participants completed a visuo-perceptual decision-making task (Fleming et al., 2014) to assess metacognition (Figure 4C). On each trial, participants were shown a fixation cross for 1000 milliseconds (ms), followed by two sunflowers with a varying number of seeds for 300ms. Participants then had unlimited time to make a judgement on which of two sunflower stimuli contained more seeds and then rated their confidence in each judgement, on a scale from ‘1=Guessing’ to ‘6=Certain’. There was a total of 210 trials, divided equally into five blocks. While participants were given feedback on which sunflower they had chosen for 500ms, there was no trial-level feedback provided on their performance. Split-half reliability was high for baseline mean confidence, as indicated by the correlation between odd and even trials across each participant in the iCBT arm (r(647)=0.98, p<0.001). Mean accuracy was tightly controlled in this task using a ‘two-down one-up’ staircase procedure, in which equally sized changes in dot difference occurred after each incorrect response and after two consecutive correct responses. This maintained objective performance across all participants at a desired level of 70% correct, crucial for estimating metacognition without any confound of real performance differences. Changes in dot difference were calculated using log-space, with a start log difference of 4.2 (+70 dots). Differences in step size changed by ± 0.4 for the first five trials, ± 0.2 for the next five trials and ± 0.1 for the remainder of the task. Dot difference on each trial could range from six dots (1.79 in log-space, the most difficult to discriminate) to 81 dots (4.39 in log-space, the easiest to discriminate).
Treatment data
Every week, participants self-reported if they had adhered to iCBT or their antidepressant medications each week. Participants also reported if they had started any new concurrent medications and/or psychological treatments for mental health each week. Starting new medication and/or psychological treatments and treatment nonadherence during the study were not grounds for exclusion in any group. Objective indicators of treatment engagement were provided by SilverCloud for 640 participants in the iCBT arm, which comprised of percentage of program viewed, time (minutes) spent in the program, and program type. In relation to program type, Space from Depression was the most common iCBT program (n=158, 24.7%), followed by Space from Generalised Anxiety Disorder (n=96, 15.0%), Life Skills Online (n=89, 13.9%), Space from Anxiety (n=84, 13.1%), and Space from Depression and Anxiety (n=77, 12.0%). Given the overlapping content of Space from Generalised Anxiety Disorder and Space from Anxiety, these programs were merged into one category ‘Space from Anxiety’ (n=180, 28.1%) for analyses. The ‘Other’ category (n=136, 21.3%) comprised of the following iCBT programs: Space from Stress (n=33, 5.2%), Space from Social Anxiety (n=20, 3.1%), Space for Resilience (n=20, 3.1%), Space for Perinatal Wellbeing (n=18, 2.8%), Space in Chronic Pain from Depression and Anxiety (n=12, 1.9%), Space from Health Anxiety (n=9, 1.4%), Space from OCD (n=7, 1.1%), Space from Panic (n=6, 0.9%), Space from Phobia (n=5, 0.8%), Space for Sleep (n=4, 0.6%), Space in Lung Conditions from Depression and Anxiety (n=1, 0.2%) and Space from Money Worries (n=1, 0.2%), which were merged together into one category for analysis due to low N in each program.
Data preparation and analysis
Self-reported psychiatric questionnaire
Individual scores on dimensions of anxious-depression, compulsivity and intrusive thought, and social withdrawal were calculated by multiplying each of the 209 item scores on the nine above-mentioned self-report clinical questionnaires by the 209 corresponding item weights from a previously published factor analysis on these scales (Gillan et al., 2016). Dimension scores were scaled to centre on zero, with higher scores indicating higher levels of transdiagnostic psychopathology.
To determine the proportion of careless/inattentive responders on the self-report clinical questionnaire, we included ‘catch’ questions that were embedded in the OCI-R (If you are paying attention to these questions, please select “A little”) and the WSAS (If you are paying attention to these questions, please select “Not at all”) at baseline and in the WSAS at all subsequent assessments. In the iCBT arm, 54 (8.3%) participants failed at least one of the checks, with just nine (1.4%) failing more than one. Additionally, eight (9.8%) individuals in the antidepressant arm and seven (8.0%) in the control group failed at least one of the catch items. Given the small number of participants that failed more than one attention checks and that all those participants passed the task exclusion criteria, these individuals were retained for subsequent analyses. Additionally, excluding those that failed more than one catch item in the iCBT arm did not affect the significance of results, including the change in confidence (β=0.16, SE = 0.02, p<0.001), change in anxious-depression (β=−0.32, SE = 0.03, p<0.001), and the association between change in confidence and change in anxious-depression (r(638)=-0.10, p=0.011).
Metacognition task
The perceptual decision-making task performance was used to quantify our primary cognitive outcome measure, metacognitive bias, the mean confidence rating across trials. Task difficulty was measured as the mean dot difference across trials, where more difficult trials had a lower dot difference between stimuli. Mean reaction time to stimuli choice across trials was measured in seconds and task accuracy was calculated as the mean proportion of correct responses across trials. As metacognitive efficiency (the extent to which confidence estimates map onto objective performance) was not previously associated with transdiagnostic dimensions cross-sectionally (Rouault et al., 2018; Hoven et al., 2022; Benwell et al., 2022), the state-dependence of efficiency was not examined in this study.
We employed a number of exclusion criteria to ensure high data quality from the metacognitive task. Firstly, we planned to exclude participants if they selected the right or left sunflower on greater than 95% of trials, but none met this criterion. Second, we excluded participants with mean accuracy less than 0.60 and greater than 0.85, indicating the staircase procedure did not converge within acceptable bounds (n=38 at baseline and n=34 at follow-up excluded in the iCBT arm; n=8 at baseline and n=7 at follow-up excluded in the antidepressant arm; n=6 at baseline and n=5 at follow-up excluded in the control group).
Treatment data
Treatment adherence was high by week 3 in both clinical arms, with over ≥95% of the iCBT arm still undergoing treatment (i.e. 98% at weekly check-in (WCI) 1, 97% at WCI 2 and 95% at WCI 3) and ≥99% of the antidepressant group reported still taking antidepressant medication (i.e. 100% at WCI 1 and WCI 2 and 99% at WCI 3). In the antidepressant arm, four participants altered the dosage of their medication during the study participation (n=2 took less than prescribed and n=2 took more than prescribed). In terms of concurrent treatment, 175 (27.0%) in the iCBT arm were receiving another treatment during the study, of which 48 (7.4%) were taking concurrent medication for a mental health problem and 145 (22.3%) were receiving a concurrent form of psychotherapy. For the antidepressant group, 33 (40.2%) were receiving another treatment during the study, with 6 (7.3%) taking at least one other medication for a mental health problem and 2 (39.0%) were receiving some form of psychotherapy. Thus, there were partial overlaps in the treatments received across the two clinical arms. Within the control group, two participants (2.3%) started taking medication during the study and none initiated psychotherapy.
Statistical analysis
We tested for relationships between baseline task measures and the psychiatric symptom dimensions using linear regression analysis, controlling for age, gender, and education. To examine pre-post changes, we carried out linear mixed-effects models with measures of metacognition or psychopathology as the dependent variable, time (baseline = 0; follow-up=1) as the independent variable and participants as random effects. To determine the association between change in confidence and change in anxious-depression, we used (1) Pearson correlation analysis to correlate change indices for both measures and, (2) regression-based repeated-measures analysis: mean confidence ~time*anxious-depression score change, where mean confidence is entered with two datapoints (one for pre- and one for post-treatment i.e., within-person) and anxious-depression change is a single value per person (between-person change score). Exploratory linear regression analyses tested the specificity of the effects, replacing anxious-depression with each of the measures of psychopathology in turn as follows: Mean confidence ~time*dimension/psychiatric scale score change. We additionally ran regression analyses to test if concurrent treatment or the degree of objective engagement in iCBT interacted with the effect of time on mean confidence. Exploratory ANOVA analyses were also conducted to compare changes in anxious-depression, task difficulty and confidence across the three arms directly. For all tests, statistical significance was defined as p<0.05, with two-tailed p-values used. All regressors were scaled as Z scores to compare the regression coefficients of independent variables within each model. Adjustments for multiple comparisons were not conducted for analyses of replicated effects, or exploratory analyses (Althouse, 2016). The code to reproduce statistical analyses are available at https://osf.io/89xzq/. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. All data from the PIP study will be made publicly available once the primary treatment prediction analysis of the project are published on a pre-print server.
Data availability
The code to reproduce statistical analyses are available at https://osf.io/89xzq/. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. All data from the PIP study will be made publicly available once the primary treatment prediction analysis of the project are published.
-
Open Science FrameworkID 89xzq. An observational treatment study of metacognition in anxious-depression.
References
-
Adjust for multiple comparisons? It’s not that simpleThe Annals of Thoracic Surgery 101:1644–1645.https://doi.org/10.1016/j.athoracsur.2015.11.024
-
Domain-general enhancements of metacognitive ability through adaptive trainingJournal of Experimental Psychology. General 148:51–64.https://doi.org/10.1037/xge0000505
-
Investigation of metamemory functioning in the at-risk mental state for psychosisPsychological Medicine 45:3329–3340.https://doi.org/10.1017/S0033291715001373
-
Metacognition: computation, biology and functionPhilosophical Transactions of the Royal Society of London. Series B, Biological Sciences 367:1280–1286.https://doi.org/10.1098/rstb.2012.0021
-
How to measure metacognitionFrontiers in Human Neuroscience 8:443.https://doi.org/10.3389/fnhum.2014.00443
-
The Obsessive-Compulsive Inventory: development and validation of a short versionPsychological Assessment 14:485–496.
-
The eating attitudes test: psychometric features and clinical correlatesPsychological Medicine 12:871–878.https://doi.org/10.1017/s0033291700049163
-
What have antidepressants been tested for? A systematic reviewInternational Journal of Risk & Safety in Medicine 31:157–163.https://doi.org/10.3233/JRS-200006
-
Abnormalities of confidence in psychiatry: an overview and future perspectivesTranslational Psychiatry 9:268.https://doi.org/10.1038/s41398-019-0602-7
-
Metacognitive approaches to the treatment of psychosis: a comparison of four approachesPsychology Research and Behavior Management 11:341–351.https://doi.org/10.2147/PRBM.S146446
-
Reliability and validity of the Apathy Evaluation ScalePsychiatry Research 38:143–162.https://doi.org/10.1016/0165-1781(91)90040-v
-
Short scales for measuring schizotypySchizophrenia Research 78:293–296.https://doi.org/10.1016/j.schres.2005.06.020
-
Metacognitive impairment in active cocaine use disorder is associated with individual differences in brain structureEuropean Neuropsychopharmacology 26:653–662.https://doi.org/10.1016/j.euroneuro.2016.02.009
-
Internet-based psychotherapy for adult depression: what about the mechanisms of change?Journal of Clinical Psychology 73:5–64.https://doi.org/10.1002/jclp.22326
-
The work and social adjustment scale: a simple measure of impairment in functioningBritish Journal of Psychiatry 180:461–464.https://doi.org/10.1192/bjp.180.5.461
-
Factor structure of the Barratt impulsiveness scaleJournal of Clinical Psychology 51:768–774.https://doi.org/10.1002/1097-4679(199511)51:6<768::aid-jclp2270510607>3.0.co;2-1
-
Concepts of metacognition in the treatment of patients with mental disordersJournal of Rational-Emotive & Cognitive-Behavior Therapy 38:173–183.https://doi.org/10.1007/s10942-019-00333-3
-
The cognitive effects of antidepressants in major depressive disorder: a systematic review and meta-analysis of randomized clinical trialsThe International Journal of Neuropsychopharmacology 19:pyv082.https://doi.org/10.1093/ijnp/pyv082
-
Treatment of neurocognitive symptoms in unipolar depression: a systematic review and future perspectivesJournal of Affective Disorders 221:205–221.https://doi.org/10.1016/j.jad.2017.06.034
-
How local and global metacognition shape mental healthBiological Psychiatry 90:436–446.https://doi.org/10.1016/j.biopsych.2021.05.013
-
BookManual for the State-Trait Anxiety Inventory (Form Y1 – Y2)Palo Alto, CA: Consulting Psychologists Press.
-
Internet-based cognitive-behavioral therapy for depression: current progress and future directionsHarvard Review of Psychiatry 25:114–122.https://doi.org/10.1097/HRP.0000000000000139
-
A SELF-RATING DEPRESSION SCALEArchives of General Psychiatry 12:63–70.https://doi.org/10.1001/archpsyc.1965.01720310065008
Article and author information
Author details
Funding
MQ: Transforming Mental Health (MQ16IP13)
- Claire M Gillan
Science Foundation Ireland (19/FFP/6418)
- Claire M Gillan
European Research Council (ERC-H2020-HABIT)
- Claire M Gillan
Government of Ireland Postgraduate Scholarship Programme (GOIPG/2020/662)
- Celine Ann Fox
Irish Research Council (Enterprise Partnership Scheme IRC/EPSPG/2020/8)
- Chi Tak Lee
- Claire M Gillan
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
This work was funded by a fellowship awarded to Claire M Gillan from MQ: transforming mental health (MQ16IP13). Claire M Gillan holds additional funding from Science Foundation Ireland’s Frontiers for the Future Scheme (19/FFP/6418), and a European Research Council (ERC) Starting Grant (ERC-H2020-HABIT). The PhD studentship of Celine Ann Fox is funded by the Government of Ireland Postgraduate Scholarship Programme (GOIPG/2020/662). The authors thank all the participants for their involvement in this study. We thank the AWARE charity and the Berkshire foundation trust that supported recruitment for the iCBT arm. We thank the individual pharmacies and General Practitioner services for their support in recruiting the antidepressant arm.
Ethics
Ethical approval for the PIP study was obtained from the Research Ethics Committee of School of Psychology, Trinity College Dublin (Approval ID: SPREC072017-01) and the Northwest-Greater Manchester West Research Ethics Committee of the National Health Service, Health Research Authority and Health and Care Research Wales (REC reference: 20/NW/0020, IRAS project ID: 270623). Informed consent, and consent to publish, was obtained from all participants prior to study participation.
Version history
- Sent for peer review:
- Preprint posted:
- Reviewed Preprint version 1:
- Reviewed Preprint version 2:
- Version of Record published:
Cite all versions
You can cite all versions using the DOI https://doi.org/10.7554/eLife.87193. This DOI represents all versions, and will always resolve to the latest one.
Copyright
© 2023, Fox et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 1,533
- views
-
- 120
- downloads
-
- 9
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading
-
- Computational and Systems Biology
- Neuroscience
Hypothalamic kisspeptin (Kiss1) neurons are vital for pubertal development and reproduction. Arcuate nucleus Kiss1 (Kiss1ARH) neurons are responsible for the pulsatile release of gonadotropin-releasing hormone (GnRH). In females, the behavior of Kiss1ARH neurons, expressing Kiss1, neurokinin B (NKB), and dynorphin (Dyn), varies throughout the ovarian cycle. Studies indicate that 17β-estradiol (E2) reduces peptide expression but increases Slc17a6 (Vglut2) mRNA and glutamate neurotransmission in these neurons, suggesting a shift from peptidergic to glutamatergic signaling. To investigate this shift, we combined transcriptomics, electrophysiology, and mathematical modeling. Our results demonstrate that E2 treatment upregulates the mRNA expression of voltage-activated calcium channels, elevating the whole-cell calcium current that contributes to high-frequency burst firing. Additionally, E2 treatment decreased the mRNA levels of canonical transient receptor potential (TPRC) 5 and G protein-coupled K+ (GIRK) channels. When Trpc5 channels in Kiss1ARH neurons were deleted using CRISPR/SaCas9, the slow excitatory postsynaptic potential was eliminated. Our data enabled us to formulate a biophysically realistic mathematical model of Kiss1ARH neurons, suggesting that E2 modifies ionic conductances in these neurons, enabling the transition from high-frequency synchronous firing through NKB-driven activation of TRPC5 channels to a short bursting mode facilitating glutamate release. In a low E2 milieu, synchronous firing of Kiss1ARH neurons drives pulsatile release of GnRH, while the transition to burst firing with high, preovulatory levels of E2 would facilitate the GnRH surge through its glutamatergic synaptic connection to preoptic Kiss1 neurons.
-
- Neuroscience
Specialized chemosensory signals elicit innate social behaviors in individuals of several vertebrate species, a process that is mediated via the accessory olfactory system (AOS). The AOS comprising the peripheral sensory vomeronasal organ has evolved elaborate molecular and cellular mechanisms to detect chemo signals. To gain insight into the cell types, developmental gene expression patterns, and functional differences amongst neurons, we performed single-cell transcriptomics of the mouse vomeronasal sensory epithelium. Our analysis reveals diverse cell types with gene expression patterns specific to each, which we made available as a searchable web resource accessed from https://www.scvnoexplorer.com. Pseudo-time developmental analysis indicates that neurons originating from common progenitors diverge in their gene expression during maturation with transient and persistent transcription factor expression at critical branch points. Comparative analysis across two of the major neuronal subtypes that express divergent GPCR families and the G-protein subunits Gnai2 or Gnao1, reveals significantly higher expression of endoplasmic reticulum (ER) associated genes within Gnao1 neurons. In addition, differences in ER content and prevalence of cubic membrane ER ultrastructure revealed by electron microscopy, indicate fundamental differences in ER function.