An observational treatment study of metacognition in anxious-depression

  1. Celine Ann Fox  Is a corresponding author
  2. Chi Tak Lee
  3. Anna Kathleen Hanlon
  4. Tricia XF Seow
  5. Kevin Lynch
  6. Siobhán Harty
  7. Derek Richards
  8. Jorge Palacios
  9. Veronica O'Keane
  10. Klaas Enno Stephan
  11. Claire M Gillan
  1. School of Psychology, Trinity College Dublin, Ireland
  2. Trinity College Institute of Neuroscience, Trinity College Dublin, Ireland
  3. Wellcome Centre for Human Neuroimaging, University College London, United Kingdom
  4. SilverCloud Science, SilverCloud Health Ltd, Ireland
  5. Department of Psychiatry, Trinity College Dublin, Ireland
  6. Tallaght Hospital, Trinity Centre for Health Sciences, Tallaght University Hospital, Ireland
  7. Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Switzerland
  8. Max Planck Institute for Metabolism Research, Germany
  9. Global Brain Health Institute, Trinity College Dublin, Ireland

Abstract

Prior studies have found metacognitive biases are linked to a transdiagnostic dimension of anxious-depression, manifesting as reduced confidence in performance. However, previous work has been cross-sectional and so it is unclear if under-confidence is a trait-like marker of anxious-depression vulnerability, or if it resolves when anxious-depression improves. Data were collected as part of a large-scale transdiagnostic, four-week observational study of individuals initiating internet-based cognitive behavioural therapy (iCBT) or antidepressant medication. Self-reported clinical questionnaires and perceptual task performance were gathered to assess anxious-depression and metacognitive bias at baseline and 4-week follow-up. Primary analyses were conducted for individuals who received iCBT (n=649), with comparisons between smaller samples that received antidepressant medication (n=82) and a control group receiving no intervention (n=88). Prior to receiving treatment, anxious-depression severity was associated with under-confidence in performance in the iCBT arm, replicating previous work. From baseline to follow-up, levels of anxious-depression were significantly reduced, and this was accompanied by a significant increase in metacognitive confidence in the iCBT arm (β=0.17, SE=0.02, p<0.001). These changes were correlated (r(647)=-0.12, p=0.002); those with the greatest reductions in anxious-depression levels had the largest increase in confidence. While the three-way interaction effect of group and time on confidence was not significant (F(2, 1632)=0.60, p=0.550), confidence increased in the antidepressant group (β=0.31, SE = 0.08, p<0.001), but not among controls (β=0.11, SE = 0.07, p=0.103). Metacognitive biases in anxious-depression are state-dependent; when symptoms improve with treatment, so does confidence in performance. Our results suggest this is not specific to the type of intervention.

eLife assessment

This valuable study advances our knowledge of the effects of anxiety/depression treatment on metacognition, demonstrating that treatment increases metacognitive confidence alongside improving symptoms. The authors provide convincing evidence for the state-dependency of metacognitive confidence, based on a large longitudinal treatment dataset. However, it is unclear to what extent this effect is truly specific to treatment, as changes in metacognitive confidence in the group receiving online therapy were not statistically different from those in the control group.

https://doi.org/10.7554/eLife.87193.3.sa0

Introduction

Metacognition refers to the ability to accurately monitor and appraise one’s own cognitive experience (Fleming and Lau, 2014). Metacognition is crucial for adaptive behaviour: it allows for flexible adjustment of behavioural strategies in order to improve performance, signals when to engage or withdraw from an activity, and guides the engagement in social interactions (Fleming and Daw, 2017; Fleming et al., 2012). Metacognitive abilities vary across individuals; one can be under- or over-confident, and these biases can be associated with maladaptive thoughts, feelings and behaviours (Philipp et al., 2020). There is a growing interest in these confidence abnormalities in psychiatry, with studies implicating alterations of metacognition in depression, obsessive-compulsive disorder, and psychosis (Hoven et al., 2019). While case-control studies have mainly found patterns of reduced confidence across several disorders, newer methods that can separate transdiagnostic dimensions of mental health using large online samples have revealed specific and bi-directional effects of confidence (Wise et al., 2023). Using these methods, studies have shown that the transdiagnostic dimension ‘anxious-depression’ is linked to under-confidence in one’s own performance, while a separate dimension ‘compulsivity and intrusive thought’ is related to elevated confidence (Rouault et al., 2018; Seow and Gillan, 2020; Hoven et al., 2022; Seow et al., 2021; Benwell et al., 2022). A major gap in this area is that studies to-date only measure metacognition and transdiagnostic psychopathology at a single time point. Therefore, it is unclear if metacognitive biases are stable, fixed traits, or if they might change with treatment response. Preliminary evidence suggests metacognition may indeed be malleable; metacognitive abilities can be improved with metacognitive interventions, such as training, in unselected online samples (Carpenter et al., 2019; Engeler and Gilbert, 2020) and in clinical populations (Jelinek et al., 2017; Lysaker et al., 2018). However, it remains unknown if metacognitive changes generalise beyond its specific training context and are associated with any real-world improvement in psychiatric symptoms. In clinical studies, research has identified confidence abnormalities in at-risk populations (Eisenacher et al., 2015; Gawęda et al., 2018), suggestive of a trait-dependence. In contrast, stimulant use disorders remitters have better metacognition than active users, suggesting state-dependence (Moeller et al., 2016). Within-subject designs are needed to extend this work and understand if metacognition can improve in parallel to symptom alleviation, or if those with greater metacognitive deficits are simply the most vulnerable to illness onset and persistence. The present study aimed to address this by examining metacognition in a large cohort of individuals before and after internet-based cognitive behavioural therapy (iCBT). iCBT has emerged as an important intervention for reducing the treatment-gap in mental healthcare provision globally; it is low-cost, scalable, geographically unconstrained and flexible (Mogoașe et al., 2017; Webb et al., 2017). iCBT offers patients standardised content and records objective metrics of treatment engagement, making it particularly well-suited to treatment-oriented research in psychiatry (Lee et al., 2023). Additionally, iCBT has demonstrated clinical effectiveness in terms of symptom improvement (Andersson et al., 2019; Eilert et al., 2021; Karyotaki et al., 2021). While one study found that iCBT modified self-reported metacognitive beliefs (Newby et al., 2014), it remains unknown if metacognitive confidence in decision-making improves following successful iCBT. In the current study, we used an objective task measure of metacognition (Fleming et al., 2014), which allowed us to test if successful treatment is linked to within-person improvements in metacognition. We also tested if any changes in metacognition were iCBT-specific, by comparing data gathered from smaller samples of individuals receiving antidepressant medication and a control group receiving no intervention. Similar to iCBT, antidepressants have established transdiagnostic efficacy (Cipriani et al., 2018; Gøtzsche and Dinnage, 2020; Skapinakis et al., 2016). However, studies examining the impact of antidepressants on cognition have typically focused on cognitive capacities other than metacognition (Prado et al., 2018; Rosenblat et al., 2015; Salagre et al., 2017; Vernon et al., 2014). Accordingly, a secondary aim of this study was to compare metacognitive changes across the different intervention arms, which may shed light on differential therapeutic mechanisms and potentially augment therapeutic decision-making in the future.

Results

Cross-sectional findings at baseline: iCBT

At baseline, age was not significantly associated with mean confidence (β=0.003, SE = 0.003, p=0.401). Males had significantly higher confidence (M=3.91, SD = 0.81), than females (M=3.75, SD = 0.86) (β=0.08, SE = 0.04, p=0.044). Mean confidence was significantly lower among those with educational attainment above undergraduate level (M=3.73, SD = 0.89), when compared to participants with attainment below undergraduate (M=3.94, SD = 0.87) (β=0.21, SE = 0.10, p=0.036). Mean confidence did not significantly differ when comparing those who had educational attainment above undergraduate (M=3.73, SD = 0.89) to those who had some or had completed an undergraduate degree (M=3.74, SD = 0.83) (β=0.01, SE = 0.08, p=0.921). When comparing task measures, mean confidence was not significantly associated with mean accuracy (β=−1.29, SE = 1.42, p=0.366), mean dot difference (task difficulty) (β=0.002, SE = 0.002, p=0.406), or mean response time (β=6.47, SE = 15.13, p=0.669).

Participants with higher levels of anxious-depression had lower levels of mean confidence (β=−0.09, SE = 0.03, p=0.008; Figure 1A and B), while those with higher levels of compulsivity and intrusive thought had elevated mean confidence (β=0.11, SE = 0.03, p=0.002; Figure 1A and C), controlling for age, gender, and education. Levels of social withdrawal were not associated with mean confidence (β=−0.05, SE = 0.03, p=0.168; Figure 1A).

Cross-sectional findings at baseline in the iCBT arm.

β=standardised beta coefficient, r=correlation coefficient, p=p-value, AD = Anxious-Depression, CIT = Compulsivity and Intrusive Thought, SW = Social Withdrawal. The error bars represent the standard error around the standardised beta coefficient. N=649. (A) AD and CIT were associated with metacognitive bias, while SW was not, using linear regression analysis. (B) The residual values for confidence (controlling for age, gender and education) were negatively correlated with AD. (C) The residual values for confidence (controlling for age, gender and education) were positively correlated with CIT.

Treatment findings: iCBT

The transdiagnostic dimensions and psychiatric scale scores all significantly improved from baseline to four-week follow-up, except for impulsivity (Figure 2A and Figure 2—figure supplement 1). In tandem with these clinical changes, there was a small but significant increase in mean confidence from baseline (M=3.78, SD=0.85) to follow-up (M=3.95, SD=0.89), (β=0.17, SE = 0.02, p<0.001, r2=0.01) (Figure 2B). Although mean accuracy did not change from baseline (M=0.71, SD=0.02) to follow-up (M=0.71, SD = 0.02) (β=0.001, SE = 0.001, p=0.299), due to the staircasing procedure, participants’ ability to detect differences between the visual stimuli improved. This was reflected as the overall increase in task difficulty to maintain the accuracy rates from baseline (dot difference: M=41.82, SD=11.61) to follow-up (dot difference: M=39.80, SD=12.62), (β=−2.02, SE = 0.44, p<0.001, r2=0.01) (Figure 2C). Additionally, the effect of time on confidence was not dependent on how much participants engaged in iCBT, as indexed by time spent in the program (β<0.01, SE <0.01, p=0.756) and percentage of the iCBT program viewed (β=0.09, SE = 0.21, p=0.650). Confidence changes did not differ across the different types of iCBT programs, when the Space from Depression and Anxiety program was compared pairwise to Space from Depression (β=−0.09, SE = 0.17, p=0.594), Space from Anxiety (β=−0.15, SE = 0.17, p=0.376), Life Skills (β=−0.15, SE = 0.19, p=0.435) or the ‘other’ category comprising miscellaneous iCBT programs (β=−0.03, SE = 0.18, p=0.881). Change in confidence was not different among those receiving concurrent treatment versus not (β=−0.03, SE = 0.06, p=0.566). To test if changes in confidence from baseline to follow-up scaled with changes in anxious-depression, we ran a repeated measure regression analyses with per-person changes in anxious-depression as an additional independent variable. We found this was the case, evidenced by a significant interaction effect of time and change in anxious-depression on confidence (β=−0.12, SE = 0.04, p=0.002). Those with the largest decrease in anxious-depression had the greatest increase in confidence. This was similarly evident in a simple correlation between change in confidence and change in anxious-depression (r(647)=-0.12, p=0.002) (Figure 2D). This effect was specific to anxious-depression; the interaction effect of time and change in compulsivity and intrusive thought on mean confidence was not significant (β=−0.06, SE = 0.05, p=0.221). Similarly, the significant interaction effect of time and anxious-depression on mean confidence held when including change in the other transdiagnostic dimensions (compulsivity and intrusive thought and change in social withdrawal) as covariates in the model (β=−0.07, SE = 0.03, p=0.005). The interaction effect of time and change in anxious-depression on task difficulty was not significant (β=0.14, SE = 0.69, p=0.835). To test the extent to which baseline differences in mean confidence or anxious depression might drive the results, we re-ran these regression analyses with baseline measures instead of change indices. There was no significant interaction effect of time and baseline confidence (β=0.06, SE = 0.04, p=0.144) or an interaction effect of time and baseline anxious-depression (β=−0.03, SE = 0.05, p=0.611) on mean confidence. Exploratory analyses further tested the specificity of these effects to anxious-depression by examining the interaction effect of time and change in each psychiatric score on mean confidence. Changes in trait anxiety (β=−0.08, SE = 0.02, p=0.002), depression (β=−0.06, SE = 0.02 p=0.011) and alcohol misuse (β=−0.05, SE = 0.02, p=0.037) also showed an association with changes in confidence (Figure 2E and Figure 2—figure supplement 2).

Figure 2 with 2 supplements see all
Treatment findings in the iCBT arm.

β=standardised beta coefficient, AD = Anxious-Depression, CIT = Compulsivity and Intrusive Thought, SW = Social Withdrawal, OCD = Obsessive compulsive disorder, r=correlation coefficient, p=p-value (unadjusted), ***=p < 0.001, **=p < 0.01, *=p < 0.05. The error bars represent the standard error around the standardised beta coefficient. Regression analyses were used for all tests. N=649. (A) Psychopathology symptoms improved with four weeks of iCBT. (B) Confidence was significantly higher and, (C) the task was more difficult at 4-week follow-up. (D) Those with the largest improvements in AD had the greater increases in confidence. (E) Change in confidence also scaled with improvements in trait anxiety, depression and alcohol misuse.

Comparing iCBT, antidepressant and control groups

When comparing the three groups directly, ANOVA analysis predicting anxious-depression scores with group and time as independent variables revealed a main effect of time (F(1, 1632)=62.99, p<0.001), a main effect of group (F(2, 1632)=249.74, p<0.001), and an interaction effect of group and time (F(2, 1632)=9.23, p<0.001). Examining simple effects in the antidepressant arm, there was a significant reduction in anxious-depression from baseline to follow-up (β=−0.61, SE = 0.09, p<0.001). Among controls, levels of anxious-depression did not significantly change (β=0.10, SE = 0.06, p=0.096). Further details of transdiagnostic clinical changes for the antidepressant and controls groups are presented in Figure 3A and Figure 3—figure supplement 1. Predicting confidence scores using ANOVA analysis with group and time as independent variables revealed a main effect of time (F(1, 1632)=16.26, p<0.001), and no significant main effect of group (F(2, 1632)=2.35, p=0.096). The interaction effect of group and time on mean confidence was not significant (F(2, 1632)=0.60, p=0.550), suggesting that change in confidence did not differ across the three groups. Tests of simple effects revealed that mean confidence significantly increased from baseline (M=3.77, SD=0.88) to follow-up (M=4.07, SD=0.79) in the antidepressant arm (β=0.31, SE = 0.08, p<0.001) (Figure 3B). Among controls, there was no significant change in confidence from baseline (M=3.68, SD=0.86) to follow-up (M=3.79, SD=0.92) (β=0.11, SE = 0.07, p=0.103) (Figure 3B). With respect to task performance, there was a significant main effect of time (F(1, 1632)=15.17, p=0.001) and group (F(2, 1632)=4.56, p=0.011) on mean dot difference when the three groups were included in the model. The interaction effect of time and group on mean dot difference was not significant (F(2, 1632)=1.91, p=0.148), suggesting no differences across the groups in task difficulty changes. In the antidepressant arm, mean dot difference decreased from baseline (M=41.2, SD=13.3) to follow-up (M=35.3, SD=13.1) (β=−5.91, SE = 1.25, p<0.001), indicating increased task difficulty. There was no significant change in task difficulty among controls from baseline (M=43.0, SD=11.8) to follow-up (M=41.4, SD=13.6) (β=−1.64, SE = 1.30, p=0.210) (Figure 3C). While our sample was underpowered to examine individual differences, we conducted an exploratory analysis examining the connection between changes in anxious-depression symptoms and changes in confidence in the antidepressant and controls groups. When examining the effects of time, group and anxious-depression change on mean confidence, there was a significant interaction effect of time and anxious-depression change on mean confidence (F(1, 1626)=4.04, p=0.045), suggesting change in confidence is associated with change in anxious-depression. There was no significant three-way interaction of anxious-depression change, time and group on mean confidence when comparing the three groups (F(2, 1626)=0.08, p=0.928), indicating that the significant association between confidence change and anxious-depression change was not specific to any group. Although not significant, the association between change in confidence and change in anxious-depression was in the expected negative direction in the antidepressant arm (r(80)=-0.10, p=0.381), and among controls (r(86)=-0.17, p=0.111) (Figure 3D).

Figure 3 with 1 supplement see all
Comparing iCBT, antidepressant and control groups.

β=standardised beta coefficient, AD = Anxious-Depression, CIT = Compulsivity and Intrusive Thought, SW = Social Withdrawal, OCD = Obsessive compulsive disorder, r=correlation coefficient, p=p-value, ***=p < 0.001, **=p < 0.01, *=p < 0.05. The error bars represent the standard error around the standardised beta coefficient. Regression analyses were used for tests. (A) The majority of psychiatric scales improved in the antidepressant arm (N=82) after 4 weeks of treatment, while the controls (N=88) only had significant reductions in OCD symptoms and alcohol misuse at follow-up. (B) While confidence increased in the antidepressant arm, there was no significant change in confidence among controls. The larger increase in confidence in the antidepressant arm compared to controls was trended towards significant. (C) The antidepressant arm had a greater increase in task difficulty (a reduction in dot difference across stimuli) from baseline to follow-up, relative to controls. (D) Although not significant, the association between change in confidence and change in anxious-depression was in the expected negative direction in the antidepressant arm and among controls.

Discussion

Metacognitive biases are linked to transdiagnostic dimensions of mental health, but it is presently unclear if these biases are stable traits, or if they fluctuate alongside symptoms and change during the course of treatment (Seow et al., 2021). To answer these questions, we administered a previously validated adaptive task of metacognitive ability that controls for objective performance differences (Fleming et al., 2014) in a large sample of individuals before and after four weeks of iCBT or antidepressant medications (Lee et al., 2023). As expected, a 4-week course of iCBT or antidepressant medication led to transdiagnostic improvements in mental health (Cipriani et al., 2018; Gøtzsche and Dinnage, 2020; Skapinakis et al., 2016). Alongside this, there was a significant increase in metacognitive confidence following four weeks of iCBT or antidepressant medication. Not simply a practice effect, we found that individuals in the iCBT arm with the greatest improvements in anxious-depression had the largest increase in confidence at follow-up. This association with clinical improvements was specific to metacognitive changes, and not changes in task performance, suggesting that changes in confidence do not merely reflect greater task familiarity at follow-up. These findings suggest that metacognitive biases in anxious-depression are state-dependent. This builds on previous findings in small samples that have shown iCBT improves self-reported metacognitive self-beliefs (Newby et al., 2014) and that metacognition can be altered through adaptive training (Carpenter et al., 2019; Engeler and Gilbert, 2020; Jelinek et al., 2017; Lysaker et al., 2018). At baseline, we replicated the previously observed bi-directional associations between metacognitive bias and anxious-depression and compulsivity and intrusive thought (Rouault et al., 2018; Seow and Gillan, 2020; Hoven et al., 2022; Benwell et al., 2022). While higher levels of anxious-depression is associated with lower confidence, those with higher levels of compulsivity and intrusive thought have elevated confidence. This is a somewhat surprising dissociation, as compulsivity and anxious-depression are themselves positively correlated in the population. One way this can be reconciled is if the mechanisms underlying these opposing confidence biases are distinct. In anxious-depression, there appears to be more pervasive metacognitive biases that affect confidence in many domains and levels of a metacognitive hierarchy (spanning confidence in low level perceptual decisions to ideas of self-worth) (Hoven et al., 2022; Seow et al., 2021). In contrast, inflated confidence in compulsivity may be based on more specific biases in learning and inference (Seow and Gillan, 2020). The present study was observational and therefore did not randomly assign participants to a different treatments. To partially remediate that limitation, we included two smaller groups receiving antidepressant medication and a control group. Levels of transdiagnostic psychiatric dimensions remained stable across time among controls, while they significantly improved in the antidepressant arm. Similarly to iCBT, we found that confidence improved in the antidepressant group, but not among controls. The interaction, however, was not significant, meaning that we cannot reject the null hypothesis that confidence improved to the same degree across the three groups. As increased task difficulty among clinical groups was not significantly greater relative to controls, changes in task difficulty may simply reflect greater task familiarity at follow-up across groups, as opposed to gains in general perceptual performance among clinical arms. Examining the three groups together, the data suggests that confidence changes are unlikely to be treatment specific, rather, confidence fluctuates in tandem with anxious-depression. This was evident in an overall association between change in anxious-depression and change in confidence that was not modified by treatment arm. Additionally, levels of iCBT engagement and concurrent treatments did not bolster changes in confidence. Overall, the results indicated that metacognition fluctuates with anxious-depression state, regardless of treatment type or exposure. Future research with larger samples are required to address this definitively.

Limitations and future directions

Confidence change and anxious-depression change were significantly but weakly associated. Similarly, the relative change in confidence across treatment arms was small. Therefore, while tests of metacognitive confidence can inform theoretical models, like most cognitive tests, they are likely of limited utility in clinical practice, at least when used in isolation (Mogoașe et al., 2017). Given the complexity of mental health causes and presentations, multivariable models are needed to see practical value from such tests. We did not assess confidence or anxious-depression to treatment cessation and so the causal path and temporal dependence, if they exist, cannot be derived from these data. Future research should consider assessing metacognition and anxious-depression continuously through treatment, in order to elucidate the causal relationship between anxious-depression and metacognition with mediation analysis (Newby et al., 2014). More intensive, repeating testing in future studies may also reveal the temporal window at which metacognition has the propensity to change, which could be more momentary in nature. While this study examined changes in metacognition with iCBT generally, future research should examine if the strength of the association between confidence change and anxious-depression change is greater following iCBT modules targeting metacognition or following metacognitive intervention (Philipp et al., 2020). The iCBT programs in this study primarily targeted depression and anxiety, which may explain why changes in confidence did not scale with improvements in compulsivity. Future research is required to assess if treatments aimed at compulsive disorders decrease the over-confidence commonly observed in those high in compulsivity and intrusive thought. As the antidepressant and control groups were much smaller than the iCBT arm, we were unable to compare changes in confidence across the types of antidepressant medications individuals received and we were underpowered more generally for individual differences analyses and multi-arm comparisons. Exploratory analyses were nonetheless presented and can form the basis for future investigations.

Conclusions

Our findings replicated the cross-sectional evidence that higher levels of anxious-depression are associated with under-confidence. We demonstrate that metacognitive confidence increases following four weeks of iCBT or antidepressant treatment. Overall, we observed that the greater the improvement in anxious-depression, the more confident participants became, which did not appear to be dependent on treatment type. This suggests that metacognitive biases in anxious-depression are state-dependent and might be normalised through clinical improvements.

Methods

Participants

Participants were recruited as part of the Precision in Psychiatry (PIP) study (Lee et al., 2023), an observational, longitudinal study in which participants underwent a 4-week course of iCBT or antidepressant medication. Further details of the PIP study procedures that are not specific to this study can be found in a prior publication (Lee et al., 2023). Ethical approval for the PIP study was obtained from the Research Ethics Committee of School of Psychology, Trinity College Dublin (Approval ID: SPREC072017-01) and the Northwest-Greater Manchester West Research Ethics Committee of the National Health Service, Health Research Authority and Health and Care Research Wales (REC reference: 20/NW/0020, IRAS project ID: 270623). Informed consent, and consent to publish, was obtained from all participants prior to study participation. A power analysis was carried out using effect sizes from a previous study examining cross-sectional associations between metacognition and anxious-depression, and compulsivity and intrusive thought (Rouault et al., 2018). Sample sizes of N=454 and N=332 respectively were required to detect these associations with 80% power. The sample sizes of the antidepressant arm and control group were smaller and used for secondary and more exploratory analyses.

iCBT Arm

Individuals initiating iCBT provided by SilverCloud Health were recruited from two sites: (1) the National Health Service Berkshire Foundation in the UK and (2) Aware mental health charity in Ireland. Participants included in the study either started their iCBT intervention ≤2 days prior to signing up, or provided a treatment start date in the near future, and scored ≥10 on the Work and Social Adjustment Scale (WSAS) at baseline, which indicated significant functional impairment due to clinical symptoms (Mundt et al., 2002). Figure 4A shows the disposition of participants throughout the study. N=2404 were screened, of whom N=1,496 were eligible, N=836 completed baseline assessments and met inclusion criteria. A final N=649 completed and met inclusion criteria for follow-up assessments. While study follow-up data was collected after four weeks of treatment, iCBT could last up to 12 weeks (Lee et al., 2023). The final sample was, on average, 32.2 years old (SD=11.0), mostly female (n=501, 77.4%), living in the United Kingdom (n=546, 84.4%), and had some or completed undergraduate level education (n=342, 52.9%) (Table 1).

Study methods.

(A) Participant flow chart (CONSORT chart). Participants were considered ‘completers’ if they had metacognitive and transdiagnostic psychiatric dimension data at baseline and follow-up and met task inclusion criteria. (B) Overview of study design from study intake (week 0) to follow-up (week 4) assessments across groups. (C) Metacognitive (visuo-perceptual decision-making) task design (N=210 trials). On each trial, participants were asked to judge and choose the sunflower that contained more seeds (i.e. higher number of dots) and then provide a confidence rating on their decision.

Antidepressant Arm

Individuals were recruited globally using advertisements placed on Google search, in addition to social media platforms, mental health websites, local pharmacies and General Practitioner waiting rooms. Participants were included if they started or planned to start treatment ≤2 days of study sign-up, scored ≥10 on the WSAS at baseline, and provided a valid photograph of an antidepressant medication prescription. N=270 individuals were screened, of whom N=174 were eligible, N=102 completed and met inclusion criteria at baseline and a final N=82 had follow-up data (Figure 4A). Participants were mostly female (n=60, 73.2%), mean age = 30.5 (10.5), were living in Ireland or the United Kingdom (n=66, 80.5%) and had some or completed undergraduate level education (n=49, 59.8%) (Table 1).

Control group

Participants in the no treatment control group were recruited through university mailings lists and advertisements posted online and around Trinity College Dublin. Participants included in this arm scored <10 on an adapted version of the WSAS (where they rated functional impairment from their general problems rather than mental health problems) and self-reported that they had no current mental health problems and were not undergoing treatment for any mental health problems at the time of screening. N=444 individuals were screened, of whom N=245 were eligible, N=113 had baseline data and a final N=88 completed follow-up assessments and met inclusion criteria for the study (Figure 4A). Participants in the control group were matched for sociodemographic characteristics in the antidepressant arm, with no significant differences across the two groups in gender (Fisher’s exact test p=0.498), or age (Welch’s t(166.66)=−0.81, p=0.420), or levels of educational attainment (χ2(2)=0.66, p=0.718). Participants in the control group came from the United Kingdom and Ireland (>93%), whereas the antidepressant arm was more international, with 20% coming from other countries (χ2(2)=13.59, p=0.001).

Table 1
Baseline sociodemographic characteristics of participants.
CharacteristiciCBT (n=649)*Antidepressant (n=82)Control (n=88)F/X2 (df)p
Gender, No. (%)0.414
Male141 (21.8)20 (24.4)22 (25.0)
Female501 (77.4)60 (73.2)66 (75.0)
Other5 (0.8)2 (2.4)0 (0.0)
Age, M (SD)32.2 (11.0)30.5 (10.5)29.1 (12.0)3.68 (2, 813)0.026
Country of residence, No. (%)211.73 (4)<0.001
Ireland89 (13.8)32 (39.0)58 (65.9)
United Kingdom546 (84.4)34 (41.5)24 (27.3)
Other12 (1.9)16 (19.5)6 (6.8)
Highest level of education, No. (%)7.11 (4)0.130
Below undergraduate147 (22.7)12 (14.6)13 (14.8)
Some/completed undergraduate342 (52.9)49 (59.8)57 (64.8)
Above undergraduate158 (24.4)21 (25.6)18 (20.5)
  1. *

    Two participants in the iCBT arm were missing data for age, gender, country of residence and highest level of education.

  2. Gender proportions were compared using Fisher’s exact due to cell count <5 in the ‘Other’ group.

When comparing sociodemographic characteristics across the three study groups, there were no significant differences between groups in gender proportions (Fisher’s exact test p=0.414), or levels of educational attainment (χ2(4)=7.11, p=0.130), as reported in Table 1. Age differed across the three arms (F(2, 813)=3.68, p=0.026), with post hoc Tukey tests indicating that mean age was higher in the iCBT arm (M=32.2, SD=11.0) compared to the controls (M=29.1, SD=12.0) (padj = 0.034), but not when compared to the antidepressant arm (M=30.5, SD=10.5) (padj = 0.376) (Table 1). The countries participants were living in varied across the study arms (χ2(4)=211.73, p<0.001), as there was a higher proportion of individuals in the iCBT arm living in the UK (n=546, 84.4%) when compared to the antidepressant arm (n=34, 41.5%) and the control group (n=24, 27.3%) (Table 1).

Procedure

Figure 4B shows an overview of the study design, including the assessments involved at each timepoint. For the purposes of this study, we focused on a select set of sociodemographic characteristics (gender, age, country of residence, level of educational attainment), self-reported psychiatric questionnaires, metacognitive task performance and treatment data from the PIP study (Lee et al., 2023).

Self-reported psychiatric questionnaires

Participants completed the WSAS (Mundt et al., 2002) at baseline and follow-up. Individuals in the iCBT and antidepressant arms received the original WSAS, which asks them to rate their level of functional impairment from mental health problems. Individuals in the control group received an adapted version of the WSAS, which asks them to rate their level of impairment from ‘their problems’ more generally. Each WSAS item was scored from 0 ‘not at all’ to 8 ‘very severely’, with overall scores ranging from 0 to 40. Higher WSAS scores indicated higher levels of functional impairment (Mundt et al., 2002). Participants completed nine standard self-report clinical questionnaires at baseline and four-week follow-up that assess a variety of psychiatric symptoms, including depression (Zung Self-Rating Depression Scale; Zung, 1965), trait anxiety (State Trait Anxiety Inventory) (Spielberger et al., 1983), schizotypy (Short Scales for Measuring Schizotypy) (Mason et al., 2005), impulsivity (Barratt Impulsiveness Scale 11) (Patton et al., 1995), obsessive-compulsive disorder (OCD) (Obsessive-Compulsive Inventory-Revised, OCI-R) (Foa et al., 2002), social anxiety (Liebowitz Social Anxiety Scale; Liebowitz, 1987), eating disorders (Eating Attitudes Test) (Garner et al., 1982), apathy (Apathy Evaluation Scale) (Marin et al., 1991), and alcohol misuse (Alcohol Use Disorders Identification Test) (Saunders et al., 1993).

Metacognitive task

Participants completed a visuo-perceptual decision-making task (Fleming et al., 2014) to assess metacognition (Figure 4C). On each trial, participants were shown a fixation cross for 1000 milliseconds (ms), followed by two sunflowers with a varying number of seeds for 300ms. Participants then had unlimited time to make a judgement on which of two sunflower stimuli contained more seeds and then rated their confidence in each judgement, on a scale from ‘1=Guessing’ to ‘6=Certain’. There was a total of 210 trials, divided equally into five blocks. While participants were given feedback on which sunflower they had chosen for 500ms, there was no trial-level feedback provided on their performance. Split-half reliability was high for baseline mean confidence, as indicated by the correlation between odd and even trials across each participant in the iCBT arm (r(647)=0.98, p<0.001). Mean accuracy was tightly controlled in this task using a ‘two-down one-up’ staircase procedure, in which equally sized changes in dot difference occurred after each incorrect response and after two consecutive correct responses. This maintained objective performance across all participants at a desired level of 70% correct, crucial for estimating metacognition without any confound of real performance differences. Changes in dot difference were calculated using log-space, with a start log difference of 4.2 (+70 dots). Differences in step size changed by ± 0.4 for the first five trials, ± 0.2 for the next five trials and ± 0.1 for the remainder of the task. Dot difference on each trial could range from six dots (1.79 in log-space, the most difficult to discriminate) to 81 dots (4.39 in log-space, the easiest to discriminate).

Treatment data

Every week, participants self-reported if they had adhered to iCBT or their antidepressant medications each week. Participants also reported if they had started any new concurrent medications and/or psychological treatments for mental health each week. Starting new medication and/or psychological treatments and treatment nonadherence during the study were not grounds for exclusion in any group. Objective indicators of treatment engagement were provided by SilverCloud for 640 participants in the iCBT arm, which comprised of percentage of program viewed, time (minutes) spent in the program, and program type. In relation to program type, Space from Depression was the most common iCBT program (n=158, 24.7%), followed by Space from Generalised Anxiety Disorder (n=96, 15.0%), Life Skills Online (n=89, 13.9%), Space from Anxiety (n=84, 13.1%), and Space from Depression and Anxiety (n=77, 12.0%). Given the overlapping content of Space from Generalised Anxiety Disorder and Space from Anxiety, these programs were merged into one category ‘Space from Anxiety’ (n=180, 28.1%) for analyses. The ‘Other’ category (n=136, 21.3%) comprised of the following iCBT programs: Space from Stress (n=33, 5.2%), Space from Social Anxiety (n=20, 3.1%), Space for Resilience (n=20, 3.1%), Space for Perinatal Wellbeing (n=18, 2.8%), Space in Chronic Pain from Depression and Anxiety (n=12, 1.9%), Space from Health Anxiety (n=9, 1.4%), Space from OCD (n=7, 1.1%), Space from Panic (n=6, 0.9%), Space from Phobia (n=5, 0.8%), Space for Sleep (n=4, 0.6%), Space in Lung Conditions from Depression and Anxiety (n=1, 0.2%) and Space from Money Worries (n=1, 0.2%), which were merged together into one category for analysis due to low N in each program.

Data preparation and analysis

Self-reported psychiatric questionnaire

Individual scores on dimensions of anxious-depression, compulsivity and intrusive thought, and social withdrawal were calculated by multiplying each of the 209 item scores on the nine above-mentioned self-report clinical questionnaires by the 209 corresponding item weights from a previously published factor analysis on these scales (Gillan et al., 2016). Dimension scores were scaled to centre on zero, with higher scores indicating higher levels of transdiagnostic psychopathology.

To determine the proportion of careless/inattentive responders on the self-report clinical questionnaire, we included ‘catch’ questions that were embedded in the OCI-R (If you are paying attention to these questions, please select “A little”) and the WSAS (If you are paying attention to these questions, please select “Not at all”) at baseline and in the WSAS at all subsequent assessments. In the iCBT arm, 54 (8.3%) participants failed at least one of the checks, with just nine (1.4%) failing more than one. Additionally, eight (9.8%) individuals in the antidepressant arm and seven (8.0%) in the control group failed at least one of the catch items. Given the small number of participants that failed more than one attention checks and that all those participants passed the task exclusion criteria, these individuals were retained for subsequent analyses. Additionally, excluding those that failed more than one catch item in the iCBT arm did not affect the significance of results, including the change in confidence (β=0.16, SE = 0.02, p<0.001), change in anxious-depression (β=−0.32, SE = 0.03, p<0.001), and the association between change in confidence and change in anxious-depression (r(638)=-0.10, p=0.011).

Metacognition task

The perceptual decision-making task performance was used to quantify our primary cognitive outcome measure, metacognitive bias, the mean confidence rating across trials. Task difficulty was measured as the mean dot difference across trials, where more difficult trials had a lower dot difference between stimuli. Mean reaction time to stimuli choice across trials was measured in seconds and task accuracy was calculated as the mean proportion of correct responses across trials. As metacognitive efficiency (the extent to which confidence estimates map onto objective performance) was not previously associated with transdiagnostic dimensions cross-sectionally (Rouault et al., 2018; Hoven et al., 2022; Benwell et al., 2022), the state-dependence of efficiency was not examined in this study.

We employed a number of exclusion criteria to ensure high data quality from the metacognitive task. Firstly, we planned to exclude participants if they selected the right or left sunflower on greater than 95% of trials, but none met this criterion. Second, we excluded participants with mean accuracy less than 0.60 and greater than 0.85, indicating the staircase procedure did not converge within acceptable bounds (n=38 at baseline and n=34 at follow-up excluded in the iCBT arm; n=8 at baseline and n=7 at follow-up excluded in the antidepressant arm; n=6 at baseline and n=5 at follow-up excluded in the control group).

Treatment data

Treatment adherence was high by week 3 in both clinical arms, with over ≥95% of the iCBT arm still undergoing treatment (i.e. 98% at weekly check-in (WCI) 1, 97% at WCI 2 and 95% at WCI 3) and ≥99% of the antidepressant group reported still taking antidepressant medication (i.e. 100% at WCI 1 and WCI 2 and 99% at WCI 3). In the antidepressant arm, four participants altered the dosage of their medication during the study participation (n=2 took less than prescribed and n=2 took more than prescribed). In terms of concurrent treatment, 175 (27.0%) in the iCBT arm were receiving another treatment during the study, of which 48 (7.4%) were taking concurrent medication for a mental health problem and 145 (22.3%) were receiving a concurrent form of psychotherapy. For the antidepressant group, 33 (40.2%) were receiving another treatment during the study, with 6 (7.3%) taking at least one other medication for a mental health problem and 2 (39.0%) were receiving some form of psychotherapy. Thus, there were partial overlaps in the treatments received across the two clinical arms. Within the control group, two participants (2.3%) started taking medication during the study and none initiated psychotherapy.

Statistical analysis

We tested for relationships between baseline task measures and the psychiatric symptom dimensions using linear regression analysis, controlling for age, gender, and education. To examine pre-post changes, we carried out linear mixed-effects models with measures of metacognition or psychopathology as the dependent variable, time (baseline = 0; follow-up=1) as the independent variable and participants as random effects. To determine the association between change in confidence and change in anxious-depression, we used (1) Pearson correlation analysis to correlate change indices for both measures and, (2) regression-based repeated-measures analysis: mean confidence ~time*anxious-depression score change, where mean confidence is entered with two datapoints (one for pre- and one for post-treatment i.e., within-person) and anxious-depression change is a single value per person (between-person change score). Exploratory linear regression analyses tested the specificity of the effects, replacing anxious-depression with each of the measures of psychopathology in turn as follows: Mean confidence ~time*dimension/psychiatric scale score change. We additionally ran regression analyses to test if concurrent treatment or the degree of objective engagement in iCBT interacted with the effect of time on mean confidence. Exploratory ANOVA analyses were also conducted to compare changes in anxious-depression, task difficulty and confidence across the three arms directly. For all tests, statistical significance was defined as p<0.05, with two-tailed p-values used. All regressors were scaled as Z scores to compare the regression coefficients of independent variables within each model. Adjustments for multiple comparisons were not conducted for analyses of replicated effects, or exploratory analyses (Althouse, 2016). The code to reproduce statistical analyses are available at https://osf.io/89xzq/. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. All data from the PIP study will be made publicly available once the primary treatment prediction analysis of the project are published on a pre-print server.

Data availability

The code to reproduce statistical analyses are available at https://osf.io/89xzq/. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. All data from the PIP study will be made publicly available once the primary treatment prediction analysis of the project are published.

The following data sets were generated
    1. Fox CA
    2. Gillan C
    (2023) Open Science Framework
    ID 89xzq. An observational treatment study of metacognition in anxious-depression.

References

    1. Fleming SM
    2. Dolan RJ
    3. Frith CD
    (2012) Metacognition: computation, biology and function
    Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 367:1280–1286.
    https://doi.org/10.1098/rstb.2012.0021
    1. Foa EB
    2. Huppert JD
    3. Leiberg S
    4. Langner R
    5. Kichic R
    6. Hajcak G
    7. Salkovskis PM
    (2002)
    The Obsessive-Compulsive Inventory: development and validation of a short version
    Psychological Assessment 14:485–496.
    1. Liebowitz MR
    (1987) Social phobia
    Modern Problems of Pharmacopsychiatry 22:141–173.
    https://doi.org/10.1159/000414022
  1. Book
    1. Spielberger C
    2. Gorsuch R
    3. Lushene R
    4. Vagg P
    5. Jacobs G
    (1983)
    Manual for the State-Trait Anxiety Inventory (Form Y1 – Y2)
    Palo Alto, CA: Consulting Psychologists Press.

Article and author information

Author details

  1. Celine Ann Fox

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
    Contribution
    Conceptualization, Formal analysis, Funding acquisition, Investigation, Visualization, Writing – original draft, Writing – review and editing
    For correspondence
    foxce@tcd.ie
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1740-3765
  2. Chi Tak Lee

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
    Contribution
    Data curation, Formal analysis, Investigation, Writing – review and editing
    Competing interests
    The PhD studentship of Chi Tak Lee is co-funded by SilverCloud Health and the Irish Research Council
  3. Anna Kathleen Hanlon

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
    Contribution
    Investigation, Project administration, Writing – review and editing
    Competing interests
    No competing interests declared
  4. Tricia XF Seow

    Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
    Contribution
    Resources, Methodology
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5930-8929
  5. Kevin Lynch

    School of Psychology, Trinity College Dublin, Dublin, Ireland
    Contribution
    Investigation, Project administration, Writing – review and editing
    Competing interests
    No competing interests declared
  6. Siobhán Harty

    SilverCloud Science, SilverCloud Health Ltd, Dublin, Ireland
    Contribution
    Methodology, Writing – review and editing
    Competing interests
    Siobhán Harty is a current employees of SilverCloud Health
  7. Derek Richards

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. SilverCloud Science, SilverCloud Health Ltd, Dublin, Ireland
    Contribution
    Methodology, Writing – review and editing
    Competing interests
    Derek Richards is a current employees of SilverCloud Health
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0871-4078
  8. Jorge Palacios

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. SilverCloud Science, SilverCloud Health Ltd, Dublin, Ireland
    Contribution
    Methodology, Writing – review and editing
    Competing interests
    Jorge Palacios is a current employees of SilverCloud Health
  9. Veronica O'Keane

    1. Department of Psychiatry, Trinity College Dublin, Dublin, Ireland
    2. Tallaght Hospital, Trinity Centre for Health Sciences, Tallaght University Hospital, Dublin, Ireland
    Contribution
    Methodology, Writing – review and editing
    Competing interests
    No competing interests declared
  10. Klaas Enno Stephan

    1. Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
    2. Max Planck Institute for Metabolism Research, Cologne, Germany
    Contribution
    Methodology, Writing – review and editing
    Competing interests
    Klaas Enno Stephan acknowledges support by the René and Susanne Braginsky Foundation and the ETH Foundation
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-8594-9092
  11. Claire M Gillan

    1. School of Psychology, Trinity College Dublin, Dublin, Ireland
    2. Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland
    3. Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
    Contribution
    Funding acquisition, Methodology, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-9065-403X

Funding

MQ: Transforming Mental Health (MQ16IP13)

  • Claire M Gillan

Science Foundation Ireland (19/FFP/6418)

  • Claire M Gillan

European Research Council (ERC-H2020-HABIT)

  • Claire M Gillan

Government of Ireland Postgraduate Scholarship Programme (GOIPG/2020/662)

  • Celine Ann Fox

Irish Research Council (Enterprise Partnership Scheme IRC/EPSPG/2020/8)

  • Chi Tak Lee
  • Claire M Gillan

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

This work was funded by a fellowship awarded to Claire M Gillan from MQ: transforming mental health (MQ16IP13). Claire M Gillan holds additional funding from Science Foundation Ireland’s Frontiers for the Future Scheme (19/FFP/6418), and a European Research Council (ERC) Starting Grant (ERC-H2020-HABIT). The PhD studentship of Celine Ann Fox is funded by the Government of Ireland Postgraduate Scholarship Programme (GOIPG/2020/662). The authors thank all the participants for their involvement in this study. We thank the AWARE charity and the Berkshire foundation trust that supported recruitment for the iCBT arm. We thank the individual pharmacies and General Practitioner services for their support in recruiting the antidepressant arm.

Ethics

Ethical approval for the PIP study was obtained from the Research Ethics Committee of School of Psychology, Trinity College Dublin (Approval ID: SPREC072017-01) and the Northwest-Greater Manchester West Research Ethics Committee of the National Health Service, Health Research Authority and Health and Care Research Wales (REC reference: 20/NW/0020, IRAS project ID: 270623). Informed consent, and consent to publish, was obtained from all participants prior to study participation.

Version history

  1. Sent for peer review: March 30, 2023
  2. Preprint posted: April 25, 2023 (view preprint)
  3. Preprint posted: July 13, 2023 (view preprint)
  4. Preprint posted: September 27, 2023 (view preprint)
  5. Version of Record published: October 11, 2023 (version 1)

Cite all versions

You can cite all versions using the DOI https://doi.org/10.7554/eLife.87193. This DOI represents all versions, and will always resolve to the latest one.

Copyright

© 2023, Fox et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,174
    views
  • 79
    downloads
  • 1
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Celine Ann Fox
  2. Chi Tak Lee
  3. Anna Kathleen Hanlon
  4. Tricia XF Seow
  5. Kevin Lynch
  6. Siobhán Harty
  7. Derek Richards
  8. Jorge Palacios
  9. Veronica O'Keane
  10. Klaas Enno Stephan
  11. Claire M Gillan
(2023)
An observational treatment study of metacognition in anxious-depression
eLife 12:RP87193.
https://doi.org/10.7554/eLife.87193.3

Share this article

https://doi.org/10.7554/eLife.87193

Further reading

    1. Neuroscience
    Ivan Tomić, Paul M Bays
    Research Article

    Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or ‘iconic’ memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

    1. Neuroscience
    Emilio Salinas, Bashirul I Sheikh
    Insight

    Our ability to recall details from a remembered image depends on a single mechanism that is engaged from the very moment the image disappears from view.