The Importance of Individual Beliefs in Assessing Treatment Efficacy: Insights from Neurostimulation Studies

  1. MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
  2. Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
  3. Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
  4. School of Psychology, University of Surrey, United Kingdom
  5. Department of Psychiatry, University of California San Diego, San Diego, United States
  6. Temerty Centre for Therapeutic Brain Intervention at the Centre for Addiction and Mental Health and Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada

Peer review process

Not revised: This Reviewed Preprint includes the authors’ original preprint (without revision), an eLife assessment, and public reviews.

Read more about eLife’s peer review process.

Editors

  • Reviewing Editor
    Noah Philip
  • Senior Editor
    Michael Frank
    Brown University, Providence, United States of America

Reviewer #1 (Public Review):

Randomized clinical trials use experimental blinding and compare active and placebo conditions in their analyses. In this study, Fassi and colleagues explore how individual differences in subjective treatment (i.e., did the participant think they received the active or placebo treatment) influence symptoms and how this is related to objective treatment. The authors address this highly relevant and interesting question using a powerful method by (re-)analyzing data from four published neurostimulation studies and including subjective treatment in statistical models explaining treatment response. The major strengths include the innovative and important research question, the inclusion of four different studies with different techniques and populations to address this question, sound statistical analyses, and findings that are of high interest and relevance to the field.

My main suggestion is that authors reconsider the description of the main conclusion to better integrate and balance all findings. Specifically, the authors conclude that (e.g., in the abstract) "individual differences in subjective treatment can explain variability in outcomes better than the actual treatment", which I believe is not a consistent conclusion across all four studies as it does not appropriately consider important interactions with objective treatment observed in study 2 and 3. In study 2, the greatest improvement was observed in the group that received TMS but believed they received sham. While subjective treatment was associated with improvement regardless of objective active or sham treatment, improvement in the objective active TMS group who believed they received sham suggests the importance of objective treatment regardless of subjective treatment. In Study 3, including objective treatment in the model predicted more treatment variance, further suggesting the predictive value of objective treatment. In addition to updating the conclusions to better reflect this interaction, I suggest authors include the proportion of participants in each subjective treatment group that actually received active or sham treatment to better understand how much of the subjective treatment is explained by objective treatment. I think it is particularly important to better integrate and more precisely communicate this finding, because the conclusions may otherwise be erroneously interpreted as improvements after treatment only being an effect of subjective treatment or sham.

The paper will have significant impact on the field. It will promote further investigation of the effects of sham vs active treatment by the introduction of the terms subjective treatment vs objective treatment and subjective dosage that can be used consistently in the future. The suggestions to assess the expectation of sham vs active earlier on in clinical trials will advance the understanding of subjective treatment in future studies. Overall, I believe the data will substantially contribute to the design and interpretation of future clinical trials by underscoring the importance of subjective treatment.

Reviewer #2 (Public Review):

This manuscript focuses on the clinical impact of subjective experience or treatment with transcranial magnetic stimulation and transcranial direct current stimulation studies with retrospective analyses of 4 datasets. Subjective experience or treatment refers to the patient level thought of receiving active or sham treatments. The analyses suggest that subjective treatment effects are an important and under appreciated factor in randomized controlled trials. The authors present compelling evidence that has significance in the context of other modalities of treatment, treatment for other diseases, and plans for future randomized controlled trials. Other strengths included a rigorous approach and analyses. Some aspects of the manuscript are underdeveloped and the findings are over interpreted. Thank you for your efforts and the opportunity to review your work.

  1. Howard Hughes Medical Institute
  2. Wellcome Trust
  3. Max-Planck-Gesellschaft
  4. Knut and Alice Wallenberg Foundation