The Importance of Individual Beliefs in Assessing Treatment Efficacy: Insights from Neurostimulation Studies

Reviewer #1 (Public Review):

Randomized clinical trials use experimental blinding and compare active and placebo conditions in their analyses. In this study, Fassi and colleagues explore how individual differences in subjective treatment (i.e., did the participant think they received the active or placebo treatment) influence symptoms and how this is related to objective treatment. The authors address this highly relevant and interesting question using a powerful method by (re-)analyzing data from four published neurostimulation studies and including subjective treatment in statistical models explaining treatment response. The major strengths include the innovative and important research question, the inclusion of four different studies with different techniques and populations to address this question, sound statistical analyses, and findings that are of high interest and relevance to the field.

My main suggestion is that authors reconsider the description of the main conclusion to better integrate and balance all findings. Specifically, the authors conclude that (e.g., in the abstract) "individual differences in subjective treatment can explain variability in outcomes better than the actual treatment", which I believe is not a consistent conclusion across all four studies as it does not appropriately consider important interactions with objective treatment observed in study 2 and 3. In study 2, the greatest improvement was observed in the group that received TMS but believed they received sham. While subjective treatment was associated with improvement regardless of objective active or sham treatment, improvement in the objective active TMS group who believed they received sham suggests the importance of objective treatment regardless of subjective treatment. In Study 3, including objective treatment in the model predicted more treatment variance, further suggesting the predictive value of objective treatment. In addition to updating the conclusions to better reflect this interaction, I suggest authors include the proportion of participants in each subjective treatment group that actually received active or sham treatment to better understand how much of the subjective treatment is explained by objective treatment. I think it is particularly important to better integrate and more precisely communicate this finding, because the conclusions may otherwise be erroneously interpreted as improvements after treatment only being an effect of subjective treatment or sham.

The paper will have significant impact on the field. It will promote further investigation of the effects of sham vs active treatment by the introduction of the terms subjective treatment vs objective treatment and subjective dosage that can be used consistently in the future. The suggestions to assess the expectation of sham vs active earlier on in clinical trials will advance the understanding of subjective treatment in future studies. Overall, I believe the data will substantially contribute to the design and interpretation of future clinical trials by underscoring the importance of subjective treatment.

Reviewer #2 (Public Review):

This manuscript focuses on the clinical impact of subjective experience or treatment with transcranial magnetic stimulation and transcranial direct current stimulation studies with retrospective analyses of 4 datasets. Subjective experience or treatment refers to the patient level thought of receiving active or sham treatments. The analyses suggest that subjective treatment effects are an important and under appreciated factor in randomized controlled trials. The authors present compelling evidence that has significance in the context of other modalities of treatment, treatment for other diseases, and plans for future randomized controlled trials. Other strengths included a rigorous approach and analyses. Some aspects of the manuscript are underdeveloped and the findings are over interpreted. Thank you for your efforts and the opportunity to review your work.