Modulating task outcome value to mitigate real-world procrastination via noninvasive brain stimulation

Reviewer #1 (Public review):

Summary:

The authors report the results of a tDCS brain stimulation study (verum vs sham stimulation of left DLPFC; between-subjects) in 46 participants, using an intense stimulation protocol over 2 weeks, combined with an experience-sampling approach, plus follow-up measures after 6 months.

Strengths:

The authors are studying a relevant and interesting research question using an intriguing design, following participants quite intensely over time and even at a follow-up time point. The use of an experience-sampling approach is another strength of the work.

Weaknesses:

There are quite a few weaknesses, some related to the actual study and some more strongly related to the reporting about the study in the manuscript. The concerns are listed roughly in the order in which they appear in the manuscript.

(1) In the introduction, the authors present procrastination nearly as if it were the most relevant and problematic issue there is in psychology. Surely, procrastination is a relevant and study-worthy topic, but that is also true if it is presented in more modest (and appropriate) terms. The manuscript mentions that procrastination is a main cause of psychopathology and bodily disease. These claims could possibly be described as 'sensationalized'. Also, the studies to support these claims seem to report associations, not causal mechanisms, as is implied in the manuscript.

(2) It is laudable that the study was pre-registered; however, the cited OSF repository cannot be accessed and therefore, the OSF materials cannot be used to (a) check the preregistration or to (b) fill in the gaps and uncertainties about the exact analyses the authors conducted (this is important because the description of the analyses is insufficiently detailed and it is often unclear how they analyzed the data).

(3) Related to the previous point: I find it impossible to check the analyses with respect to their appropriateness because too little detail and/or explanation is given. Therefore, I find it impossible to evaluate whether the conclusions are valid and warranted.

(4) Why is a medium effect size chosen for the a priori power analysis? Is it reasonable to assume a medium effect size? This should be discussed/motivated. Related: 18 participants for a medium effect size in a between-subjects design strikes me as implausibly low; even for a within-subjects design, it would appear low (but perhaps I am just not fully understanding the details of the power analysis).

(5) It remains somewhat ambiguous whether the sham group had the same number of stimulation sessions as the verum stimulation group; please clarify: Did both groups come in the same number of times into the lab? I.e., were all procedures identical except whether the stimulation was verum or sham?

(6) The TDM analysis and hyperbolic discounting approach were unclear to me; this needs to be described in more detail, otherwise it cannot be evaluated.

(7) Coming back to the point about the statistical analyses not being described in enough detail: One important example of this is the inclusion of random slopes in their mixed-effects model which is unclear. This is highly relevant as omission of random slopes has been repeatedly shown that it can lead to extremely inflated Type 1 errors (e.g., inflating Type 1 errors by a factor of then, e.g., a significant p value of .05 might be obtained when the true p value is .5). Thus, if indeed random slopes have been omitted, then it is possible that significant effects are significant only due to inflated Type 1 error. Without more information about the models, this cannot be ruled out.

(8) Related to the previous point: The authors report, for example, on the first results page, line 420, an F-test as F(1, 269). This means the test has 269 residual degrees of freedom despite a sample size of about 50 participants. This likely suggests that relevant random slopes for this test were omitted, meaning that this statistical test likely suffers from inflated Type 1 error, and the reported p-value < .001 might be severely inflated. If that is the case, each observation was treated as independent instead of accounting for the nestedness of data within participants. The authors should check this carefully for this and all other statistical tests using mixed-effects models.

(9) Many of the statistical procedures seem quite complex and hard to follow. If the results are indeed so robust as they are presented to be, would it make sense to use simpler analysis approaches (perhaps in addition to the complex ones) that are easier for the average reader to understand and comprehend?

(10) As was noted by an earlier reviewer, the paper reports nearly exclusively about the role of the left DLPFC, while there is also work that demonstrates the role of the right DLPFC in self-control. A more balanced presentation of the relevant scientific literature would be desirable.

(11) Active stimulation reduced procrastination, reduced task aversiveness, and increased the outcome value. If I am not mistaken, the authors claim based on these results that the brain stimulation effect operates via self-control, but - unless I missed it - the authors do not have any direct evidence (such as measures or specific task measures) that actually capture self-control. Thus, that self-control is involved seems speculation, but there is no empirical evidence for this; or am I mistaken about this? If that is indeed correct, I think it needs to be made explicit that it is an untested assumption (which might be very plausible, but it is still in the current study not empirically tested) that self-control plays any role in the reported results.

(12) Figures 3F and 3H show that procrastination rates in the active modulation group go to 0 in all participants by sessions 6 and 7. This seems surprising and, to be honest, rather unlikely that there is absolutely no individual variation in this group anymore. In any case, this is quite extraordinary and should be explicitly discussed, if this is indeed correct: What might be the reasons that this is such an extreme pattern? Just a random fluctuation? Are the results robust if these extreme cells are ignored? The authors remove other cells in their design due to unusual patterns, so perhaps the same should be done here, at least as a robustness check.

(13) The supplemental materials, unfortunately, do not give more information, which would be needed to understand the analyses the authors actually conducted. I had hoped I would find the missing information there, but it's not there.

In sum, the reported/cited/discussed literature gives the impression of being incomplete/selectively reported; the analyses are not reported sufficiently transparently/fully to evaluate whether they are appropriate and thus whether the results are trustworthy or not. At least some of the patterns in the results seem highly unlikely (0 procrastination in the verum group in the last 2 observation periods), and the sample size seems very small for a between-subjects design.

Reviewer #2 (Public review):

Summary:

Chen and colleagues conducted a cross-sectional longitudinal study, administering high-definition transcranial direct stimulation targeting the left DLPFC to examine the effect of HD-tDCS on real-world procrastination behavior. They find that seven sessions of active neuromodulation to the left DLPFC elicited greater modulation of procrastination measures (e.g., task-execution willingness, procrastination rates, task aversiveness, outcome value) relative to sham. They report that tDCS effects on task-execution willingness and procrastination are mediated by task outcome value and claim that this neuromodulatory intervention reduces procrastination rates quantified by their task. Although the study addresses an interesting question regarding the role of DLPFC on procrastination, concerns about the validity of the procrastination moderate enthusiasm for the study and limit the interpretability of the mechanism underlying the reported findings.

Strengths:

(1) This is a well-designed protocol with rigorous administration of high-definition transcranial direct current stimulation across multiple sessions. The approach is solid and aims to address an important question regarding the putative role of DLPFC in modulating chronic procrastination behavior.

(2) The quantification of task aversiveness through AUC metrics is a clever approach to account for the temporal dynamics of task aversiveness, which is notoriously difficult to quantify.

Weaknesses:

(1) The lack of specificity surrounding the "real-world measures" of procrastination is problematic and undermines the strength of the evidence surrounding the DLPFC effects on procrastination behavior. It would be helpful to detail what "real-world tasks" individuals reported, which would inform the efficacy of the intervention on procrastination performance across the diversity of tasks. It is also unclear when and how tasks were reported using the ESM procedure. Providing greater detail of these measures overall would enhance the paper's impact.

(2) Additionally, it is unclear whether the reported effects could be due to differential reporting of tasks (e.g., it could be that participants learned across sessions to report more achievable or less aversive task goals, rather than stimulation of DLPFC reducing procrastination per se). It would be helpful to demonstrate whether these self-reported tasks are consistent across sessions and similar in difficulty within each participant, which would strengthen the claims regarding the intervention.

(3) It would be helpful to show evidence that the procrastination measures are valid and consistent, and detail how each of these measures was quantified and differed across sessions and by intervention. For instance, while the AUC metric is an innovative way to quantify the temporal dynamics of task-aversiveness, it was unclear how the timepoints were collected relative to the task deadline. It would be helpful to include greater detail on how these self-reported tasks and deadlines were determined and collected, which would clarify how these procrastination measures were quantified and varied across time.

(4) There are strong claims about the multi-session neuromodulation alleviating chronic procrastination, which should be moderated, given the concerns regarding how procrastination was quantified. It would also be helpful to clarify whether DLPFC stimulation modulates subjective measures of procrastination, or alternatively, whether these effects could be driven by improved working memory or attention to the reported tasks. In general, more work is needed to clarify whether the targeted mechanisms are specific to procrastination and/or to rule out alternative explanations.

Reviewer #3 (Public review):

This manuscript explores whether high-definition transcranial direct current stimulation (HD-tDCS) of the left DLPFC can reduce real-world procrastination, as predicted by the Temporal Decision Model (TDM). The research question is interesting, and the topic - neuromodulation of self-regulatory behavior - is timely.

However, the study also suffers from a limited sample size, and sometimes it was difficult to follow the statistics.

The preregistration and ecological design (ESM) are commendable, but I was not able the find the preregistration, as reported in the paper.

Overall, the paper requires substantial clarification and tightening.

Author response:

Reviewer #1:

(1) We fully thank you to point out the risks of sensationalizing ramification of procrastination on psychopathology, and would rewrite the Introduction section by adding balanced evidence and overall toning down such inappropriate claims meanwhile.

(2) Thank you to raise this crucial question. We are sorry for this fundamental technical issue to preregistration. This occurs from a seriously technical hurdle. The OSF has banned my OSF account, as it claimed to detect “suspicious user’s activities” in my account. This causes no accesses to all materials that already deposited in this OSF account, including this preregistration. We have contacted OSF team, but received no valid technical solution. We reckon that this may be mistaken by my affiliation changes to Third Military Medical University of People’s Liberation Army (PLA). To tackle with this technical issue, we shall upload preregistration in a new repository soon.

(3) This is a back-to-back study to conceptually probe into whether strengthening left DLPFC can mitigate procrastination via reducing task aversiveness or weighting outcome value. Thus, the current study selected a medium effect size in aprior by following the previous one (Xu et al., 2023). This effect size is calculated by the new tool called “Power Contours” (Baker et al., 2021), which weights statistical power by increasing within-subject repeated measures. As you kindly pointed out, we shall clarify effect size calculation in the revised manuscript.

(4) Yes, both groups come in the same number of times into the lab for tDCS stimulation, except to the type (active vs sham).

(5) We shall add full details for clarifying TDM and hyperbolic discounting modeling.

(6) Thank you to raise this very crucial statistical question. We shall double-check whether multiple sessions are modeled as random slopes, and would like to reanalysis it in case which those random slopes are omitted.

(7) Thank you. We have no intentions of confusing you by adding those complicated statistics, but indeed enrich understanding of how we can interpret those findings.

(8) Yes, as mentioned above, we shall add balanced evidence to clarify both left and right DLPFC may function to self-control capability in the Introduction section.

(9) Yes, this is a conceptual hypothesis --- actively stimulating left DLPFC could improve self-control functions. Thank you for this very nuanced but crucial insight, and we could explicitly clarify the nature of our conclusions.

(10) Yes, we ensure that all the participants successfully completed their tasks before deadline at session 6 and 7, and the procrastination rates have been all decreased to 0. Personally speaking, this is somewhat surprise to us as well, but we affirmed this case. For a portion of participants included in the active group, we have received written letters of thanks from them. Thus, this is surprise but exciting finding. Furthermore, thank you for this helpful suggestion, and we would like to do this robustness check by iteratively removing each session, to obviate the statistical biases from an extreme pattern.

(11) Yep, we fully agree with you to add full details in the main text rather in Supplemental materials, and would like to do so in the first round of revision.

Reviewer #2:

(1) Thank you for this very crucial suggestion. We are sorry for this case that much details are omitted to comply with editorial requirement at Nature Human Behaviour (last submission). We do apologize to confuse you as those ambiguous descriptions, and would like to clearly clarify how we measure participants’ procrastination in the real-world tasks. In brief, we asked participant to report a real task that would really happen in the tomorrow and its deadline is also no more than tomorrow. When tomorrow comes, we used ESM to require participant reporting real task completion rate (0-100%) at five time points before the deadline. The five time points are determined by a hyperbolic discounting model (see how and why we set those five time points in the full author’s response letter later). When participant reports the real task completion rate (0-100%) at a given time point, she/he is required to provide a photo to prove its authenticity. The dependent variable --- real-world procrastination rates --- is thus calculated as 100% subtracts the task completion rate (0-100%) when the deadline meets. That is to say, if participant reports task has been fully completed before or when deadline meets, his/her real-world procrastination rate is 100% - 100% = 0%; if reporting task has been completed 60% when deadline meets, the real-world procrastination rate is determined as 100% - 60% = 40%. Do not worry for spurious reporting, we asked all the participants to provide photo verifying the real task completion rate. This is merely a short instance. We shall show the full details in the formal author response letter later.

(2) This is a very meaningful point. We agree with you for this case that participants may learn how to complete this experiment task swiftly rather benefit from neuromodulation. This speculation makes sense, but is compromised by experimental control and empirical observations. Firstly, we do not say “You must complete this task” or “The task completion is associated with bonus/rewards you may get” for participants, which indicates no motivations to do so. Then, the measures to task completion rate are not yet fully based on self-reporting, and we mandate them to provide photos for verification. Thus, this controls the marked risks of spurious reporting. Lastly, all the participants, including ones in either active or sham group, received all the same treatments, excepting “real simulation” and “sham simulation” protocol. Results demonstrated the significant amelioration in the active group rather sham one, indicating no significant “placebo” or “task learning” side effect.

(3) Thank you. As you kindly suggested, we would like to add huge details for those measures in the revised manuscript. While this is a great idea, we did not collect procrastination scores from scales after neuromodulation, and would like to warrant this point into the Limitation section.

(4) Yep, this is a conceptual hypothesis --- actively stimulating left DLPFC could improve self-control functions. We cannot rule out possibilities of amplifying working memory, attention or other cognitive components from this neuromodulation protocol. We fully agree with you for this helpful recommendation --- we would like tone down those claims regarding the roles of DLPFC on self-control, and explicitly warrant that this mechanism may be specialized to the procrastination.

Reviewer #3:

(1) Thank you for taking valuable time to review our manuscript. Yep, limited sample size should warrant cautions to draw a solid conclusion. We would like to claim it into the limitation section. Also, we have streamlined and tightened statistic section by removing complicated and redundancy statistical models.

(2) As mentioned above, we are sorry for this fundamental technical issue to preregistration. This occurs from a seriously technical hurdle. The OSF has banned my OSF account, as it claimed to detect “suspicious user’s activities” in my account. This causes no accesses to all materials that already deposited in this OSF account, including this preregistration. We have contacted OSF team, but received no valid technical solution. We reckon that this may be mistaken by my affiliation changes to Third Military Medical University of People’s Liberation Army (PLA). To tackle with this technical issue, we shall upload preregistration in a new repository soon.

(3) Yep, thank you for this very helpful suggestion. As you kindly indicated, we would like to clarify measures, analyses, methods, and protocols, as well as tighten the whole manuscript.

References

Baker, D. H., Vilidaite, G., Lygo, F. A., Smith, A. K., Flack, T. R., Gouws, A. D., & Andrews, T. J. (2021). Power contours: Optimising sample size and precision in experimental psychology and human neuroscience. Psychological methods, 26(3), 295–314. https://doi.org/10.1037/met0000337

Xu, T., Zhang, S., Zhou, F., & Feng, T. (2023). Stimulation of left dorsolateral prefrontal cortex enhances willingness for task completion by amplifying task outcome value. Journal of experimental psychology. General, 152(4), 1122-1133. https://doi.org/10.1037/xge0001312

Again, we wholeheartedly appreciate all of those very helpful and insightful comments, with each one to contribute substantially for the quality of this manuscript. Notably, those response we presented above are merely provisional and initial. We shall revise our manuscript following those suggestions, one-by-one, along with a full-length response letter.