Confidence over competence: Real-time integration of social information in human continuous perceptual decision-making

Reviewer #1 (Public review):

Summary:

This paper reports an interesting and clever task that allows the joint measurement of both perceptual judgments and confidence (or subjective motion strength) in real/continuous time. The task is used together with a social condition to identify the (incidental, task-irrelevant) impact of another player on decision-making and confidence.

Strengths:

The innovation on the task alone is likely to be impactful for the field, extending recent continuous report (CPR) tasks to examine other aspects of perceptual decision-making and allowing more naturalistic readouts. One interesting and novel finding is the observation of dyadic convergence of confidence estimates even when the partner is incidental to the task performance, and that dyads tend to be more risk-seeking (indicating greater confidence) than when playing solo. The paper is well-written and clear.

Weaknesses:

(1) One concern with the novel task is whether confidence is disambiguated from a tracking of stimulus strength or coherence. The subjects' task is to track motion direction and use the eccentricity of the joystick to control the arc of a catcher - thus implementing a real-time sensitivity to risk (peri-decision wagering). The variable-width catcher has been used to good effect in other confidence/uncertainty tasks involving learning the spread of targets (the Nassar papers). But in the context of an RDK task, one simple strategy here is to map eccentricity directly to (subjective) motion coherence - such that the joystick position at any moment in time is a vector with motion direction and strength. This would still be an interesting task - but could be solved without invoking metacognition or the need to estimate confidence in one's motion direction decision (the analyses in Supplementary Figure 2 are nice in showing a dissociation from (objective) coherence, such that even within a coherence level, changes in eccentricity scale with direction precision - but this does not get around the potential conflation of confidence with fluctuations in motion energy).

In other words, in this deflationary framing, what the subjects might be doing is tracking two features of the world - motion strength and direction. This possibility needs to be ruled out if the authors want to claim a mapping between eccentricity and decision confidence (for instance, an ideal observer model of the task that set eccentricity proportional to instantaneous motion strength presumably would also sensibly accrue reward targets, without the need to compute confidence in the direction response). This would be straightforward to simulate and would establish a baseline model against which to compare claims about confidence (eg when evaluating additional social modulations). More generally it casts doubt on claims such as the one on line 210 that eccentricity was "chosen freely via metacognitive assessment of the current perceptual process, [and] can be treated as a proxy measure of subjective perceptual confidence."

One route to doing this would be to ask whether the eccentricity reports show statistical signatures of confidence that have been established for more classical punctate tasks. Here a key move has been to identify qualitative patterns in the frame of reference of choice accuracy - with confidence scaling positively with stimulus strength for correct decisions, and negatively with stimulus strength for incorrect decisions (the so-called X-pattern, for instance Sanders et al. 2016 Neuron https://pubmed.ncbi.nlm.nih.gov/27151640/).

(2) I was surprised not to see more analysis of the continuous report data as a function of (lagged) task variables. Some of this analysis is shown in Figure 2b relative to an (objective) direction change, and also in the cross-correlation plots in Supplementary Figure 1d. But to fully characterise the task behaviour it also seems important to ask how and whether fluctuations in motion energy (assuming that the RDK frames were recorded) during a steady state phase are affecting continuous reporting of direction and eccentricity, prior to asking how social information is incorporated into subjects' behaviour.

Minor points:

(1) Lines 295-298, isn't it guaranteed to observe these three behavioural patterns (both participants improving, both getting worse, only one improving while the other gets worse) even in random data?

(2) Lines 703-707, it wasn't clear what the AUC values referred to here (also in Figure 3) - what are the distributions that are being compared? I think part of the confusion here comes from AUC being mentioned earlier in the paper as a measure of metacognitive sensitivity (correct vs. incorrect trial distributions), whereas my impression here is that here AUC is being used to investigate differences in variables (eg confidence) between experimental conditions.

(3) Could the findings of the worse solo player benefitting more than the better solo player (Figure 4c) be partly due to a compressive ceiling effect - eg there is less room to move up the psychometric function for the higher-scoring player?

Reviewer #2 (Public review):

Summary:

Schneider et al examine perceptual decision-making in a continuous task setup when social information is also provided to another human (or algorithmic) partner. The authors track behaviour in a visual motion discrimination task and report accuracy, hit rate, wager, and reaction times, demonstrating that choice wager is affected by social information from the partner.

Strengths:

There are many things to like about this paper. The visual psychophysics has been undertaken with much expertise and care to detail. The reporting is meticulous and the coverage of the recent previous literature is reasonable. The research question is novel.

Weaknesses:

The paper is difficult to read. It is very densely written, with little to distinguish between what is a key message and what is an auxiliary side note. The Figures are often packed with sometimes over 10 panels and very long captions that stick to the descriptive details but avoid clarity. There is much that could be shifted to supplementary material for the reader to get to the main points.

Example: In lines 176-181, we read about reaction times in the motion task with a level of detail and repetition that has very little relevance to the message of the paper. When we get to social condition and we read about RT in lines 239-243, it is not quite clear what it is that we should take away from this.

Another example: the word "eccentricity" is used to refer to "deviation from central position" as a measure of wager. But we see in Figure 1 that it actually refers to the width of the ARC straddling the reported direction of motion. The confusion is compounded when we see in Figure 2b that the two subjects' different levels of confidence are (short red and long green) arcs at the SAME Eccentricity and overlap one another. The use of the word eccentricity is clearly driven by the Joystick action description and is in direct conflict with the meaning of what eccentricity is in visual perception.

A third and very important one is what the word "dyadic" refers to in the paper. The subjects do not make any joint decisions. However, the authors calculate some "dyadic score" to measure if the group has been able to do better than individuals. So the word dyadic sometimes refers to some "nominal" group. In other places, dyadic refers to the social experimental condition. For example, we see in Figure 3c that AUC is compared for solo vs dyadic conditions. This is confusing.

A key problem with the paper is that it introduces many terms and the main text often overlooks defining them clearly. I still do not understand the difference between Accuracy and Hit in the paper's jargon. The same goes for "score". Please note that the answer "this is defined in the supplementary method" is not acceptable. These are key constructs in the paper. The flow of the paper's main text depends on them.

Author response:

We sincerely thank you for your constructive and insightful feedback on our manuscript, including the assessment of its strengths and suggestions for improvements. This will allow us to enhance the clarity and impact of our work. In our revised manuscript, we will address your recommendations as follows:

(1) Disambiguating whether the joystick eccentricity reflects the subject’s confidence or simply the perceived stimulus strength or coherence

We agree that this is a pivotal issue for the interpretation of our results. We are confident that the joystick “eccentricity” (i.e., radial joystick deviation from the center) does not simply correlate with the moment-to-moment fluctuations of stimulus coherence. The observations that the radial joystick response varied considerably more than the stimulus fluctuations within each subject and each coherence level, and the analysis of metacognitive sensitivity, suggest that subjects indeed incorporated confidence judgements into their continuous reports. As proposed, we will further explore the established signatures of metacognitive confidence reports, and we will quantify the motion energy fluctuations within time intervals where the nominal stimulus parameters remained constant, to examine whether accuracy and confidence levels vary in response to these fluctuations. This approach will provide deeper insights into continuous dynamics within our paradigm.

(2) Rationale for Social Investigation

We will clarify the rationale and methodology of the social aspects in our experiments to better contextualize our approach and findings and their relationship to the field of collective decision-making. In particular, we will further emphasize that while our paradigm indeed did not impose integrating the information from the partner and did not involve incentives for collectively solving the task, the participants could (and did) incorporate the social information into their judgements and mostly improved their earnings. In this way, our approach complements the studies that required joint decisions.

(3) Streamlining and Terminology

We will streamline the text and figure legends to present our main arguments more concisely and improve the overall flow of the manuscript. Additionally, we will include a glossary to the main text to clarify terminology, enhancing accessibility and ensuring consistent understanding of key terms throughout the paper.

To clarify two of the points upfront, we indeed used the term “eccentricity” not in a visual science sense but as the measure of radial joystick deviation from the center and the corresponding angular width of the response arc; we now realize that this is confusing in the context of visual psychophysics paper and will use another word. The term “dyadic” was meant to describe the experimental condition when two participants worked on the task, and associated measures of performance in this condition. The “dyadic score”, defined as the average score across the two participants in the dyadic condition, will be renamed as “combined score”.

(4) Incorporation of Additional Literature

We acknowledge and appreciate the recommendations for additional relevant literature, which we will incorporate into our discussion. This will allow us to contextualize our findings more thoroughly within the existing body of research and highlight the broader implications of our work.