Continuous perceptual report task.

(A) Experimental setup: Two participants sat in adjacent experimental booths. Subjects played a motion tracking game with a joystick either alone (‘solo’) or together with a partner (‘dyadic’, mixed order: see Supplementary Figure 1A). In dyadic experiments, subjects watched the shared visual stimulus on a screen. Joystick responses of both players as well as visual feedback were mutually visible in real-time. (B) Random dot pattern (‘RDP’) with circular aperture and blacked out central fixation area was continuously presented for intervals of about 1 minute. Subjects were instructed to look at the central fixation cross. Joystick-controlled cursors (‘response arcs’, color-coded) are located at the edge of the fixation area. The stimulus motion signal was predictive of the location of behaviorally relevant reward targets (gray disc). Alignment of the cursor arc with the target resulted in target collection (‘hit’) and reward. The joystick tilt was linked to size of the cursor: less tilt – wide; more tilt - narrow. (C) Example trace of stimulus motion signal and dyadic responses: the stimulus was frequently changing in motion direction and coherence level. Participants tracked the stimulus direction with the joystick to obtain rewards. Darker hues indicate less joystick tilt. Target presentation occurred in pseudorandom time intervals (see Supplementary Figure 1B). The visual and auditory feedback representing the reward score was provided after each target hit. (D) Top: Upon target hit, the reward score was based on joystick tilt and accuracy, individually for each player. Bottom: Each dot corresponds to the accuracy-tilt combination during target presentation. The shaded greyscale background illustrates the non-zero reward map. The non-shaded area denotes missed targets with no reward (reward score = 0). The accuracy and tilt responses of both dyadic players are summarized with the histograms (color-coded). Positive trend of reward scores, shown in Supplementary Figure 1C, indicates perceptual learning over time.

Solo behavior during the continuous perceptual report.

(A) Coherence-dependent modulation of hit rates (left), accuracy (center, shown with absolute, target-aligned angular joystick error as dashed blue line) and state-aligned radial joystick tilt (right). Gray lines show averages for individual participants. Red shading illustrates the 99% confidence intervals of the mean across participants. Bold, black lines show the mean across the population. Data were first averaged within-subject, before pooling coherence conditions across-subjects. (B) Estimation of the response lag after a stimulus direction change with a cross-correlation between stimulus and joystick signal for an example subject (dashed lines). Low coherence levels resulted in a breakdown of response reliability, indicated by the low cross-correlation coefficients (see also Supplementary Figure 1D). Here and in other panels, stimulus coherence is color-coded. (C) Top: average population response lag. Black data points show individual data; white dot displays population median. Bottom: variability of the population lags, displayed by the standard deviation. (D) Average time course of joystick accuracy (left) and tilt (right) during a stable stimulus period aligned to the next stimulus direction change at time point 0, across all subjects. Here, the first 500 ms of each stimulus state as well as all samples after the first target appearance were excluded before averaging. The shaded background illustrates the 95% confidence interval of the mean. We calculated the slopes of the average time course and tested if they were significantly different from zero across subjects (n = 38, Bonferroni-corrected). Statistics is illustrated with color-coded triangles, indicating which coherence condition has a significant slope and in which direction (triangles point up for positive slopes). (E) Cross-correlation between the mean-detrended time course of joystick tilt and accuracy, indicating that changes in tilt follow changes in accuracy within half a second. Peak cross-correlation coefficients are marked with a dashed line. Statistics is illustrated with the color-coded triangles, indicating which coherence condition has a significantly shifted from 0 cross-correlation peak across subjects (n = 38, Bonferroni-corrected).

Social modulation: dyadic vs solo.

(A) Reward score in dyadic vs solo sessions. All solo and (human-human) dyadic sessions were pooled within-subject. Inset: coherence-wise averaging of reward scores. Here and in other panels, stimulus coherence is color-coded. Each subject contributes one data point per stimulus coherence level. The median score across all subjects for each coherence condition is overlaid in brighter color hues. Error bars show 99% confidence intervals of the median in solo and dyadic conditions. (B) Social modulation between solo and dyadic experiments, measured as AUC, for state-aligned accuracy (top) and confidence (i.e., tilt, bottom, Wilcoxon rank sum test, Bonferroni-corrected) of individual participants. Coherence was pooled within-subjects. A value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions, 1 and 0 imply perfect separation. See Supplementary Figure 3A for average performance in dyadic experiments and Supplementary Figure 3C for examples of social modulation and how the AUC captures the its directionality. (C) Performance-dependent social modulation. Schematic: social modulation can increase performance (values above the horizontal dashed line, green) or decrease it (values below the horizontal dashed line, red). First column: statistical comparison between joystick accuracy and confidence in solo and dyadic experiments, for each coherence. Sessions were pooled according to experimental condition within-subject. The percentage of participants with significantly different distribution in solo and dyadic sessions is displayed (Wilcoxon rank sum test, Bonferroni-corrected). The directionality of the significant effect in each subject was established with the AUC. Average AUC per coherence level is shown in gray with 99% confidence intervals (shaded background). Second column: average social modulation displayed for different solo performance quartiles, separately for each corresponding performance measure (hit rate, accuracy, confidence). See Supplementary Figure 3B for comparison of raw solo joystick responses with social modulation, and Supplementary Figure 4 for quartile grouping across response dimensions.

Social modulation in human-human dyads.

Left column: schematic depiction of hypothesized effects; middle and right column: actual data for state-aligned confidence and accuracy. (A) Absolute difference between partners in solo and dyadic setting for confidence (middle) and accuracy (right). Each dyad is represented by one data point. Dyads show convergence when differences between players in dyadic setting are smaller than differences in solo experiments (see left schematic). (B) Social modulation difference between players (better in solo minus worse in solo) as a function of better minus worse inter-player performance difference, for confidence (middle) and accuracy (right). Each dyad is represented by one data point. The solid line illustrates the correlation between solo difference and social modulation difference (Linear regression, n = 50, Confidence: r = −0.535, p < 0.0001; Accuracy: r = −0.26, p = 0.068; Converging dyads only (n = 40): Confidence: r = −0.483, p < 0.01; Accuracy: r = −0.474, p < 0.01). (C) Social modulation displayed separately for better and worse solo players. Dyads are connected with a colored line (blue: convergence; red: divergence). Histograms show the overall distribution of social modulations across all participants. Means of the distributions are illustrated with a colored line.

Comparison of social modulation in human-human (HH) dyads and human-computer (HC) dyads.

(A) Left: average subject-wise score in the two dyadic experiments compared to the score of the same participant in solo experiments. Average score (middle) and hit rate (right) of the population in different experimental conditions. Error bars correspond to the 99% confidence intervals of the mean. (B) Comparison of average hit rate (top), accuracy (middle) and confidence (bottom) for each participant and each stimulus coherence (color-coded) in human-human and human-computer dyads. Individual data (subject-wise, averaged across several HH sessions for each subject) are shown in darker hue. Medians are overlaid for each coherence condition with bright colors. Error bars show 99% confidence intervals of the median. (C) Statistical comparison of accuracy and confidence in HH and HC dyadic experiments, for each coherence condition. Sessions were pooled according to experimental condition within-subject. The percentage of participants with significantly different distribution in HH vs HC dyadic sessions is displayed (Wilcoxon rank sum test, Bonferroni-corrected). The directionality of the significant effect in each subject was established with the AUC. Average AUC per coherence level is shown in gray with 99% confidence intervals (shaded background). For hit rates, the average difference is displayed.

Relationship between social modulation of accuracy and confidence in each participant.

Each dot represents one player in a dyad. Left: Social modulation in human-human (HH) dyadic condition vs. solo (n = 100); Middle: Human-Computer (HC) dyadic condition vs solo (n = 33); Right: Correlation of confidence vs accuracy difference between two dyadic conditions (HC vs HH, n = 98). Values above 0.5 on each axis correspond to positive social modulation: increased accuracy or confidence in the “first condition” (e.g. HH) compared to the “second condition” (e.g. solo), and vice versa.

Related to Figure 1 and Figure 2. Additional information regarding experiments and joystick responses of individual participants.

(A) Number and identity of experimental sessions for each subject. A session comprised two experimental blocks that were recorded in different setups. The order of the session type (solo or dyadic) was mixed (color-coded). All participants, except two, contributed data to each session type, specifically, solo CPR as well as both dyadic conditions. (B) Statistics of target occurrences during stimulus presentation. Distributions of inter-target intervals and target count per stimulus state for an example session. Targets were flashed with a 1% probability every 10 ms. Once a target was presented it remained in the screen for 50 ms followed by a minimum inter-target interval of 300 ms. (C) Final reward scores of participants over the course of the individuals’ data acquisition period (gray lines). Reward score increased over time. A linear regression was fitted to the cumulative scores of each experimental block for each participant (black lines). Note that each experimental session comprised two blocks, one in each setup. Independent of the session type (see panel (A) for more details), scores increased over time, likely due to perceptual learning. The final cumulative score is comprised of the hit rate, accuracy and tilt, all of which are affected by number of the experimental block, with later sessions resulting in higher hit rate as well as more accurate and tilted responses (Supplementary Table 1 - Supplementary Table 4). (D) Normalized cross-correlation coefficients between random dot motion direction and joystick response direction illustrated for each subject. Lighter hues indicate higher cross-correlation coefficients at the respective signal lag, darker hues suggest low correlation between stimulus and joystick response at that lag. Cross-correlations broke down consistently with lower stimulus coherence.

Related to Figure 2. Joystick tilt is a proxy measure of perceptual confidence.

(A) Metacognitive sensitivity of joystick response for one example subject. Left: Distribution of joystick accuracy in stimulus states with low (gray) vs high joystick tilt (colored, median split). Accuracy and tilt were averaged for the last 30 frames (250 ms) prior to a stimulus direction change. Coherence is color-coded. Right: Corresponding receiver-operating characteristics (‘ROC’) between the two distributions for each coherence level (color-coded). A ROC curve along the diagonal would indicate similar accuracy distributions between hits and misses, suggesting no metacognitive sensitivity. (B) Population AUC values (black dots) are consistently above 0.5 (p<0.001, Two-sided Wilcoxon signed rank test for distribution with median 0.5), demonstrating that high tilt was more often associated with high accuracy, suggesting metacognitive-sensitive confidence readouts.

Related to Figure 3. Additional information for human-human dyadic performance.

(A) Performance summary in (human-human) dyadic experiments. Hit rate (left), accuracy (center) and confidence (right) illustrated for each subject contributing human-human dyadic data. Same conventions as in Figure 2A. (B) Comparison of social modulation with response measures of solo experiments. Hit rate differences (left, compared to solo hit rate) and social modulation of accuracy (center, compared to solo accuracy) and tilt (right, compared to solo confidence) are displayed for the entire population. Joystick data was recorded in a normalized fashion for both accuracy (180deg difference = 0; Perfect match of stimulus direction = 1) and confidence (center = 0; Max. joystick tilt = 1). Each participant contributes on data point per coherence condition (color-coded). The median score across all subjects is overlaid for each coherence condition in brighter color hues. Error bars show 99% confidence intervals of the median in solo and dyadic conditions. (C) Social modulation of confidence for three example participants. Data for solo (light, left) and human-human dyadic experiments (dark, right) are displayed for each stimulus coherence level (color-coded). Each dot corresponds to the time-window average for a single stimulus state. All sessions of the same experimental condition are pooled. Corresponding AUC values, used to quantify the direction and magnitude of social modulation between dyadic and solo experiments, are shown below. A value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions, 1 and 0 imply perfect separation between experimental conditions.

Related to Figure 3. Social modulation grouped by different metrics.

Illustration of average social modulation for hit rates (first row), accuracy (second row) and confidence (third row) based on differently grouped solo performance quartiles: first column - hit rate quartiles, second column - accuracy quartiles, third column - confidence quartiles. Coherence is color-coded. Same conventions as in Figure 3C. Social modulation between dyadic and solo experiments was measured by AUC. An AUC value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions. AUC > 0.5 imply better accuracy or higher confidence in dyadic experiments. AUC < 0.5 imply better accuracy or higher confidence in solo experiments.

Related to Figure 4. Across-player correlation in solo and dyadic setting.

Confidence (left) and accuracy (right) of dyadic partners in solo and dyadic settings. Only in the dyadic condition confidence (but not accuracy) correlate significantly between players.

Related to Figure 4. Relationship between social modulation and solo performance difference.

Average (across dyad members) social modulation for confidence (top) and accuracy (bottom) displayed as a function of the absolute solo differences in confidence (left) and accuracy (right) between dyadic partners. Each data point corresponds to one dyad (N=50). Average social modulation did not correlate with absolute solo difference between players.

Related to Figure 5. Additional information regarding computer player performance.

(A) Illustration of computer player behavior in an example session. Each dot corresponds to the accuracy-confidence combination during target presentation. Distribution of computer behavior is summarized with histograms. Same conventions as in Figure 1D. Coherence is color-coded. (B) Cross-correlation between stimulus direction changes and cursor responses of computer player. Similar human-like response lag was built in for all coherence conditions (color-coded). See Figure 2C for comparison. (C) Average computer player performance (reward score, hit rate, accuracy, and confidence) as a function of stimulus coherence. Shaded background corresponds to the 99% confidence intervals of the median. See Figure 2A for comparison to human behavior.

Related to Figure 5. Social modulation in human-computer (HC) dyads vs solo.

(A) Comparison of average hit rate (top), accuracy (center) and confidence (bottom) for each stimulus coherence in solo and human-computer dyads, color-coded for coherence level. Individual data is shown in darker hues. Each subject contributes one data point per coherence condition. Medians across subjects overlaid for each coherence condition (bright color). Error bars show 99% confidence intervals of the median. Same conventions as in Figure 5. (B) Social modulation of humans between dyadic (human-computer, HC) and solo experiments. (C) Population comparison of social modulation (between solo and HH experiments, top) and HH and HC contrast (bottom).

Random effects structure and sample size of each model

Results of the full model with hit probability as response

Results of the full model with confidence being the response

Results of the full model with accuracy being the response