Continuous perceptual report task and behavioral response profile in solo experiments.

(a) Experimental setup: Two participants in were seated in adjacent experimental booths in the same room. Subjects played a motion tracking game with a joystick either alone (‘solo’) or together with a partner (‘dyadic’, mixed order: see Supplementary Figure 1a). In dyadic experiments, subjects watched the same visual stimulus on a screen and their joystick responses as well as feedback were mutually visible in real-time. (b) Stimulus layout in dyadic condition: Random dot pattern (‘RDP’) with circular aperture and blacked out central fixation area was continuously presented for intervals of about 1 minute. Subjects were instructed to look at the central fixation cross. Joystick-controlled cursors (‘response arcs’, color-coded) are located at the edge of the fixation area. The stimulus motion signal was predictive of the location of behaviorally relevant reward targets (gray disc). Alignment of the cursor arc with the target resulted in target collection (‘hit’) and reward. The joystick eccentricity was linked to size of the cursor: Central position – wide; Eccentric position - narrow. (c) Example trace of stimulus motion signal and dyadic responses: the stimulus was frequently changing in motion direction and coherence level. Participants tracked the stimulus direction with the joystick to obtain rewards when collecting reward targets. Joystick eccentricity is illustrated with hue of colored example trace. Darker hues indicate a more central joystick position. Target presentation occurred in pseudorandom time intervals (see Supplementary Figure 1b). The visual and auditory feedback representing the reward score was provided after each target hit. (d) Reward contingencies of the CPR game. Top: Upon target hit, the reward score was calculated based on joystick eccentricity and accuracy, individually for each player. Bottom: Joystick responses at the time of target presentation for an example dyad. Each dot corresponds to the accuracy-eccentricity combination during target presentation. The shaded greyscale background illustrates the non-zero reward map. The non-shaded, white area denotes missed targets with no reward (score = 0). The accuracy and eccentricity responses of both dyadic players are summarized with the histograms (color-coded). Positive trend of reward scores, shown in Supplementary Figure 1c, indicates perceptual learning over time.

Solo behavior during the continuous perceptual report.

(a) Coherence-dependent modulation of hit rates (left), accuracy (center) and eccentricity (right). Gray lines show averages for individual participants. Red shading illustrates the 99% confidence intervals of the mean across participants. Bold, black lines show the mean across the population. Data was first averaged within-subject, before pooling coherence conditions across-subjects. (b) Estimation of the response lag after a stimulus direction change with a cross-correlation between stimulus and joystick signal for an example subject. The lag of the maximum cross-correlation coefficient was set to be the response lag. Low coherence levels resulted in a breakdown of response reliability, indicated by the low cross-correlation coefficients (see also Supplementary Figure 1d). (c) Top: average population response lag displayed with a violin plot. Black data points show average data of individuals. White dot displays population median. Bottom: variability of the population lags, displayed by the standard deviation. Response lags and response lag reliability after stimulus change depended on stimulus coherence (color-coded).

Dyadic vs solo social modulation.

(a) Reward score in dyadic vs solo sessions. Scores were averaged across all targets, including misses. All solo and (human-human) dyadic sessions were pooled within-subject. Inset: coherence-wise averaging of reward scores. Stimulus coherence is color-coded, see legend on the right in panel c. Each subject contributes one data point per stimulus coherence level. The median score across all subjects for each coherence condition is overlaid in brighter color hues. Error bars show 99% confidence intervals of the median in solo and dyadic conditions. (b) Social modulation between solo and dyadic experiments, measured as AUC, for joystick accuracy (top) and eccentricity (bottom, Wilcoxon rank sum test, Bonferroni-corrected) of individual participants. Coherence was pooled within-subjects. See Supplementary Figure 3a for average performance in dyadic experiments and Supplementary Figure 3c for three examples of social modulation and how the AUC captures the directionality of the behavioral change. (c) Coherence dependent modulation of hit rate (top row), accuracy (center row) and eccentricity (bottom row) in dyadic vs solo setting. First column: quantification of dyadic vs solo change in behavior for each participant and coherence condition. All dyadic and solo sessions were pooled, respectively. AUC was used to quantify the direction and magnitude of social modulation. A value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions, 1 and 0 imply perfect separation between experimental conditions (see Supplementary Figure 3c). Gray lines correspond to the AUC of individual participants. Red shading illustrates the 99% confidence intervals of the mean across participants. Bold, black lines show the mean across the population. Data were averaged within-subject first, before pooling across coherence conditions. Second column: average social modulation displayed for different solo performance quartiles. Coherence is color-coded. The grouping into quartiles was done for each response corresponding dimension separately. Please see Supplementary Figure 3b for comparison of raw solo joystick responses with social modulation. See Supplementary Figure 4 for quartile grouping across response dimensions. Third column: statistical comparison between joystick accuracy and eccentricity in solo and dyadic experiments, for each coherence condition. Sessions were pooled according to experimental condition within-subject. The percentage of participants with significantly different distribution in solo and dyadic sessions is displayed (Wilcoxon rank sum test, Bonferroni- corrected). The directionality of the significant effect in each subject was established with the AUC shown in the first column.

Social modulation in human-human dyads.

Left column: schematic depiction of hypothesized effects; middle and right column: actual data for eccentricity (confidence) and accuracy. (a) Left: social modulation might increase monotonically with solo performance difference (‘bow-tie’). Alternatively, similar performance has been shown to result in dyadic improvements (‘Gaussian’). Middle and right: within-dyad social modulation (dyadic vs solo), measured by AUC, as a function of the within-dyad solo difference. Dyads consist of two participants: player 1 (P1, filled circle) and player 2 (P2, open circle), which are connected with a gray line. Histograms summarize social modulation across all participants. Red lines illustrate the median of the distribution. (b) Absolute social modulation difference between dyadic players, corresponding to the length of the line connecting player 1 and player 2 in (a). Each dyad is represented by one data point. We expected a U-shaped function if the social modulation differences would be larger in more heterogeneous dyads. The data was fitted with a 2nd order polynomial function (red). A moving average (window size = 12, black) confirmed the fits. (c) Signed social modulation difference between players for eccentricity (middle) and accuracy (right). Each dyad is represented by one data point. The red line illustrates the correlation between solo difference and social modulation (Linear regression, Accuracy: r = −0.6, p < 0.01; Eccentricity: r = −0.7, p < 0.05; P-values significantly different from randomly permutated data). We hypothesized a negative correlation if the worse solo player had a larger AUC value than the dyadic partner (left). (d) Absolute difference between partners in solo and dyadic setting for eccentricity (middle) and accuracy (right). Each dyad is represented by one data point. Dyads show convergence when differences between players in dyadic setting are smaller than differences in solo experiments (left).

Comparison of social modulation in human-human (HH) dyads and human-computer (HC) dyads.

(a) Left: average subject-wise score in the two dyadic experiments compared to the score of the same participant in solo experiments. Average score (middle) and hit rate (right) of the population in different experimental conditions. Error bars correspond to the 99% confidence intervals of the mean. (b) Comparison of average hit rate (top), accuracy (middle) and eccentricity (bottom) for each participant and each stimulus coherence (color-coded) in human-human and human-computer dyads. Individual data (subject-wise, averaged across several HH sessions for each subject) are shown in darker hue. Medians are overlaid for each coherence condition with bright colors. Error bars show 99% confidence intervals of the median. (c) Response difference between HH and HC dyadic experiments measured by AUC. All HH and HC sessions were pooled, respectively. A value of 0.5 corresponds to perfect overlap between HH and HC dyadic response distributions, 1 and 0 imply perfect separation between experimental conditions (see Supplementary Figure 3c). Gray lines correspond to the AUC of individual participants. Bold black lines and red shading illustrate the mean and 99% confidence intervals of the mean across participants. Data were averaged within-subject first, before pooling across coherence conditions. (d) Population comparison of social modulation (between solo and HH experiments, top) and HH and HC contrast (bottom).

Relationship between social modulation of accuracy and eccentricity.

Left: Human-Human (HH) dyads vs. solo; Middle: Human-Computer (HC) dyads vs solo; Right: Correlation of eccentricity vs accuracy difference between dyadic experiments (HC vs HH). Values above 0.5 correspond to increased accuracy or confidence in the “first condition” (e.g. HH dyad) compared to the “second condition” (e.g. Solo), and vice versa.

Additional information regarding experiments and joystick responses of individual participants.

(a) Number and identity of experimental sessions for each subject. A session comprised two experimental blocks that were recorded in different setups. The order of the session type (solo or dyadic) was mixed (color-coded). All participants, except two, contributed data to each session type, specifically, solo CPR as well as both dyadic conditions. (b) Statistics of target occurrences during stimulus presentation. Distributions of inter-target intervals and target count per stimulus state for an example session. Targets were flashed with a 1% probability every 10 ms. Once a target was presented it remained in the screen for 50 ms followed by a minimum inter-target interval of 300 ms. (c) Final reward scores of participants over the course of the individuals’ data acquisition period (gray lines). Reward score increased over time. A linear regression was fitted to the cumulative scores of each experimental block for each participant (black lines). Note that each experimental session comprised two blocks, one in each setup. Independent of the session type (see panel (a) for more details), scores increased over time, likely due to perceptual learning. The final cumulative score is comprised of the hit rate, accuracy and eccentricity, all of which are affected by number of the experimental block, with later sessions resulting in higher hit rate as well as more accurate and eccentric responses (Supplementary Table 1 - Supplementary Table 4). (d) Normalized cross-correlation coefficients between random dot motion direction and joystick response direction illustrated for each subject. Lighter hues indicate higher cross-correlation coefficients at the respective signal lag, darker hues suggest low correlation between stimulus and joystick response at that lag. Cross-correlations broke down consistently with lower stimulus coherence.

Joystick eccentricity is a proxy measure of perceptual confidence.

(a) Metacognitive sensitivity of joystick response for one example subject. Left: Distribution of joystick accuracy for low (gray) vs high eccentricity stimulus states (colored, median split). Accuracy and eccentricity were averaged for the last 30 frames (250 ms) prior to a stimulus direction change. Coherence is color-coded. Right: Corresponding receiver-operating characteristics (‘ROC’) between the two distributions for each coherence level (color-coded). A ROC curve along the diagonal would indicate similar accuracy distributions between hits and misses, suggesting no metacognitive sensitivity. (b) Population AUC values (black dots) are consistently above 0.5 (p<0.001, Two-sided Wilcoxon signed rank test for distribution with median 0.5), demonstrating that high eccentricity was more often associated with high accuracy, suggesting metacognitive-sensitive confidence readouts.

Additional information for human-human dyadic performance.

(a) Performance summary in (human-human) dyadic experiments. Hit rate (left), accuracy (center) and eccentricity (right) illustrated for each subject contributing human-human dyadic data. Same conventions as in Figure 2a. (b) Comparison of social modulation with response measures of solo experiments. Hit rate differences (left, compared to solo hit rate) and social modulation of accuracy (center, compared to solo accuracy) and eccentricity (right, compared to solo eccentricity) are displayed for the entire population. Joystick data was recorded in a normalized fashion for both accuracy (180deg difference = 0; Perfect match of stimulus direction = 1) and eccentricity (center = 0; Max. eccentricity = 1). Each participant contributes on data point per coherence condition (color-coded). The median score across all subjects is overlaid for each coherence condition in brighter color hues. Error bars show 99% confidence intervals of the median in solo and dyadic conditions. (c) Social modulation of the eccentricity responses for three example participants. Data for solo (light, left) and human-human dyadic experiments (dark, right) are displayed for each stimulus coherence level (color-coded). Each dot corresponds to the time-window average for a single stimulus state. All sessions of the same experimental condition are pooled. Corresponding AUC values, used to quantify the direction and magnitude of social modulation between dyadic and solo experiments, are shown below. A value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions, 1 and 0 imply perfect separation between experimental conditions.

Social modulation grouped by different metrics.

Illustration of average social modulation for hit rates (first row), accuracy (second row) and eccentricity (third row) based on differently grouped solo performance quartiles: first column - hit rate quartiles, second column - accuracy quartiles, third column - eccentricity quartiles. Coherence is color-coded. Same conventions as in Figure 3c. Social modulation between dyadic and solo experiments was measured by AUC. An AUC value of 0.5 corresponds to perfect overlap between solo and dyadic response distributions. AUC > 0.5 imply better accuracy or higher eccentricity in dyadic experiments. AUC < 0.5 imply better accuracy or higher eccentricity in solo experiments.

Additional information regarding the relationship between dyadic and solo behavior.

(a) Average (across dyad members) social modulation for eccentricity (top) and accuracy (bottom) displayed as a function of the absolute solo differences in eccentricity (left) and accuracy (right) between dyadic partners. Each data point corresponds to one dyad. Average social modulation did not correlate with absolute solo difference between players. (b) Dyadic performance, defined as average (across dyad members) score (top) and average hit rate (bottom), displayed as a function of the absolute solo differences in eccentricity (left) and accuracy (right) between dyadic partners. Dyadic performance did not correlate with absolute solo difference between players. (c) Correlations between the accuracy (left) and eccentricity (right) of dyadic partners in solo and dyadic setting. Only in dyadic situations accuracy and eccentricity correlate between players.

Additional information regarding computer player performance.

(a) Illustration of computer player behavior in an example session. Each dot corresponds to the accuracy-eccentricity combination during target presentation. Distribution of computer behavior is summarized with histograms. Same conventions as in Figure 1d. Coherence is color-coded. (b) Cross-correlation between stimulus direction changes and cursor responses of computer player. Similar human-like response lag was built in for all coherence conditions (color-coded). See Figure 2c for comparison. (c) Average computer player performance (reward score, hit rate, accuracy, and eccentricity) as a function of stimulus coherence. Shaded background corresponds to the 99% confidence intervals of the median. See Figure 2a for comparison to human behavior.

Social modulation in human-computer (HC) dyads vs solo.

(a) Comparison of average hit rate (top), accuracy (center) and eccentricity (bottom) for each stimulus coherence in solo and human-computer dyads, color-coded for coherence level. Individual data is shown in darker hues. Each subject contributes one data point per coherence condition. Medians across subjects overlaid for each coherence condition (bright color). Error bars show 99% confidence intervals of the median. Same conventions as in Figure 5b. (b) Social modulation of humans between dyadic (human-computer, HC) and solo experiments. Same conventions as in Figure 5c.

Absolute solo eccentricity difference between dyadic partners did not correlate with average (across dyad members) dyadic accuracy modulation.

Random effects structure and sample size of each model

Results of the full model with hit probability as response

Results of the full model with eccentricity being the response

Results of the full model with accuracy being the response