Timescales of dyadic interactions.

Classical economic game theory mostly focuses on trial-by-trial decisions: each agent learns about the mutual outcomes of dyadic choices at the end of each round, or trial, e.g., both select option “square” in the trial T3 (row 2). Across trials, decision strategies, reflecting recent history of interactions and predictions of future choices of the other agent, can emerge and transition gradually or abruptly, e.g. from S1 to S2 (row 3). Our approach aims at expanding the classical games, enabling “transparency” in dyadic decision-making paradigms so that each agent can monitor the other agent’s ongoing social cues and actions continuously in real-time (row 1). Instantaneously coordinated actions may give rise to new strategies, e.g. leader-follower dynamics that emerge spontaneously, based on time- and space-continuous actions. Of course, decision-making in a social context requires agents to integrate longer-term experiences and predict consequences beyond situational strategies, for example, adapting to partners with different levels of competence or cooperative dispositions (row 4). Immediate partner visibility in naturalistic face-to-face interactions allows for efficient partner-specific learning and behavioural adjustments.

Overview and schematic of the composite panels for two variants of DIP visual displays.

The top row presents the configuration and the panels that are combined with the OLED display (implemented in DIP1 and DIP2); note that in the macaque/human DIP1, the eyetracker and the hand/body cameras are illustrated on only one side, and only one side of the panel composition is shown, for clarity. The bottom row presents the configuration and the panels that are combined in the projector-based design (implemented in DIP3 and DIPc).

Characterization of transparent displays

Description of the four DIPs

Dyadic Interaction Platforms in action.

Top row: OLED-based DIP1 and DIP2, bottom row: double projection-based DIP3 and DIPc. See Table 2 for descriptions.

Dynamic coordination in transparent Bach-or-Stravinsky decision game.

(A) Top panel: “fraction choosing own” i.e., choice of the individually preferred target for agent A (human confederate, red) and agent B (monkey, blue) in one session (running average of eight trials). The visual access to other’s actions was occluded in the middle part of the session (opaque). The confederate (red) switched between own and monkey’s preferred targets in blocks of 20 trials. Bottom panel: the average joint reward. Dashed green line - maximal attainable average joint reward, given the used payoff matrix. (B) Human vs monkey reaction time difference histograms for the three prevalent outcomes: coordinated selection of human’s preferred target (red), monkey’s preferred target (blue), and selection of own preferred target by each agent (magenta), in the two action visibility conditions. Modified from Moeller et al. (2023).

(A) Flow of the competitive foraging task. Left: two human players in DIP, using a joystick to collect targets, in a starting position. Middle: in each trial, two out of four targets are activated. Right: the active targets may be collected by the same subject or different subjects. (B) Effect of initial proximity. Left: a confederate chose pseudorandom locations along the horizontal axis at the beginning of each trial. Right: the dependency of subject’s choice (left or right active target) as a function of self and confederate’s initial distance from the two active targets. To combine the results across trials with various combinations of the active targets, we assigned the two active targets as the left and right targets, then pooled a cursor’s distance to the left and right targets across trials. (C) Dynamic decisions in two subjects during example trials, revealed by their cursor’s trajectory. Left: green player’s change of mind. Right: purple player’s change of mind.

Example of a perceptual decision-making paradigm on DIP, with corresponding behavioural data.

Left: Human subjects watched a 100% coherent random dot pattern (RDP) on both sides of the transparent OLED screen (DIP2). Using a joystick, they had to indicate whether the stimulus direction was moving leftward or rightward of the vertical midline. The stimulus direction changed instantly after pseudorandom time intervals. Right: Psychometric curves of two example subjects measured on both sides of the DIP screen (dark/bright). Data points indicate the percentage of reporting rightward direction as a function of stimulus difficulty (deviation from vertical direction) and direction (positive - rightward). The data are fitted with a bounded logistic function.

Eyetracking data collected using the DIPc.

(A) The proportion of time children spent looking at their social partner’s face relative to the images on screen across the trial. The lines depict the mean and shaded area the SE across the trial. The vertical line indicates the point at which children or their partner tapped on one of the images onscreen. (B) Children’s proportion of looking at the image that was chosen, i.e., tapped on, across trials based on whether they or their social partner in the task chose the image. The vertical line indicates the point at which children or their partner tapped on one of the images onscreen. Across trials where they or their social partner tapped on an image, children fixated this image prior to it being chosen showcasing the extent to which children were able to pre-empt their partner’s choice given the transparency of the DIPc setup.