Trial structure.

1) Amodal detection: Participants were asked to indicate if they perceived or not a stimulus, irrespective of its modality. In frame: different possible stimulus types: the auditory stimulus was a sinusoidal tone of 1kHz presented in pink noise, the visual stimulus a light gray circle presented in dynamic Gaussian noise. 2) Amodal confidence: Participants indicated their confidence in their detection answer. 3) Modality-specific judgments and confidence: By moving a cursor on a bi-dimensional scale, participants indicated simultaneously whether they perceived a stimulus in each modality, and with which level of confidence.

Amodal detection.

Experiment 1 is represented by circles, while Experiment 2 is represented by diamonds; error bars represent the standard error. A) Amodal performance: amodal d’ and criterion. B) Amodal response: Percentage of stimuli judged to be present as a function of the experimental condition.

Effect of the presence of a stimulus in the other modality

Confidence by judgment.

Experiment 1 is represented by circles, while Experiment 2 is represented by diamonds. A) Amodal confidence as a function of hits, false alarms (FA), correct rejections (CR), and misses; error bars represent the standard error. B) Amodal metacognitive efficiency (response-conditional Mratio) as a function of the type of judgments; error bars represent the highest density interval. In both panels, presence judgements are represented in blue, and absence judgements in pink.

Multisensory effects.

Experiment 1 is represented by circles, while Experiment 2 is represented by diamonds. A) Amodal metacognitive sensitivity by modality as a function of the experimental condition, a slope superior to 0 indicates higher confidence in correct than incorrect responses and below 0 higher confidence in incorrect than correct responses; error bars represent the standard error. B) Amodal metacognitive efficiency (response-conditional Mratio) for presence judgments as a function of the modality of presentation; error bars represent the highest density interval.

Computational model.

A) Model architecture. The observer is assumed to have access to a visual sensor and an auditory sensor, probabilistically tuned to the presence of visual and auditory evidence. The probability of activation is controlled by the parameter θ. The model agent observes the activations and updates their beliefs about the presence of a signal in each modality separately, using Bayes’ rule. The agent then integrates the two beliefs into an amodal belief in the presence of a target. Based on this belief, they decide whether to commit to a decision or accumulate more evidence by following an optimal policy, derived using backward induction. B) Example trial: modality-specific log-likelihood ratios (LLR, in green) are updated following sensor inactivations and activations. C) Integration rules: The top plot represents the disjunctive rule and the bottom plot the conjunctive rule. Amodal LLR is plotted as a function of the number of sensor activations in each modality 50 time points (i.e., 2.5s) into the trial. Black contours indicate regions in which the best action is to decide present, wait, or decide absent.

Reproduction of perceptual effects.

Error bars represent the standard error from the data. Rectangles represent data simulated from the model, centered on the mean value and with height equal to the standard error. Left panels show the fit for Experiment 1 and right panels for Experiment 2. A-F) Percentage of stimuli judged to be present as a function of the condition of presentation for Experiments 1 and 2, at the amodal, auditory, or visual level. G-J) Observed and simulated source monitoring: Modality detected as a function of the modality of presentation in Experiments 1 and 2.

Confidence fits according to the different integration rules.

Error bars represent the standard error from the data. Rectangles represent data simulated from the model, centered on the mean value and with height equal to the standard error. Top plots represent the fit for Experiment 1 and bottom plots for Experiment 2. Each plot represents the amodal confidence as a function of condition of presentation and as a function of amodal hits, false alarms (FA), correct rejections (CR), and misses. The left panels represent the confidence based on the disjunctive rule. The middle panels represent the confidence based on the conjunctive rule. The right panels represent the confidence when absence is based on the disjunctive rule, while presence is based on the conjunctive rule.

Reproduction of modality-specific confidence effects.

Error bars represent the standard error from the data. Rectangles represent data simulated from the model, centered on the mean value and with height equal to the standard error. Left panels represent the fit for Experiment 1 and right panels for Experiment 2. A) Auditory confidence as a function of auditory hits, false alarms (FA), correct rejections (CR), and misses for Experiment 1. B) Visual confidence as a function of visual hits, false alarms (FA), correct rejections (CR), and misses for Experiment 1. C) Correlation between observed and simulated data for the confidence asymmetry index for Experiment 1. D) Auditory confidence as a function of auditory hits, false alarms (FA), correct rejections (CR), and misses for Experiment 2. E) Visual confidence as a function of visual hits, false alarms (FA), correct rejections (CR), and misses for Experiment 2. F) Correlation between observed and simulated data for the confidence asymmetry index for Experiment 2. In all panels, presence judgements are represented in blue, and absence judgements in pink.

Illustration of the integration rules process.

Detection decisions (red for absence, blue for presence) are based on the disjunctive integration rule (disjunction and negation of disjunction). Confidence decisions (dashed line for not sure, full line for sure) are either based on a conjunctive rule (confidence in presence) or a negation of disjunction (confidence in absence).

Fitted models (rows) with different or identical parameters (columns) across the visual and auditory sensors.

Visual psychometric curve for each participant of Experiment 1

Visual psychometric curve for each participant of Experiment 2

Auditory psychometric curve for each participant of Experiment 1

Auditory psychometric curve for each participant of Experiment 2

Modality-specific results.

Experiment 1 is represented by circles, while Experiment 2 is represented by diamonds. A) Auditory detection performance: Percentage of stimuli judged to be present at the auditory level as a function of the experimental condition; error bars represent the standard error. B) Auditory metacognitive sensitivity as a function of the experimental condition; error bars represent the standard error. C) Auditory metacognitive efficiency (response-conditional Mratio) for auditory present judgments as a function of the visual modality; error bars represent the highest density interval. D) Auditory metacognitive efficiency (response-conditional Mratio) for auditory absent judgments as a function of the visual modality; error bars represent the highest density interval. E) Visual detection performance: Percentage of stimuli judged to be present at the visual level as a function of the experimental condition; error bars represent the standard error. F) Visual metacognitive sensitivity as a function of the experimental condition; error bars represent the standard error. G) Visual metacognitive efficiency (response-conditional Mratio) for visual present judgments as a function of the auditory modality; error bars represent the highest density interval. H) Visual metacognitive efficiency (response-conditional Mratio) for visual absent judgments as a function of the auditory modality; error bars represent the highest density interval.

Comparison of the different models tested.

Reproduction of reaction times.

Reaction times (in seconds) as a function of the condition of presentation for Experiments 1 and 2. Error bars represent the standard error from the data. Rectangles represent data simulated from the model, centered on the mean value and with height equal to the standard error

Parameter recovery.