Neuroscience

Crossmodal representations in the auditory cortex of deaf and hearing individuals

Barbara Manini
Konstantin Grin
Martin Eimer
Samuel Evans
Luigi Tamè
Bencie Woll
Rita Bertani
Dilay Ercelik
Lucy Core
Yueming Gao
Matthew Longo
Valeria Vinogradova
Velia Cardin author has email address

Deafness, Cognition and Language Research Centre and Department of Experimental Psychology, University College London (UCL), London, United Kingdom
School of Science, College of Science and Engineering, University of Derby, Derby, United Kingdom
Department of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
Neuroimaging Department, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, London, United Kingdom
Institute of Cognitive Neuroscience, University College London (UCL), London, United Kingdom
School of Psychology, University of Kent, Canterbury, United Kingdom
Center for Language and Brain, HSE University, Moscow, Russian Federation

https://doi.org/10.7554/eLife.112044.1

Open access
Copyright information

Figures and data

Demographics of deaf participants

Experimental design and stimuli.
Panel A: structure of an example trial in the visual and somatosensory modalities. Frequency of stimulation is shown in red. Red arrows in the visual trials indicate the lateral movement of the grating structure; these are for display purposes and were not present in the actual stimuli. Panel B: Stimuli features, showing the temporal frequency and spatial location of the visual and somatosensory stimuli. ITI: inter-trial interval. L: location. Red rectangles indicate the stimulated fingers.

RSA analysis.
The figure shows the structure of the Representational Dissimilarity Matrix (RDM) and a graphical representation of the theoretical models tested. There were three dimensions of interest: 1) Sensory Modality (Visual, Somatosensory); 2) Task (Temporal, Spatial) and 3) Stimuli features (Frequency, Location). L: location. See also Supplementary Figure 1.

Behavioural results.
Participants’ accuracy in the delay-to-match experiment averaged separately across groups, modalities and tasks. On each box, the edges indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median. See the Behavioural ANOVA here.

Results from Control ROIs.
A: Univariate analysis results. Each data point represents the contrast values for a participant for a given condition vs the implicit baseline. x=outlier not shown in the graph. preSMA: pre-supplementary motor area. B: Representational Dissimilarity Matrices (RDMs) for each group and each ROI. The colour bar represents the value of the group-average cross-validated Mahalanobis (crossnobis) distance. Notice that the colour scale was set to the maximum and minimum value of each ROI to highlight within ROI differences across conditions, rather than highlighting between ROI differences which are not informative in our analysis. C: Fisher-transformed z values for the Pearson correlation coefficients of relevant theoretical models. The triangles above the the bar plots show the regions of the dissimilarity matrix that were evaluated in each model. VL_M: Visual Location Model. VF_M: Visual Frequency Model. VFR_M: Visual Frequency Reduced Model. SL_M: Somatosensory Location Model. SF_M: Somatosensory Frequency Model. SFR_M: Somatosensory Frequency Reduced Model. TMD_M: Task Modality-Dependent Model. TMI_M: Task Modality-Independent Model. MOD_M: Sensory Modality Model. In the boxplots, the edges of each box indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median.

One Sample T-Tests for RSA Results in Control ROIs.

Univariate results for auditory Regions of Interest (ROIs).
Each data point represents participant-specific contrast values for a given condition vs the implicit baseline. On each box, the edges indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median. Results for the comparison of interest are indicated above each panel (see Supplementary Materials 2). Full ANOVA results can be found here: Auditory ROIs ANOVAs. Temp: Temporal. Som: Somatosensory. Spat: Spatial. RHG: right Heschl’s gyrus; LHG: left Heschl’s gyrus; RSTG/S: right superior temporal gyrus and sulcus; LSTG/S: left superior temporal gyrus and sulcus.

Representational Dissimilarity Matrices (RDMs) for each group and each auditory ROI.
RHG: right Heschl’s gyrus; LHG: left Heschl’s gyrus; RSTG/S: right superior temporal gyrus and sulcus; LSTG/S: left superior temporal gyrus and sulcus. The colour bar represents the value of the group-average cross-validated Mahalanobis (crossnobis) distance. Notice that the colour scale was set to the maximum and minimum value of each ROI to highlight within ROI differences across conditions, rather than highlighting between ROI differences which are not informative in our analysis.

Representation of stimuli-features in auditory ROIs.
The figure shows the Fisher-transformed z values for the Pearson correlation coefficients between the theoretical models for stimuli-features and each participant’s RDM. On each box, the edges indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median. RHG: right Heschl’s gyrus; LHG: left Heschl’s gyrus; RSTG/S: right superior temporal gyrus and sulcus; LSTG/S: left superior temporal gyrus and sulcus. VL_M: Visual Location Model. VF_M: Visual Frequency Model. VFR_M: Visual Frequency Reduced Model. SL_M: Somatosensory Location Model. SF_M: Somatosensory Frequency Model. SFR_M: Somatosensory Frequency Reduced Model. See also VL_M ANOVA, VF_M ANOVA, VFR_M ANOVA, SL_M ANOVA, SF_M ANOVA and SFR_M ANOVA.

Representation of task information in auditory ROIs.
The figure shows the Fisher-transformed z values for the Pearson correlation coefficients between the theoretical models and each participant’s RDM. On each box, the edges indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median. RHG: right Heschl’s gyrus; LHG: left Heschl’s gyrus; RSTG/S: right superior temporal gyrus and sulcus; LSTG/S: left superior temporal gyrus and sulcus. See also TMD_M ANOVA and TMI_M ANOVA.

Representation of sensory modality information.
The figure shows the Fisher-transformed z values for the Pearson correlation coefficients between the theoretical models and each participant’s RDM. On each box, the edges indicate the 25^th and 75^th percentiles, and the black horizontal line inside the box indicates the median. RHG: right Heschl’s gyrus; LHG: left Heschl’s gyrus; RSTG/S: right superior temporal gyrus and sulcus; LSTG/S: left superior temporal gyrus and sulcus. See also MOD_M ANOVA.

Number of voxels per ROI.

Univariate Analysis Independent Samples T-tests: Auditory ROIs.

RSA Models Independent Samples T-tests: Auditory ROIs.

Examples of all visual and somatosensory samples.
The order in the figure follows the order in the RSA matrices.

Examples of all visual and somatosensory samples.
The order in the figure follows the order in the RSA matrices.

Auditory ROIs (group overlap).
The figure shows the spatial overlap of auditory regions of interest (ROIs): Heschl’s Gyrus (HG, green) and Superior Temporal Gyrus/Sulcus (STG/S, red) for the deaf group (top panel) and the hearing group (bottom panel). Color intensity indicates the number of participants for whom a given voxel was included in the respective ROI (HG in green, STG/S in red).

Auditory ROIs (individual examples).
The figure shows examples of participant-specific auditory regions of interest (ROIs): Heschl’s Gyrus (HG, green) and Superior Temporal Gyrus/Sulcus (STG/S, red) from five deaf participants and five hearing participants. Numbers in white indicate the y (top rows) and z (bottom rows) coordinates.

Sign up for email alerts