Crossmodal representations in the auditory cortex of deaf and hearing individuals

Reviewer #1 (Public review):

Summary:

The authors conducted a carefully constructed experiment to test reorganization in the auditory cortex in deafness in response to task (spatial vs. temporal working memory) and modality (visual vs. somatosensory). They found a complex pattern of results, which included changes to univariate response strength in deafness that differed between the primary and association auditory cortex. HG showed a preference for the somatosensory working memory task, whereas STG/S responded more in both modalities for the temporal task. They further showed multivariate similarity of their results to models representing both task and modality in both groups, which were increased in deafness. Curiously, the task effect was for a sensorily-bound model, which shows mid-level representation, and not high-level task ("metamodal") code.

Strengths:

I appreciated the matched design, rigorous analysis, and careful interpretation of the nuanced results.

Weaknesses:

Only minor weaknesses: behavior and residual hearing can be better controlled.

Reviewer #2 (Public review):

Manini and colleagues present an interesting study on the consequences of early deafness on the organization of temporal regions chiefly engaged in audition in hearing people. Mainly relying on representational similarity analyses, they show that the auditory cortex in deaf individuals represents information about task, sensory modality, and somatosensory frequency. Critically, task and modality representations were also found in the auditory cortex of hearing individuals. There were significant differences between groups, implying that these representations are enhanced as a consequence of deafness.

Overall, I feel that the paper could gain in clarity and impact if the hypothesis space tested in the introduction and discussion was made clearer, if some new analyses were provided to support some claims, and if the authors better matched their conclusions to the observed results.

Reviewer #3 (Public review):

Summary:

This manuscript examines functional plasticity in auditory cortices of people born deaf (or early deaf). Deaf participants (N=13, all native BSL signers) and hearing controls (N=18) performed a delay-match-to-sample working memory task in visual and somatosensory modalities, attending either to frequency (temporal task) or spatial pattern (spatial task). Using fMRI and univariate and representational similarity analysis (RSA), the authors test what type of information is represented in auditory areas of hearing controls and deaf participants. Different types of tasks and stimulus features are examined, including low-level sensory features (vibration/movement frequency and spatial position of stimuli on screen/hand) as well as task features (attending to space vs. frequency) and stimulus modality (somatosensory vs. visual).

There are a number of interesting findings. In early auditory cortices (right Heschl's gyrus), only Deaf participants show above baseline responses and only in the somatosensory task. In secondary auditory/multisensory STS, only deaf participants show above-rest responses to both visual and somatosensory stimuli. In RSA analysis, d/Deaf, but not hearing participants, show sensitivity to the frequency of somatosensory vibration when finger position is held constant (SFRm model). RSA in auditory cortices finds sensitivity to sensory modality (visual vs. somatosensory) information in both hearing and deaf participants, with an enhanced effect in the d/Deaf group. A similar d/Deafness enhancement effect was observed for task (frequency vs. spatial) but only within modality, not across, suggesting a less abstract representation.

Strengths:

The paper has a number of strengths. Running tasks in multiple modalities in the same study is a technical challenge and adds valuable information, since, as it turns out, early auditory areas of deaf people are sensitive to somatosensory but not visual frequency. Moreover, it makes it possible to look for modality coding.

The RSA analysis finds sensitivity to task and modality features not detectable with univariate analyses.

Overall, the findings of differences across modalities (visual and somatosensory) in auditory cortices are very interesting. Having parallel tasks in the two modalities makes it all the more important that only somatosensory stimuli activate HG. The discussion of this finding and ideas about alternative possible paths of somatosensory and visual information to the auditory cortices in d/Deaf individuals is also interesting.

The authors conclude that there is both evidence for shared and different function across hearing and deaf groups, and this makes sense. It's refreshing that the authors acknowledge that the simplicity dichotomy of preservation vs. change present in the literature and presented in the introduction turns out not to explain the findings.

Weaknesses:

The participant sample is not very large, but this is a difficult-to-recruit population, and it is within an acceptable range since the authors have taken care to run a robust study design and collect a large amount of data from each person.

Some weakness of the paper includes incomplete presentation of the results and conclusions that do not follow from the data.

The results are presented in a way that makes it hard to track what is significant and how the auditory ROIs are different or similar to the control regions. It is also difficult to connect the written text results with the figures. In some cases, the figures look like there is no significant effect, but the results report that there is one.

It is not clear that three-way interactions were tested for (e.g., group, by task, by modality). This complicates the interpretation of significant two-way interactions, e.g., task by group. For example, in STG a task by group interaction is reported, but the plot suggests that this is driven primarily by the somatosensory modality.

The paper suggests that frequency/temporal information is coded in auditory areas of d/Deaf participants but not location/spatial information. This is stated in the Results and in the Discussion as a major point. But this is not quite true in the somatosensory case and not true in the visual case at all, as far as I can tell. In the somatosensory case, since frequency is only coded in a finger-dependent manner, location is in fact coded in this regard. This pattern differs from what is observed in the somatosensory cortex, where frequency is coded in a finger-dependent and independent manner as well as location as such. In the visual case, it seems like temporal, i.e., frequency information, is not coded at all in the auditory ROIs. Although it is also puzzling that visual frequency is not coded in the visual 'control' ROI. An incomplete presentation of results motivates some conclusions that are not warranted, e.g., coding in the auditory cortex reflects a preservation of its function - i.e., frequency but not location coding.

A second related issue is that some dimensions (e.g., visual frequency, modality-independent task) show no neural response anywhere in the brain, including in the canonical visual, somatosensory, and amodal networks. For these dimensions, the current experiment does not offer a good test. This is fine but should be clearly stated in the results and the Discussion so there is no confusion about which hypotheses are really tested. Right now, the results say things like "the control ROI shows the expected pattern", but in some cases it's more complicated and failures to observe effects constrain what can ultimately be expected in auditory areas. This is okay, but needs to be made clear in the results and discussion, and auditory results need to be interpreted in this context.

Relatedly, the paper has no whole cortex searchlight analyses, and it is not clear why. If nothing comes out in the small sample of n=13, that is okay, but at least the collapsed hearing and d/Deaf sample data should be shown for each dimension. This will give the reader a sense of what is to be expected and contextualize the results.

Some claims are made which are not supported by the data: "However, it is likely that the representation of somatosensory frequency in deaf individuals relies on the same mechanisms used to represent auditory temporal frequency in hearing individuals." Likewise, the paper goes on to say, 'the underlying computations might be the same'. True, they might be, but they also might be different. No evidence is presented to support one or the other hypothesis, so both should be stated, and it should be stated that these cannot be distinguished based on the presented data.

Conclusions-wise, the paper sometimes makes sweeping claims that go well beyond what the evidence supports and fails to provide caveats. The first paragraph of the Discussion states: "Overall, these findings suggest that crossmodal plasticity relies on representational and functional configurations that are present across individuals and modulated by sensory experience." Such a sweeping conclusion about cross-modal plasticity in general is not supported or refuted by the present data. There is no evidence that 'representations' or 'functional configurations' are the same across groups. What are functional configurations? What representations are shared? Second, it is far too general to make claims about 'cross-modal plasticity' based on one study with one population.

Author response:

We would like to thank the editor and reviewers for their thoughtful and constructive feedback. We appreciate the time and care devoted to reviewing our manuscript, as well as the recognition of rigorous experimental design, the technical challenges involved in conducting an fMRI study with two sensory modalities and two tasks in both deaf and hearing participants, and the value of the findings for understanding crossmodal plasticity and cortical organisation in deafness. We are encouraged by the overall assessment of the study, and appreciate the suggestions for strengthening the manuscript. Below, we provide a summary of how we plan to address the reviewers’ comments in our formal revision of the manuscript:

(1) Additional analyses

(a) We will incorporate behavioural performance measures into the relevant analyses to disentangle potential behavioural contributions to the observed effects.

(b) We will calculate the noise ceiling value for each of the RSA analyses. 

(c) We will conduct a correlation analysis between RDMs of auditory and control regions, to investigate the similarity between these computations and whether this is influenced by sensory experience.

(d) Regarding the suggestion to conduct whole-brain searchlight analyses, we respectfully do not believe that this approach would address the primary research question of the study, namely whether and how representations within auditory cortex differ between deaf and hearing individuals. Our central hypotheses specifically concern representational content within predefined auditory cortical regions, making the ROI-based approach the most appropriate and sensitive method for testing these questions.

Furthermore, the searchlight approach would require adequately powered group comparisons at the whole-brain level. Given the challenges associated with recruiting deaf native signers participants and the resulting sample size, we do not believe the study is sufficiently powered to draw reliable conclusions from this analysis. We will further clarify this rationale in the revised manuscript.

(2) Presentation of the results

We will revise the presentation of the findings to better guide the reader through the analyses and facilitate interpretation of the figures. In particular, we will ensure that significant effects, interactions, and their relationship to the corresponding figures are described more explicitly throughout the manuscript.

(3) Revision of the discussion

Following the reviewers’ feedback, we will revise the Discussion to more clearly distinguish between results that directly support a conclusion and hypotheses that remain speculative.