Semantic representations in the visual cortex of blind and sighted humans

Reviewer #1 (Public review):

Summary:

This fMRI study shows that two regions of the visual cortex (BA18 and BA19) of blind and sighted individuals carry information about the physical similarity of objects denoted by words. This effect was found for written words (Braille in blind, visual in sighted) but not spoken words. The evidence complements earlier studies reporting physical similarity effects in the occipitotemporal cortex of blind and sighted individuals (e.g., Peelen et al., 2014).

Strengths:

The study addresses an important question in the fields of neural plasticity and visual cortex organization. The study is generally well-conducted and the findings are clearly presented.

Weaknesses:

While the evidence is statistically strong, it is currently incomplete because of missing control analyses (see below). The framing of the results, as arguing against the pluripotent cortex account, is not entirely convincing as it was not clear that the study addressed the key predictions of that account.

Main comments:

(1) The study is framed as a test of Bedny's "cognitively pluripotent cortex" proposal (2017) that attributes the increased visual cortex response to linguistic stimuli in blind individuals to high-level cognitive functions. Key evidence for this account came from studies showing increased responses in blind visual cortex to certain grammatical manipulations and to solving mathematical equations. The current study did not include such manipulations. Instead, the current study focused on the representation of objects denoted by single words. Bedny's account did not make a strong argument that the physical similarity of word referents should be differently represented in blind and sighted individuals - if it did, please state this explicitly. Indeed, evidence that (some regions of) the visual cortex represent objects similarly in blind and sighted individuals does not seem incompatible with it.

(2) Throughout the manuscript (including the abstract) it was not clear what was meant with "visual cortex" or "visual areas"; whether this refers to early visual cortex (V1/BA17) or to visual cortex more generally (e.g., BA17-BA19, occipitotemporal cortex (MT, etc)). This is important for the theoretical arguments and for the interpretation of the results. If visual cortex = BA17, the current results point to potentially important differences between blind and sighted individuals, with the physical similarity of objects only observed in the visual cortex of the blind. If visual cortex is meant to include areas beyond BA17, the blind and sighted show similarities in the current study, although such similarities have been observed before using similar research approaches.

(3) Related to the point above, the abstract does not accurately describe the results, as it only describes the similarities between blind and sighted but not the differences. The study revealed differences between groups, particularly in BA17 - primary visual cortex. The differences between the groups are also illustrated by the strikingly different searchlight results in the two groups separately (Figure S6). These differences do not reach significance in a whole-brain-corrected contrast, but that likely reflects a lack of power (particularly for a between-group contrast).

(4) Results were found for written words but not spoken words (Figure S9). This is somewhat surprising considering that the visual cortex was more strongly activated for written words in the sighted, with this activation presumably not adding any information about the physical properties of word referents. Together with the widespread significance of clusters correlating with the physical similarity matrix (Figure 6), this raises the possibility of a confound. It would be good to ensure that this is not the case, e.g., you could create similarity matrices based on word length, word visual similarity (e.g., overlap in letters), and word frequency, and correlate these matrices with the physical similarity matrix to ensure that these correlations are not positive (or if they are, partial it out).

(5) The study included a task manipulation, with participants either judging physical or conceptual properties. This task manipulation is a central aspect of the design but does not feature anywhere in the results, and is also not discussed or introduced in the text. It would be interesting to know whether the results depend on the property (physical/conceptual) being task-relevant. But more importantly, a potential concern is that the responses in the task (given for each object using a two-response button box) correlate with physical or conceptual similarity and that this explains the fMRI findings. For example, two objects that are elongated would both receive a "yes" button press when participants answer the question "is this elongated"; these objects would also be rated as physically similar. This may apply more to physical than conceptual similarity. To exclude this possibility, the responses need to be analysed and included in the fMRI analyses, either as a regressor in the GLM or as another matrix to be partialed out at the final stage of analysis.

(4) Many of the blind participants had some residual vision (9/20 had light perception, 2/20 had contour perception); this could possibly have prevented the reorganization of visual cortex.

Reviewer #2 (Public review):

Summary:

The authors show, through rigorous and extensive analyses, that the visual cortex in both congenitally blind and sighted participants represented differences between individual words presented across sensory modalities. In both groups, the activation patterns for words in the visual cortex reflected physical, but not conceptual similarity between word referents. This suggests a similar representation for both groups of words, one derived from vision-oriented mechanisms, and does not reflect significant functional reorganization in blindness.

Strengths:

The theoretical question is sound, as is the analysis approach. The authors' literature discussion is thorough, and the writing is clear.

Weaknesses:

I have only minor concerns left open.

(1) In the representational connectivity analysis, what is the average value across the brain? The authors compare the representational correlation across brain regions to the average value, but the average itself is not reported.

(2) Can the authors add a map showing the representational connectivity values across the brain in addition to the bar plot? It would make it easier to see what networks show similar neural representation to the visual cortex.

(3) Are the participants in the behavioral experiment from which the physical and conceptual similarity between word referents were collected matching in age or education with the fMRI participants?

(4) Although there are no group differences in the correlation of the physical similarity, I think it is important to acknowledge that the effect is only significant at the searchlight level in the blind early visual cortex (Figure S6).

Reviewer #3 (Public review):

Summary:

This study examines semantic processing in the visual cortex of both congenitally blind and sighted individuals using fMRI and multivariate pattern analysis (MVPA). The key finding is that the visual cortex in both groups encodes the physical properties of word referents, rather than their conceptual similarities. These results suggest that the same representational mechanisms operate in both the blind and sighted brain.

Strengths:

(1) The findings contribute to a broader understanding of cortical reorganization and provide evidence for top-down processing of word referents, even in the absence of visual experience.

(2) The experiment incorporates both spoken and written word presentations (Braille for blind participants), ensuring that the results are not confounded by modality effects.

(3) The study employs a rigorous methodological approach, combining multivariate and univariate analyses to strengthen the validity of its findings.

(4) The paper is well-structured and clearly written, making it easy to follow.

Weaknesses:

(1) The word stimuli consists of only 20 nouns referring to concrete entities. However, in the behavioral experiment, participants rated the physical and conceptual similarity of only 30 word pairs, which represents just a subset of all possible word pair combinations. The average similarity ratings across subjects were then used to construct stimuli similarity matrices, which were correlated with the fMRI similarity matrices in the MVPA analysis. What is the rationale for presenting only a small subset of all possible word pair combinations to participants? Additionally, the instruction to rate the "conceptual similarity" of word pairs seems somewhat ambiguous. Would "conceptual similarity" correlate with "physical similarity"? Instead of subjective ratings, why not use cosine similarity scores from pretrained language models to construct the "conceptual similarity" matrices? This approach could provide a more objective and reproducible measure of conceptual similarity.

(2) There are only six questions each for assessing the physical and conceptual properties of the words in the fMRI experiment. Most of the physical property questions focus on shape-related attributes (e.g., round, angular, elongated, symmetrical), while the conceptual properties are limited to three pairs of antonyms (living/non-living, natural/manufactured, pleasant/unpleasant). These aspects seem insufficient to comprehensively characterize the physical and conceptual properties of the nouns. What was the rationale behind selecting only these six questions? Could this limited set of attributes introduce bias in how the neural representations in the visual cortex are interpreted?

(3) Two of the blind participants are right-handed, and two may have some form of contour vision. What was the rationale for including these participants? In addition, the sample size for blind participants is relatively small (N = 20). Does the sample size provide sufficient justification for the main conclusion that the visual cortex in both blind and sighted groups represents the physical properties of word referents? Additionally, could individual differences among blind participants impact the results, and were any analyses conducted to account for such variability?

(4) I appreciate the authors' effort to integrate both univariate and multivariate approaches in their analyses. However, the results appear somewhat contradictory: The MVPA results suggest similar neural representations of word referents in the visual cortex for both blind and sighted participants. However, the univariate analyses indicate higher activation in the visual cortex of blind participants. How can these two findings be reconciled? The authors attributed the increased activation in the visual cortex of blind participants to their "enhanced excitability", but what exactly does "excitability" mean in this context? Could this increased activation instead reflect an alternative neural strategy for processing semantic information in the blind brain? If so, how does this align with the claim that similar representational mechanisms exist in both blind and sighted individuals?

(5) The authors interpret their findings to suggest that the visual cortex can represent the physical properties of words even without visual experience, attributing this to top-down modulation from higher cognitive regions, which then backprojects to the visual cortex. However, it is unclear why only physical properties, and not conceptual properties, are backprojected. If higher cognitive regions modulate the visual cortex in a top-down manner, wouldn't both physical and conceptual attributes be expected to influence its activity? Could the authors clarify the mechanism that selectively supports physical property encoding over conceptual representation?