Dual-feature selectivity enables bidirectional coding in visual cortical neurons

Reviewer #1 (Public review):

This manuscript used deep learning to highlight the role of inhibition in shaping selectivity in primary and higher visual cortex. The findings hint at hitherto unknown axes of structured inhibition operating in cortical networks with a potentially key role in object recognition.

The multi-species approach of testing the model in macaque and mouse is excellent, as it improves the chances that the observed findings are a general property of mammalian visual cortex. However, it would be useful to delineate any notable differences between these species, which are to be expected given their lifestyle.

The overall performance of the model appears to be excellent in V1, with over 80% performance, but it falls substantially in V4. It would be important to consider the implications of this finding; for example, in the context of studying temporal lobe structures that are central to recognizing objects. Would one expect that model performance decreases further here, and what measures could be taken to avoid this? Or is this type of model better restricted to V1 or even LGN?

While the manuscript delineates novel axes of inhibitory interactions, it remains unclear what exactly these axes are and how they arise. What are the steps that need to be taken to make progress along these lines?

Reviewer #2 (Public review):

The classic view of sensory coding states that (excitatory) neurons are active to some preferred stimuli and otherwise silent. In contrast, inhibitory neurons are considered broadly tuned. Due to the gigantic potential image space, it is hard to comprehensively map the tuning of individual neurons. In this tour de force study, Franke et al. combine electrophysiological recordings in macaque (V1, V4) and mouse (V1, LM, LI) visual cortex with large-scale screens based on digital twin models, as well as beautiful systems identification (most/least activating stimuli). Based on these digital twins, they discover dual-feature selectivity (which they validate both in macaques and mice). Dual-feature selectivity involves a bidirectional modulation of firing rates around an elevated baseline. Neurons are excited by specific preferred features and systematically suppressed by distinct, non-preferred features. This tuning was identified by excellently combining advances in AI & high-throughput ephys.

The study is comprehensive and convincing. Overall, this work showcases how in silico experiments can generate concrete hypotheses about neuronal coding that are difficult to discover experimentally, but that can be experimentally validated! I think this work is of substantial interest to the neuroscience community. I'm sure it will motivate many future experimental and computational studies. In particular, it will be of great interest to understand when and how the brain leverages dual-feature selectivity. The discussion of the article is already an interesting starting point for these considerations.

Strengths:

(1) Using computational models to predict neuronal responses allowed them to go through millions of images, which may not be possible in vivo.

(2) The cross-species and cross-area consistency of the results is another major strength. Pointing out that the results may be a fundamental strategy of mammalian cortical processing.

(3) They show that the feature causing peak excitation in one neuron often drives suppression in another. This may be an efficient coding scheme where the population covers the visual manifold. I'd like to understand better why the authors believe that this shows that there are low-dimensional subspaces based on preferred and non-preferred stimulus features (vs. many more, but some axes are stronger).

Author response:

We thank the reviewers for their constructive and helpful feedback on our manuscript. We are delighted that they found the study to be "comprehensive and convincing" and a "tour de force" in its combination of electrophysiological recordings with large-scale digital twin screening. We appreciate that the reviewers highlighted the strengths of our multi-species approach and the "cross-species and cross-area consistency" of the results, noting that the work showcases how in silico experiments can generate concrete, experimentally validatable hypotheses.

The reviewers also raised several important points that we plan to address in the final version of the manuscript to improve clarity and interpretation. These center on:

Model performance in V4: Reviewer #1 raised questions regarding the comparative drop in model performance in V4 and the implications for the validity of the results (including the use of "high confidence" neurons and a request for clarification on the number of animals in the V4 dataset).

Species differences: Both reviewers noted the value of the macaque-mouse comparison but requested a more explicit delineation of the differences between these species given their distinct ethological niches.

The nature of inhibitory dimensions: The reviewers asked for further details on how to identify these inhibitory dimensions and the specific relationship between excitation and inhibition. We believe unraveling these mechanisms represents an exciting direction for future work, and we will explicitly mention this in the Discussion section of the final manuscript, alongside a clearer contextualization with prior literature.

Technical clarifications: Reviewer #2 requested clarifications on specific technical details, such as the skewness thresholds used for sparsity analysis.

In the final version of the manuscript, we will address these points by adding necessary clarifications to the text—including confirming the animal cohort details—explicitly contrasting the mouse and macaque data to highlight coding differences, and expanding our discussion. We will also ensure all technical inquiries, such as those regarding skewness and reference citations, are fully resolved.

We believe addressing these points will significantly strengthen the manuscript.