A Python Toolbox for Representational Similarity Analysis

Reviewer #1 (Public review):

Summary

This manuscript presents an updated version of rsatoolbox, a Python package for performing Representational Similarity Analysis (RSA) on neural data. The authors provide a comprehensive and well-integrated framework that incorporates a range of state-of-the-art methodological advances. The updated version extends the toolbox's capabilities.

The paper outlines a typical RSA workflow in five steps:

(1) Importing data and estimating activity patterns.

(2) Estimating representational geometries (computing RDMs).

(3) Comparing RDMs.

(4) Performing inferential model comparisons.

(5) Handling multiple testing across space and time.

For each step, the authors describe methodological advances and best practices implemented in the toolbox, including improved measures of representational distances, evaluators for representational models, and statistical inference methods.

While the relative impact of the manuscript is somewhat limited to the new contributions in this update (which are nonetheless very useful), the general toolbox - here thoroughly described and discussed - remains an invaluable contribution to the field and is well-received by the cognitive and computational neuroscience communities.

Strengths:

A key strength of the work is the breadth and integration of the implemented methods. The updated version introduces several new features, such as additional comparators and dissimilarity estimators, that closely follow recent methodological developments in the field. These enhancements build on an already extensive set of functionalities, offering seamless support for RSA analyses across a wide variety of data sources, including deep neural networks, fMRI, EEG, and electrophysiological recordings.

The toolbox also integrates effectively with the broader open-source ecosystem, providing compatibility with BIDS formats and outputs from widely used neuroscience software. This integration will make it easier for researchers to incorporate rsatoolbox into existing workflows. The documentation is extensive, and the scope of functionality - from dissimilarity estimation to statistical inference - is impressive.

For researchers already familiar with RSA, rsatoolbox offers a coherent environment that can streamline analyses, promote methodological consistency, and encourage best practices.

Weaknesses:

While I enjoyed reading the manuscript - and even more so exploring the toolbox - I have some comments for the authors. None of these points is strictly major, and I leave it to the authors' discretion whether to act on them, but addressing them could make the manuscript an even more valuable resource for those approaching RSA.

(1) While several estimators and comparators are implemented, Figure 4 appears to suggest that only a subset should be used in practice. This raises the question of whether the remaining options are necessary, and under what circumstances they might be preferable. Although it is likely that different measures are suited to different scenarios, this is not clearly explained in the manuscript. As presented, a reader following the manuscript's guidance might rely on only a few of the available comparators and estimators without understanding the rationale. It would be helpful if the authors could provide practical examples illustrating when one measure might be preferred over another, and how different measures behave under varying conditions-for instance, in what situations the user should choose manifold similarity versus Bures similarity?

(2) The comparison to other RSA tools is minimal, making it challenging to place rsatoolbox in the broader landscape of available resources. Although the authors mention some existing RSA implementations, they do not provide a detailed comparison of features or performance between their toolbox and alternatives.

(3) Finally, given the growing interest in comparing neural network models with brain data, a more detailed discussion of how the toolbox can be applied to common questions in this area would be a valuable addition.

Reviewer #2 (Public review):

Summary:

The manuscript, "A Python Toolbox for Representational Similarity Analysis", presents an overview of the RSAToolbox, including a review of the methods it implements (some of which are more recently developed) and recommendations for constructing RSA analysis pipelines. It is encouraging to see that this toolbox, which has existed in both Python and other forms, continues to be actively developed and maintained.

Strengths:

The authors do a nice job reviewing the history of RSA analysis while introducing the methods within the toolbox. It is helpful that the authors discuss when and how to apply specific measures to different data types (e.g., why Euclidean or Mahalanobis distances are suboptimal for spike data). The manuscript strikes a valuable balance between theoretical background and hands-on instruction. The inclusion of decision-making aids, such as the Euler diagram for selecting similarity measures, and well-maintained demo scripts (available on GitHub), enhance the manuscript's utility as a practical guide.

Overall, this paper will be particularly useful to researchers new to RSA and those interested in performing a rigorous analysis using this framework. The manuscript and accompanying toolbox provide everything a researcher needs to get started, provided they take the time to engage with the methodological details and references offered

Weaknesses:

While the links to the demos in the figure legend did not work for me, it was easy to locate the current demos online, and it's encouraging to see that they are actively maintained. One small issue is that a placeholder ("XXX") remains in the description of Figure 3b and should be corrected.

Author response:

We thank the reviewers for their valuable feedback. We will prepare a revision of the manuscript based on these suggestions and comments. We are sure these revisions will improve the paper.

The only major point we wish to clarify is that this is the first and only manuscript describing the toolbox; it is not a version update. Although it shares a similar name with its 2015 MATLAB predecessor (Nili et al., PLoS Comput Biol), rsatoolbox was designed from scratch. Also, they have no code or structural overlap beyond implementing some similar methods.

Developed publicly since 2019, rsatoolbox reflects a decade of research in RSA methodology across multiple labs and incorporates new dissimilarity metrics, RDM comparators, inferential procedures, and visualization methods. Importantly, although we cite several papers describing methods implemented in the toolbox, this is the first manuscript to present the toolbox as a whole, its design principles, and the unified analytical framework it offers.

We are sorry about the forgotten placeholder and the links not working. The links work for us in the pdf at least and we will certainly fix the placeholder as soon as possible.