Feedback control of recurrent circuits imposes dynamical constraints on learning

Reviewer #1 (Public review):

Summary:

Gurnani et al. explore how dynamical properties of neural networks influence capacity for and mechanisms of learning. Specifically, they focus on Brain Computer Interface (BCI) learning, in which manipulations are applied to a decoder that maps neural activity onto computer cursors. This paradigm was introduced by Sadtler et al. 2014, and has become an influential part of the neuroscience motor learning literature. A particularly fascinating outcome of that body of work is the observation that "within-manifold" perturbations (WMPs), which preserve covariance structure in the neural population, are easier to learn than "outside-manifold" perturbations (OMPs), which break this. Since deep network parameter access is challenging (to say the least) in monkey experiments, the intuition for this split in learnability is ripe for modeling and theory work. Indeed, the authors here introduce a feedback-driven recurrent neural network model whose output drives a simulation of a neural decoder commonly used in BCI studies like the Sadtler paper. While there have now been several modeling studies exploring how neural networks could solve this task, the feedback control perspective gives the authors' new model an interesting niche. Overall, this is a thoroughly done and well-written modeling study, and a solid contribution to the literature on within- and outside-manifold perturbations.

Strengths:

Reframing the OMP and WMP learning from a feedback-driven dynamical systems perspective, not just a geometric one, is an interesting take. The controllability analysis (along with the clear difference in input-driven and recurrence-driven learning) is quite a cool result that helps better frame what might be happening in the primate brain during similar tasks.

Weaknesses:

Some of the more interesting aspects, especially the controllability) and the differences between input-driven and recurrence-driven learning could be further developed, either by showing more analyses or running more comparisons. A few sections could benefit from some additional clarity on the strength and significance of results.

Reviewer #2 (Public review):

Summary:

The constraints on learning in the brain remain elusive. Using BCIs, Sadtler et al. demonstrated that the brain can rapidly learn new decoders that lie within the intrinsic neural manifold (short-term adaptation), while showing substantial difficulty learning decoders that lie outside the manifold. This finding suggests that neural manifolds impose constraints on learning. However, even among within-manifold decoders, there was considerable variability in learning rates that could not be explained solely by geometric factors.

Here, Gurnani et al propose that, in addition to manifold structure, neural dynamics (i.e., the flow field across states) impose critical constraints on learning. To test this idea, the authors trained RNNs that received real-time feedback (e.g., position error signals) during a BCI task in which the network controlled a cursor. The authors showed that short-term adaptation to a new decoder is facilitated by plasticity in sensory inputs, and that pre-existing dynamics influence the speed of adaptation across different decoders. These findings may explain previously unresolved constraints observed in BCI learning and suggest an important role for neural dynamics in constraining sensorimotor learning in the brain.

Strengths:

Overall, the work is highly impactful and is likely to motivate a new generation of BCI and learning experiments combining large-scale neural recordings with latent dynamical systems analyses. The paper is clearly written, and I only have minor comments, primarily for clarification.

Weaknesses:

There are no major weaknesses. Please see below for minor comments.

(1) If I understand correctly, most analyses do not distinguish between the preparatory phase and the movement phase. Given that the preparatory phase is largely controlled by feedforward input, I suspect that most of the dynamical constraints underlying learning variability arise during the movement phase. Is this correct? If so, could the authors clarify or directly test this distinction?

(2) P4: Position vs. velocity decoders: It would be helpful to describe whether and how the choice of velocity versus position decoders influences whether perturbations are learnable, and whether input-driven constraints arising in this task are similar.

(3) The variance criteria used to screen decoder perturbations may themselves covary with learning rate, behavioral asymmetry, and overlap with controllable subspaces. A quantification of this relationship would help contextualize the findings and inform the design of future BCI experiments.

(4) To support the comparison between Figures 3 and 7, and the conclusion that Figure 3 better matches the experimental data, which is an important point of the manuscript, could the authors provide quantitative values from the experimental data (e.g., how large is the change in variance within oPCs, etc)?

(5) Figure 8h: Is the variability in learning rates in models with different controller networks explained by the same dynamical constraints described in Figure 6? Demonstrating consistent dynamical constraints across model architectures would strengthen the paper's central conclusion.

(6) Figure 8f: Why does feedforward controllability differ between conditions? This is mentioned in the text, but no explanation is provided.

Author response:

We thank the reviewers for such positive and constructive feedback, and for their enthusiasm about our use of controllability and dynamical systems perspectives to understand learning variability. We are glad to see that they believe this work will be “highly impactful” and “directly motivate new learning experiments”. We agree that these findings suggest new experimental tests of dynamical constraints on learning, in BCIs and motor control as well as other computations that depend on neural dynamics, such as decision-making tasks. Combined with new tools for data-driven identification of latent dynamics, we are excited to see how dynamical constraints can help understand learning outcomes across different tasks, brain areas, and individuals.

Based on reviewer comments, we identified three sets of analyses that will improve the clarity and strength of evidence for our primary conclusions.

(1) As the reviewers identified, a central contribution of this study is to show that continuous within-class variability becomes explainable by considering underlying dynamical structure. We realize this was insufficiently emphasized in Figure 6. All regression models included group-specific intercepts, so improvements from dynamical features reflect prediction beyond class-level differences. To quantify this directly, we compared against an intercept-only model and evaluated prediction of within-class residual variability (mean-subtracted). Geometric features did not improve performance beyond class means, whereas dynamical features significantly improved prediction (p<10^-5 for both behavioral measures). Moreover, only dynamical features predicted within-class residual variability (cross-validated R^² = 0.19 and 0.30 for learning speed and hit-rate change, respectively; p < 10<sup-8). We will add these analyses and revise the text to clarify this point.

Author response image 1.

Cross-validated R² for (left) learning speed and (right) change in hit rate, for true behavioral outcomes (total variability, blue) and after subtracting class means for OMPs and WMPs (residual variability, orange).

(2) We appreciate the reviewers’ comments to clarify what changes in neural structure are small, and to provide a quantitative comparison to changes observed in the primate BCI experiments.

We referred to published analyses of within-manifold perturbations (WMPs) in the primate BCI experiments, which reported <10% reduction in fractional variance within the intrinsic manifold for most sessions (Golub et al., 2017). (No comparable analysis was reported for OMP sessions.) For adaptation to WMPs, changes in variance within the intrinsic manifold in RNN models with input plasticity closely matched experimental observations (75th percentile: 94% of pre-learning variance in the model versus 90% in data), whereas recurrent plasticity RNN models produced substantially larger departures (78%). In fact, the entire distribution with recurrent plasticity was shifted to larger changes than those observed in most primate WMP sessions. A second comparison based on covariance changes along BCI dimensions (Figure 5 in [1]) yielded a similar conclusion. The authors estimated ~5-20% changes in covariance along both the intuitive and perturbed decoder dimensions during WMP sessions. For our RNN models trained with input plasticity, we observed similar changes: changes along the perturbed decoder were <10% although changes along the intuitive decoder were ~40%. We borrowed the terminology of “small” from the experimental findings in [1], where comparisons were made to alternative learning hypotheses (with predicted changes as >10-fold higher). These analyses now provide more quantitative evidence that neural reorganization under input plasticity is largely consistent with primate neural data. We will add these comparisons as a supplementary figure in the revised manuscript.

Author response image 2.

Proportion of maps with normalized variance in intrinsic manifold (IM) above a certain minimum value. Results with training RNNs on WMPs, with either input plasticity (blue) or recurrent plasticity (orange), overlaid on primate data from Golub et al, 2017 (black). Dashed lines indicate the 75th percentile value.

We agree with reviewers that under input plasticity, both statistical and dynamical changes are relatively modest, particularly when compared to the behavioral changes. Rather than focusing on the magnitude of these changes, our regression analyses in Figure 6 highlight that the dynamical changes are a better predictor of continuous variability of behavioral outcomes. Moreover, OMPs are misaligned with both the intrinsic manifold and the controllable subspace. Thus, mean OMP learning performance alone cannot disentangle the contribution of these different sources of misalignment. By showing that variability within each class is explained by considering dynamics (Figure 4, Figure 6), and using the dissociation between task manifold and controllable subspace by varying controller architecture (Figure 8), we provide evidence that dynamical constraints provide a more comprehensive picture of learning variability, beyond categorical differences.

(3) Finally, we tested whether the same dynamical features explain learning variability across the alternative controller architectures in Figure 8. They remained predictive of learning speed (cross-validated R² of 0.35 and 0.33 for low-D and high-D controller networks respectively), supporting the generality of the proposed dynamical constraints. We will add this analysis to the revised manuscript.

As per reviewer suggestions, we will also perform additional analyses to examine the relationship of learning outcomes to initial behavioral metrics for different decoders, assess flowfield changes during the preparatory phase, report the relevant statistics for stated comparisons, and clarify that learning with only one set of inputs (either feedforward or feedback) was poorer.  We will also clarify several points raised by the reviewers, including:

(i) the compatibility of overlapping confidence intervals of WMP/OMP learning outcomes with prior experimental data in Sadtler et al, 2014;

(ii) the distinction between flow-field changes in the full neural state space (Figure 5D) and along behavioral readout dimensions (Figure 5E);

(iii) that autonomous dynamics contribute to controllability and how differences in pre-trained autonomous dynamics across controller architectures could indirectly vary feedforward controllability (Figure 8); and

(iv) the relationship between controllability and reachable manifolds in position-decoder BCIs.

References:

(1) [Golub et al, 2017]   Golub, M.D., Sadtler, P.T., Oby, E.R., Quick, K.M., Ryu, S.I., Tyler-Kabara, E.C., Batista, A.P., Chase, S.M. and Yu, B.M., 2018. Learning by neural reassociation. Nature neuroscience, 21(4), pp.607-616.

(2) [Sadtler et al, 2014]   Sadtler, P.T., Quick, K.M., Golub, M.D., Chase, S.M., Ryu, S.I., Tyler-Kabara, E.C., Yu, B.M. and Batista, A.P., 2014. Neural constraints on learning. Nature, 512(7515), pp.423-426.