Potential neural signatures of learning.

A) Learning a new task may cause changes in the neural activity used for a familiar task, and these signatures may help maintain knowledge about the new task to lead to retention savings if the new task is encountered again. B) Losey et al18 studied a BCI learning task where monkeys performed a center-out reaching task with a familiar Map A (Task A1, pink), learned a new Map B (Task B1, light blue), then returned to the familiar Map A (Task A2, red). They identified a memory trace that made neural activity (colored dots) during movement execution for the familiar Map A more useful for a new Map B after learning this new Map B. Importantly, the memory trace existed without compromising the performance for Map A. C) Sun et al11 studied a motor learning task where monkeys performed a baseline center-out reaching task (pink), learned to counteract a force field (light blue), then returned to the baseline task (red). They identified a uniform shift (colored dots) in the neural population space during movement preparation when monkeys learned to adapt to the force field that was sustained after returning to the baseline task.

RNNs recapitulate experimental results in sequential learning.

A) Networks were trained to perform center-out reaches by using different BCI maps which linearly transformed network activity to output velocities. Map A was the intuitive map based on the “intrinsic manifold” –a low-dimensional flat surface that captured the dominant covariation patterns of the network activity after baseline learning–, while Map B was a permutation of Map A. Produced velocities were fed back into the network after a 120 ms delay. B) Sequential training procedure: we first trained the recurrent, input, and output network weights during an initial training phase where networks had to produce random center-out reaches. Then, networks were trained to produce center-out reaches to eight fixed targets using different BCI maps (A or B) as a fixed output layer. Networks learned to produce output through these maps in blocks: from Map A (Task A1), to Map B (Task B1), then back to Map A (Task A2). Loss curves during each task block for an example network. Inset: position trajectories at the end of each block. C) Mahalanobis distance between population activity means for Task A1 and Task A2 for each target (black), and for controls where the task labels were shuffled (gray). Dotted lines, means across 10 different maps for each random seed (n=8 random seeds).

Uniform shifts and memory traces arise from sequential learning without explicit mechanisms.

A) Population activity during preparation (first 200 ms of each trial) for an example session. Circles, time-averaged, trial-averaged activity for each target (mean target activity) in each task block. Triangles, time-, trial-, and target-averaged activity (i.e., mean task activity) in each block. The uniform shift axis is quantified by the direction of the vector connecting the mean task activity from Task A1 to Task B1. The uniform shift across tasks is calculated by projecting the mean task activity onto the uniform shift axis and calculating the resulting distance between tasks. B) Uniform shift from Task A1 to Task B1 and Task A1 to Task A2. Dotted lines, means across 10 different maps for each random seed. C) Normalized uniform shift, calculated by dividing by the uniform shift from Task A1 to Task B1 for each session. D) Uniform shift from Task A1 to Task B1 across targets. Big circle and error bars, means and 95% confidence intervals with bootstrapping for 10 different maps for each random seed (n=8 random seeds); small circles, different maps. Note that uniform shifts are consistently greater than zero (dotted line) indicating a shift in population activity. E) Task A1 and Task A2 activity were projected into Map A online or into Map B offline. The resulting output was used to assess how useful the activity is for each map, i.e., how close to the target output it brings the cursor. F) MSE between the produced and target output for each projection for an example target in an example session. Dashed lines, mean MSE across trials for that target. The memory trace is quantified as the decrease in mean MSE (i.e., the increase in performance) from Task A1→Map B to Task A2→Map B. G) MSE across all targets for projections onto the two different maps. Circles and error bars, means and 95% confidence intervals with bootstrapping for 10 different maps for each random seed (n=8 random seeds). Traces, different sessions. H) Same as D but for the memory trace. Note that memory traces are consistently greater than zero (dotted line) indicating increased offline performance for Task B with activity used for Task A2 compared to that for Task A1.

The magnitude of memory traces but not that of uniform shifts correlates with retention savings.

A) Mean memory trace across targets compared to the retention savings. Small circles, mean for each map (n=10 maps) for each random seed (n=8 random seeds); large circle, example shared across A-D. B) Mean memory trace across targets compared to the initial velocity error (MSE) when Task A1 activity is projected into the new Map Bs before learning in Task B1 (i.e. Task A1→Map B MSE). C) Uniform shift from Task A1 to Task B1 across targets compared to the retention savings. D) Uniform shift from Task A1 to Task B1 across targets compared to the initial velocity error (MSE).

Learning new outside-manifold maps leads to larger memory traces than learning new within-manifold maps due to less forgetting.

A) Networks were trained on different Map Bs representing different within- and outside-manifold perturbations that had the same Task A1→Map B MSE. B) Map B MSE during initial learning of the new Map B in Task B1. Lines and shaded surfaces, mean and 95% confidence interval across 10 maps for each perturbation across networks of different seeds (n=8 random seeds). Lines, different maps for different seeds. C) Map B MSE during reversion to the familiar Map A in Task A2. D) Mean memory trace across targets compared to the Task A1→Map B MSE. Circles, mean for each map for each random seed. Note that points with the same Task A1→Map B MSE belong to the same seed since this was controlled for. E) Uniform shift from Task A1 to Task B1 across targets compared to the Task A1→Map B MSE. F) Uniform shift from Task B1 to Task A2 compared to forgetting, defined as the decrease in performance from Task B1→Map B to Task A2→Map B, for within-manifold perturbations. Lines and shaded surfaces, mean and 95% confidence interval across different maps for different seeds; circles, different maps for different seeds.

Context cues alter uniform shifts and memory traces.

A) Networks were given an additional context cue that signaled whether the familiar Map A or the new Map B was being used. The context signal was a one-hot encoded vector, and the vector was multiplied by different factors to create different context magnitudes. B) Uniform shifts from Task A1 to Task B1 for different context magnitudes. Line and shaded surfaces, mean and 95% confidence interval across 10 maps for each perturbation across networks of different seeds (n=8 random seeds). C) Normalized uniform shifts from Task A1 to Task B1 and from Task A1 to Task A2, averaged across targets for each session. Shifts were normalized by the length of the shift from Task A1 to Task B1. Opacities, different context magnitudes (most transparent for smallest magnitude). Dashed lines, mean for each context magnitude. D) Memory trace for different context magnitudes. Line and shaded surfaces, mean and 95% confidence interval across maps for each perturbation across networks of different seeds. E) MSE when activity from the end of Task B1, the beginning of Task B2, and the end of Task A2 are projected into Map B for different context magnitudes for within-manifold perturbations. Task B1→Map B MSE indicates learning difficulty, the difference between Task B1→Map B and beginning of Task B2→Map B MSE indicates forgetting, and the difference between the beginning of Task B2→Map B and Task B1→Map B MSE indicates activity changes due to context. F) Same as E but for outside-manifold perturbations. G) Mean memory trace across targets compared to the retention savings. Small circles, mean for each map for each random seed.

Neural activity changes from Task A1 to Task A2.

A) Mahalanobis distance between population activity means for Task A1 and Task A2 for each target during preparation (first 200 ms of each trial, black), and for controls where the task labels were shuffled (gray). Dotted lines, means across 10 different maps for each random seed (n=8 random seeds). B) Classification accuracy for classifying tasks (Task A1, Task B1, Task B2) from neural activity during preparation using linear discriminant analysis. Large circle, mean across maps and random seeds; small circles, accuracy for each map for each random seed; colors, different random seeds; dotted line, chance accuracy level.

Uniform shifts for different simulations.

A) Uniform shifts can be quantified per target for each session rather than averaged across all targets for each session as in Fig. 3B. To quantify uniform shifts per target, mean target activity (circles) was projected on the uniform shift axis determined based on mean task activity (triangles) for Task A1 and Task B1. The uniform shifts were then the resulting distances between these projected mean states. B) Uniform shifts for 10 different maps based on within-manifold permutations that shuffled all manifold dimensions (used in Figs 2-4). B.i) Uniform shifts from Task A1 to Task B1 and Task A1 to Task A2 for each target. Dotted lines, means across 10 different maps for each random seed. B.ii) Uniform shift per target, normalized by the uniform shift averaged across targets from Task A1 to Task B1 for each session. C) Uniform shifts for 24 different maps based on within-manifold perturbations that shuffled the first four high-variance dimensions (used in Fig 4). C.i) Uniform shift averaged across targets (as in Fig. 3B). C.ii) Normalized uniform shift averaged across targets (as in Fig. 3C). C.iii-iv) Same as B.i-ii but for high-variance maps. D) Same as C but for 24 different maps based on within-manifold perturbations that shuffled the last four low-variance dimensions (used in Fig S6). E) Same as C but for 10 different within-manifold perturbations with the same initial output MSE (used in Fig 5). F) Same as C but for 10 different outside-manifold perturbations with the same initial output MSE (used in Fig 5). G) Same as C but for 10 outside-manifold and 10 within-manifold perturbations with the same initial output MSE for networks with different context magnitudes (used in Fig 6). Opacities, different context magnitudes (most transparent for smallest magnitude).

Memory traces consistently arise and are due to learning Map B.

A) MSE between the produced and target output for Task A1 and Task A2 activity when projected into Map A and Map B for each target in an example session (same session as Fig 3E). B-C) To show that memory traces arise specifically due to changes incurred from learning Map B, networks were trained on Task A2 without altered weights (standard, blue), with weight changes from Task B randomly reallocated before Task A2 (dark gray), or with weights reset before Task A2 to those at the end of Task A1 (light gray). B) Weight changes from end of Task A1 to beginning of Task A2. C) Memory traces for each target for each session for 10 different maps for each random seed (n=8 random seeds) after the weight manipulations outlined in B.

Relationship between memory trace and retention savings.

A) Schematic for how memory trace and retention savings are measured based on MSEs at different stages of learning without context cues. Note that the memory trace is closely related to retention savings. The memory trace is defined as the increase in performance from Task A1→Map B to Task A2→Map B while retention savings is defined as the increase in performance from the beginning of Task B1→Map B to the beginning of Task B2→Map B. Without context cues, the network dynamics are identical at the end of Task A1 (Task A2) compared to the beginning of Task B1 (Task B2), other than minimal contributions from feedback, such that the memory trace is equivalent to retention savings, other than the contributions from feedback. B) Schematic for how memory trace and retention savings are measured based on MSEs at different stages of learning with context cues. Note that memory trace is not equivalent to retention savings when context cues are added because the network dynamics at the end of Task A1 (Task A2) are not identical to those at the beginning of Task B1 (Task B2).

The magnitude of memory traces and uniform shifts compared to different measures.

A) Networks were trained on different Map Bs representing within-manifold perturbations made from permutations in “all” manifold dimensions (blue, n=10 maps), only the first four “high-variance” dimensions (purple, n=24 maps), or only the last four “low-variance” dimensions (green, n=24 maps). Top: Permutation matrices for example maps for each category. Bottom: Output positions when Task A1 activity is projected into the new maps before learning in Task B1. B) Mean memory trace across targets. Big circle and error bars, means and 95% confidence intervals with bootstrapping across different maps for each random seed (n = 8 random seeds); small circles, different maps for different random seeds. C) Uniform shift from Task A1 to Task B1 across targets. D-H) Mean memory trace across targets compared to different measures: from left to right, retention savings, mean principal angle between Map A and Map B, variance accounted for (VAF) in Task A1 activity by Map B, Task A1→Map B angular error, and Task A1→Map B velocity MSE (Methods). Open circles, different maps for different random seeds; closed circles, examples for each permutation type shared across D-M. I-M) Uniform shift from Task A1 to Task B1 across targets compared to different measures.

Relationship between memory trace and uniform shift and different factors once initial MSE is fixed.

A) Task A1 activity was projected into thousands of candidate BCI maps representing within- and outside-manifold perturbations to obtain the initial Task A1→Map B MSE (Methods). Dashed lines, median for maps for each random seed. Opacities, different random seeds (n=8 random seeds). The 10 maps for each type of perturbation with MSEs closest to the median MSE across all maps for each seed were chosen for training. B) Uniform shift from Task A1 to Task B1 across targets compared to the mean memory trace across targets. Circles, mean for each map for each random seed. C-G) Mean memory trace across targets compared to the retention savings. C-G) Mean memory trace across targets compared to different measures: from left to right, retention savings, mean principal angle between Map A and Map B, variance accounted for (VAF) in Task A1 activity by Map B, Task A1→Map B angular error, and Task A1→Map B velocity MSE (Methods). Circles, different maps for different random seeds. H-L) Uniform shift from Task A1 to Task B1 across targets compared to different measures.

Learning within- and outside-manifold maps under different training regimes.

A) Networks were trained with the same Map A and Map B pairings as in the main text (Fig 5), but the within- or outside perturbation (Map B) was used as the familiar map while the intuitive map (Map A) was used as the new map, such that learning proceeded from baseline learning, to Map B, to Map A, and back to Map B. Memory trace compared to Task A1→Map B MSE. Circles, mean for each map (n=10 maps per perturbation type) for each random seed (n=8 random seeds). B) Networks were trained with the same Map A and Map B pairings, but they were trained simultaneously during baseline learning after network initialization. Map A MSE compared to Map B MSE after baseline learning. Circles, mean for each map for each random seed; dashed line, identity line. Note that Map B had worse performance than Map A, especially for within-manifold perturbations.

Larger memory traces for outside-manifold learning are not due to performance tradeoffs.

A) Memory traces compared to Task B1→Map B MSE. Greater memory traces for outside-manifold perturbations were not due to better initial learning of Map B in Task B1. Circles, mean for each map (n=10 maps per perturbation type) for each random seed (n=8 random seeds). B) Memory traces compared to Task B1→Map A MSE. Greater memory traces for outside-manifold perturbations were not due to worse retention of Map A in Task B1. C) Memory traces compared to Task A2→Map A MSE. Greater memory traces for outside-manifold perturbations were not due to worse relearning of Map A in Task A2.

Context disrupts the correlation between memory traces and savings.

A) Uniform shift from Task A1 to Task B1 across targets compared to mean retention savings for different context magnitudes. Circles, mean for each map (n=10 maps per perturbation type) for each random seed (n=8 random seeds). B) Mean memory trace across targets compared to mean retention savings for different context magnitudes.

Manifold overlap for different context magnitudes.

Manifold overlap between the activity after learning the new map in Task B1 and the original intuitive manifold for within- and outside-manifold perturbations. Line and shaded surfaces, mean and 95% confidence interval across 10 maps for each perturbation across networks of different seeds (n=8 random seeds).

Memory traces and sustained uniform shifts arise when learning is confined to upstream of motor cortex.

A) Schematic for network models with an additional RNN module added upstream. Activity from the downstream module is directly used to control the BCI, similar to how activity from the motor cortex is often used to control BCIs. The additional upstream module models brain regions upstream of the motor cortex. We examined how memory traces and uniform shifts differed when learning was restricted to the upstream module (frozen downstream weights) compared to when learning was allowed in the downstream module (plastic downstream weights). B) Mean memory trace across targets. Big circle and error bars, means and 95% confidence intervals with bootstrapping for 10 different maps for each random seed (n=8 random seeds). Small circles, different maps; colors, different random seeds. Note that memory traces were consistently greater than zero (grey dotted line), even when learning is restricted upstream. C) Uniform shift from Task A1 to Task B1 across targets. D) Uniform shift from Task A1 to Task A2 across targets, normalized to the uniform shift from Task A1 to Task B1. Note that the uniform shifts were sustained since they were consistently greater than zero (grey dotted line) and close to the shift from Task A1 to Task B (purple dotted line).