Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex
- Stanford University, United States
- Cold Spring Harbor Laboratory, United States
- Columbia University Medical Center, United States
- Palo Alto Medical Foundation, United States
Figures
Figure 1 with 1 supplement
Decision-maze task.
(A) Illustration of task setup. Two targets were presented along with four virtual barriers and a frame. The monkey performed the task with a cursor projected above his fingertip. Targets were rewarded equally. The cursor left a white trail on the screen. (B) Task timeline. (C and D) The two families of mazes used: ‘T-maze’ (C) and ‘S-maze’ (D). Key barriers could take one of three positions, making each target easy, difficult, or blocked (shown here as shades of gray). Reaches for trials with ≥300 ms delay shown. Faded colors, reach trajectories on forced choice trials; saturated colors, reach trajectories on free choice trials. (E and F) Overt changes of mind on free-choice trials with no barrier changes. Dataset J1 (A–E); dataset N3 (F).
Figure 1—figure supplement 1
Overt changes of mind for the other five datasets (labeled on panels).
Displayed as in Figure 1E,F.
Figure 2 with 2 supplements
Firing rates and decoding.
Blue represents eventual leftward reaches, red indicates eventual rightward reaches. (A and B) Responses of example units, dataset J2-S. Thick traces, mean; thin traces, s.e.m, Maze, maze onset. Time in ms. Selectivity for left vs right movements was common, and responses were almost always similar for forced and free reaches (A). Less commonly, forced and free evoked somewhat different responses (B). (C) Schematic of decoder. Each dot represents neural state in a window of time on a single trial. (D–F) Decoded choice plots for forced-choice trials, generated by leave-one-out cross-validation. Percentages show fraction correct classification. Datasets J2-T (D), J2-S (E), N1-S (F). Small dots are at last decoded time point. (G–I) Decoded choice plots for free-choice trials (saturated colors) with forced-choice trials shown for context (faded colors). Datasets same as D–F. (J–L) Cross-validated decoded choice at final point for forced-choice trials. Datasets same as D–F.
Figure 2—figure supplement 1
PSTHs for more example units.
Both the preparatory epoch and the peri-movement epoch are shown. For display, we selected units that attained different firing rates for right vs left during preparation and which reflected the diversity of responses observed. (A–C) Units from J1-T. (D–F) Units from J1-S. (G–I) Units from J2-T. (J–L) Units from J2-S. (M–O) Units from N1-S. Units shown in panels J and K are the same units from Figure 2A,B. MAZE, target/maze onset; MOVE, movement onset.
Figure 2—figure supplement 2
Decoding forced and free choices.
These plots are like Figure 2D–I, for the remaining datasets. (A) Cross-validated decoded choices for forced-choice trials. Column headers indicate dataset. (B) Decoded choices for free choice trials using the decoder trained on forced-choice trials (saturated colors). Forced-choice trials shown in faded colors for context. (C) Cross-validated decoded choices at final point for forced-choice trials.
Figure 3 with 1 supplement
Validating the decoder.
(A) Decoder performance vs time, forced-choice trials. (B) Maze icons illustrate free-to-forced trials. Saturated colors show the decoded choice for free-to-forced trials, which began as free but became forced when a barrier changed during the delay epoch. Faded colors show forced-choice trials for context. Large red dots indicate time of changes that made the rightward target the only available option; blue dots, leftward. Percentage indicates the fraction of trials for which the final decoded time point matched the monkey's choice. Dataset J2-S. (C) Same for dataset N3-S. (D) RT distributions for free-to-forced trials in which the barrier changed around the time of the Go cue. Black, trials where the monkey initially prepared a reach to the now-blocked side; gray, to the unchanging side. Data for monkey J pooled. Arrows, medians. (E) Same for monkey N.
Figure 3—figure supplement 1
Decoded choices for free-to-forced trials.
These plots are like Figure 3B,C, for the remaining datasets. (A) J1-T. (B) J1-S. (C) J2-T. (D) N1-S. (E) N2-S.
Figure 4 with 5 supplements
Covert changes of mind.
Maze icons indicate situation for adjacent decoded choice plots (saturated colors), with faded colors showing forced-choice trials for context. (A and B) Apparent ‘vacillation’ on free choice trials. Datasets J1-S (A) and J2-S (B). (C and D) Encouraged switch trials, which began as forced but became free when a barrier changed mid-delay. Red dots on traces indicate time of changes that made the rightward target more attractive; blue dots, leftward more attractive. J2-T. (C) Trials where the monkey chose the newly-available target. (D) Trials where the monkey chose the always-available target. (E and F) Biasing change trials. These trials were free throughout, but the difficulty of one side changed mid-delay. (E) Trials where the monkey reached to the target that was initially more difficult, and thus likely changed his mind. (F) Trials where the monkey reached to the target that was initially easier, and thus likely retained his initial decision. Dataset J1-S. (G and H) Distance from mean ‘baseline state’ (−300 to −40 ms from maze onset) for different trials and time epochs: during baseline (gray), during free-choice vacillations around the time of the change in the decoded choice (filled pink), during biasing-change trials around the time of the change in the decoded choice (filled green), during all times for free (hollow pink) or biasing-change (hollow green) trials. J2-S (G) and N1-S (H). (I) Probability that a trial of the given type exhibited a neural change of mind (a large change in the decoded choice during the delay period). See ‘Results’ for details. Forced vs free and untaken vs taken switch, p < 10−9; unlikely vs likely change, p < 10−4 (χ2 2 × 2 contingency test). Error bars show Wilson binomial confidence intervals equivalent to 1 s.e.m.
Figure 4—figure supplement 1
Free choice vacillation.
All datasets contained free-choice trials exhibiting large, late changes in the decoded choice, presumably indicating spontaneous changes of mind. (A) Algorithmically-identified vacillation trials (‘Materials and methods’) shown in saturated colors; forced-choice trials shown in faded colors for context. Data from monkey J. Dataset identity indicated by column header. (B) All free-choice trials not included in (A) shown in saturated colors; faded colors same as in (A). (C) Same as (A) for monkey N. (D) Same as (B) for monkey N.
Figure 4—figure supplement 2
Example vacillation trials with choice decoded using PMd and M1 separately.
Each row represents a free-choice trial (saturated-color trace) that was identified as containing a vacillation using both brain areas. Faded colors show forced-choice trials for context. The first column shows the decoded choice over time using both brain areas; the second column shows the decoded choice over time using PMd alone; the third column shows the decoded choice over time using M1 alone. (A–C) Trials in which vacillations appear similar using all three decoders. (D) Trial in which the M1 decoder disagreed with the combined and PMd decoders.
Figure 4—figure supplement 3
Decoded choices for taken switch trials (left column) and untaken switch trials (right column).
These plots are like Figure 4C,D, for the remaining datasets. (A and B) J1-T. (C and D) J1-S. (E and F) J2-S. (G and H) N1-S. (I and J) N2-S. (K and L) N3-S.
Figure 4—figure supplement 4
Biasing changes in the barriers encouraged changes of mind.
These plots are like Figure 4E,F, for the remaining datasets. Left column, trials where the monkey reached toward the target that was initially more difficult, and thus changes of mind were likely; right column, trials where changes of mind were unlikely. (A and B) J1-T. (C and D) J2-T. (E and F) J2-S. (G and H) N1-S. (I and J) N3-S.
Figure 4—figure supplement 5
Neural states during changes of mind typically did not resemble the baseline state.
These plots are like Figure 4G,H, for the remaining datasets. (A) J1-T. (B) J1-S. (C) J2-T. (D) N2-S. (E) N3-S.
Figure 5 with 2 supplements
Hesitation and indecision.
(A) RT distribution for forced-choice trials (gray) and encouraged switch trials in which the monkey reached to the always-available target (black). Arrows indicate medians; difference, 21 ms, p = 0.014, Mann–Whitney U test. Dataset J1-T. (B) Decoded choice for the untaken switch trials from (A) with RTs >450 ms (saturated colors), with forced-choice trials for context (faded colors). Red dots, time of barrier change. (C) RT distributions for non-delayed forced (gray) and free (black) trials. Arrows indicate medians; difference, 56 ms, p < 0.001, Mann–Whitney U test. Dataset J2-S. (D) Strength of decoded choice in direction of eventual reach (positive toward reach) at 100 ms after maze onset, vs RT for non-delayed free-choice trials. One point per trial; line indicates regression fit, dashed lines show 95% CI of fit.
Figure 5—figure supplement 1
Decoded choice is undisturbed on slow-RT, untaken-switch trials.
These plots are the same as Figure 5A,B, for the remaining datasets. (A and B) J1-S. (C and D) J2-T. (E and F) J2-S. (G and H) N1-S. (I and J) N2-S. (K and L) N3-S.
Figure 5—figure supplement 2
Slow-RT free-choice trials were rare in other datasets (J2-S shown in Figure 5C,D).
All trials shown had no instructed delay. Left column, RT histograms; right column, RT vs strength of decoded choice in direction of eventual reach. (A and B) J1-T. (C and D) J1-S. Free choices for this dataset were significantly slower than forced choices (p = 0.011), but this difference was small. (E and F) J2-T. (G and H) N1-S. (I and J) N2-S. (K and L) N3-S.
Author response image 1
This figure shows how the decoder depends on the units (top two rows) and the GPFA dimensions (bottom row) for dataset J1-T. We allowed a bias term for the power law fits; without the bias term, the double-exponential was clearly the better fit, and thus is the distribution mentioned in the manuscript.
Videos
Video 1
Two example free choice vacillation trials.
The left panel shows the display that the monkey saw, plus descriptive text. The right panel shows the decoded choice trajectory for the same trial (black), and the decoded choice trajectories for forced choice trials for context (blue for left, red for right). Video is presented at 1/4 real-time speed.
Video 2
Two example ‘taken-switch’ trials.
These trials began as forced then became free, and the monkey reached to the newly-accessible target. A red or blue dot appears on the decoded choice trace at the time of the barrier change (dot color indicates which side the change favored). Video is presented at 1/4 real-time speed.
Video 3
Two example ‘likely change’ trials.
These trials began with one side hard and the other easy, then a change made the initially easy side hard or the initially hard side easy. The monkey reached to the target that was not initially easy. Video is presented at 1/4 real-time speed.
Tables
Table 1
Choice probabilities
| p(left) easy|easy | p(left) hard|hard | p(left) easy|hard | p(left) hard|easy | Δp for left-biasing change | Δp for right-biasing change | |
|---|---|---|---|---|---|---|
| J1-T | 0.66 | 0.02 | 0.95 | 0.01 | 27% | −38% |
| J1-S | 0.16 | 0.57 | 0.66 | 0.17 | 32% | −19% |
| J2-T | 0.92 | 0.00 | 0.92 | 0.00 | 20% | −44% |
| J2-S | 0.23 | 0.99 | 1.00 | 0.06 | 45% | −38% |
| N1-S | 0.98 | 1.00 | 1.00 | 0.20 | 37% | −37% |
| N2-S | 0.79 | 0.83 | 0.88 | 0.70 | −4%* | −15% |
| N3-S | 0.79 | 0.54 | 0.79 | 0.45 | 18% | −16% |
-
The first four columns show the probability that the monkey chose the leftward target given the particular barrier configuration. ‘Easy|hard’ means that the barrier configuration was easy for the left and hard for the right. Trials included for these columns had no barrier changes. The Δp values show the change in p(left) when a trial presented a free choice throughout, but the difficulty of a key barrier changed during preparation (from 100 ms after maze onset to 50 ms after Go). These are separated by whether the change favored the left target (the left barrier became easier or the right harder) or the right target. Because dataset N2-S did not have consistent behavioral effects on trials with a change in barrier difficulty (starred entry), N2-S was not used for one analysis of biasing change trials.
Table 2
Decoding using PMd and M1 separately, all trials
| Forced-choice decoder performance (leave-one-out) | Decoder agreement | |||||
|---|---|---|---|---|---|---|
| Combined | PMd | M1 | PMd-combined | M1-combined | PMd-M1 | |
| J1-T | 0.99 | 0.99 | 0.90 | 0.92 | 0.85 | 0.80 |
| J1-S | 1.00 | 1.00 | 0.80 | 0.93 | 0.86 | 0.81 |
| J2-T | 0.98 | 0.97 | 0.80 | 0.96 | 0.87 | 0.86 |
| J2-S | 1.00 | 1.00 | 0.85 | 0.98 | 0.84 | 0.83 |
| N1-S | 0.90 | 0.84 | 0.95 | 0.93 | 0.72 | 0.67 |
-
Statistics for decoding choice from PMd and M1 data separately. Decoder performance refers to the fraction of trials for which the final decoded point agreed with the target that the animal reached to. This was assessed on forced-choice trials with delays of at least 300 ms, using leave-one-out cross-validation. Decoder agreement refers to the mean fraction of time points per trial for which the decoded choice was the same using the two datasets indicated. For this statistic, all successful trials with delay periods of at least 300 ms were included. ‘Combined’ refers to the decoder trained using both PMd and M1 data.
Table 3
Decoding using PMd and M1 separately, free choice trials
| Free-choice decoder performance | Decoder agreement | |||||
|---|---|---|---|---|---|---|
| Combined | PMd | M1 | PMd-combined | M1-combined | PMd-M1 | |
| J1-T | 0.94 | 0.86 | 0.77 | 0.90 | 0.83 | 0.77 |
| J1-S | 0.96 | 0.94 | 0.89 | 0.86 | 0.82 | 0.73 |
| J2-T | 0.97 | 0.96 | 0.82 | 0.96 | 0.82 | 0.80 |
| J2-S | 0.95 | 0.95 | 0.86 | 0.98 | 0.78 | 0.78 |
| N1-S | 0.88 | 0.82 | 0.79 | 0.91 | 0.72 | 0.67 |
-
Same as Table 2, but only considering free-choice trials with delay periods of at least 300 ms. The decoders were trained using forced-choice trials.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex
eLife 4:e04677.
https://doi.org/10.7554/eLife.04677
