Cognitive sensorimotor neuroscience investigates how the brain processes sensory information, develops an action plan, and ultimately performs a corresponding action. Experimental setups with non-human primates typically make use of physical restraint, such as a primate chair, to control for spatial parameters like head position, gaze direction, and body and arm posture. This approach led to numerous important insights into neural correlates of visually guided hand and arm movements. The frontoparietal reach network, in particular, including the posterior parietal cortex, premotor cortex, and motor cortex, has been studied in terms of force encoding (Cheney and Fetz, 1980), direction encoding (Georgopoulos et al., 1986), spatial reference frames of reach goal encoding (Batista et al., 1999; Buneo et al., 2002; Kuang et al., 2016; Pesaran et al., 2006), context integration (Gail and Andersen, 2006; Martínez-Vázquez and Gail, 2018; Niebergall et al., 2011; Westendorff et al., 2010), obstacle avoidance (Kaufman et al., 2013; Mulliken et al., 2008), bimanual coordination (Donchin et al., 1998; Mooshagian et al., 2018), eye-hand coordination (Hwang et al., 2012; Mooshagian and Snyder, 2018; Sayegh et al., 2017; Wong et al., 2016), and decision making (Christopoulos et al., 2015; Cisek, 2012; Klaes et al., 2011; Suriya-Arunroj and Gail, 2019). Because of the physical restraint, the scope of previous studies was mostly limited to hand or arm movements, and those were restricted to the immediately reachable space. Well-controlled planning and execution of spatially and temporally structured goal-directed movements in larger workspaces, including reach goals beyond immediate reach, could not be investigated in monkeys.

Neuropsychological and neurophysiological evidence suggest that frontoparietal areas encode the space near the body differently from the space far from the body (see Farnè et al., 2016 for review). Visuospatial neglect can be restricted to the near or far space as shown by patients with large-scale lesions comprising also parietal cortex (Halligan and Marshall, 1991; Vuilleumier et al., 1998) and transcranial magnetic stimulation over the parietal cortex (Bjoertomt et al., 2002). Bimodal neurons in premotor cortex and the posterior parietal cortex of non-human primates respond to visual and somatosensory stimulation, with visual receptive fields being congruent with somatosensory receptive fields and thereby covering the space near the body (Colby and Goldberg, 1999; Graziano et al., 1997; Rizzolatti et al., 1981; Rizzolatti et al., 1997). In addition, mirror neurons in the ventral premotor cortex can respond differently to an observed reach if the reach goal is within its own reach or not. (Bonini et al., 2014; Caggiano et al., 2009). These findings indicate that encoding of bimodal sensory information and information about observed actions seems to be dependent on one’s own body boundaries. Moreover, those findings suggest that premotor and parietal cortex are affected by this distinction. The frontoparietal network encodes motor goals within immediate reach, but it is unclear if this also holds true for motor goals beyond immediate reach. Because of the physical restraint of conventional setups, it has not been possible to investigate naturalistic goal-directed movements that require the monkey to walk towards targets at variable positions and, thus, to investigate how monkeys plan to acquire a reach goal beyond the immediately reachable space.

In conventional experiments, tethered connections prohibit recording from freely moving primates, at least in the case of larger species such as macaques. Tethered recordings in freely moving smaller primate species, such as squirrel monkeys (Ludvig et al., 2004) or marmosets (Courellis et al., 2019; Nummela et al., 2017) have been demonstrated. One study also showed tethered recordings in Japanese macaques; however these were in an environment with no obstacles and with low channel count (Hazama and Tamura, 2019). Using wireless recording technology in combination with chronically implanted arrays, recent studies achieved recordings of single unit activity in nonhuman primates investigating vocalization (Hage and Jurgens, 2006; Roy and Wang, 2012), simple uninstructed behavior (Schwarz et al., 2014; Talakoub et al., 2019), treadmill locomotion (Capogrosso et al., 2016; Foster et al., 2014; Schwarz et al., 2014; Yin et al., 2014), chair-seated translocation (Rajangam et al., 2016), sleep (Yin et al., 2014; Zhou et al., 2019), and simple movements to a food source (Capogrosso et al., 2016; Chestek et al., 2009; Fernandez-Leon et al., 2015; Hazama and Tamura, 2019; Schwarz et al., 2014; Shahidi et al., 2019). An alternative to wireless transmission can be data logging for which the data are stored separately from behavioral data on the headstage (Zanos et al., 2011). This has been used in investigations of simple uninstructed behavior and sleep (Jackson et al., 2006; Jackson et al., 2007; Xu et al., 2019). However, none of the experiments with neural recordings in unrestrained monkeys presented an experimental environment that instructs temporally and spatially precise movement behavior (Supplementary file 1). To study goal-directed motor planning and spatial encoding of motor goals, we developed the Reach Cage in which we can instruct precise movement start times and multiple distributed movement goals independent from the food source.

Here, we present an experimental environment, the Reach Cage, which is equipped with a visuo-haptic interaction system (MaCaQuE) and allows investigating movement planning and goal-directed movements of unrestrained rhesus monkeys while recording and analyzing in real-time cortical single-unit activity. We trained monkeys to perform controlled memory-guided reach movements with instructed delay to targets within and beyond the immediately reachable space. Using an open source markerless video-based motion capture software (Mathis et al., 2018), we measured 3-dimensional head, shoulder, elbow, and wrist trajectories. We used wireless recording technology to extract single unit activity in real-time from three cortical areas (parietal reach region PRR, dorsal premotor cortex PMd, and primary motor cortex M1) at a bandwidth suitable for brain-machine interface (BMI) applications. We show that the Reach Cage is suitable for sensorimotor neuroscience with physically unrestrained rhesus monkeys providing a richer set of motor tasks, including walk-and-reach movements. With the Reach Cage we were able to study motor goal encoding beyond the immediate reach and during ongoing walking movements. We show that PRR and PMd, but not M1, already contain target location information of far-located walk-and-reach targets during the planning period before and during the walk-and-reach movement.

We developed the Reach Cage to expand studies of visual guided reaching movements to larger workspaces and study movements of rhesus monkeys performing structured whole-body movement tasks while being physically unrestrained. We report on quantitative assessment of the animals’ behavior in the Reach Cage, and neuroscientific analysis of walk-and-reach goal encoding. The timing of the monkeys’ reaching behavior can be precisely controlled and measured with the touch and release times of our touch-sensitive cage-mounted targets (1^st section). Additionally, multi-joint 3-dimensional reach kinematics can be measured directly with the video-based motion capture system (2^nd section). We will show that high channel count wireless neural recording is possible in the Reach Cage and report on single-unit activity during such structured task performance (3^rd section). Finally, we demonstrate the suitability of the Reach Cage for studying motor goal encoding beyond the immediate reach and show that premotor and parietal cortical activity contain information about far-located walk-and-reach targets position during movement planning (4^th section).

Real-time control of instructed behavior in physical unrestrained rhesus monkeys in the Reach Cage

The core element of our newly developed Reach Cage (Figure 1) is the Macaque Cage Query Extension (MaCaQuE). Using this interaction device, we were able to train two fully unrestrained rhesus monkeys to conduct spatially and temporally well-structured memory-guided reaches, a behavioral task common to sensorimotor neuroscience in primates. Here we report the technical details of MaCaQuE and its use with physically unrestrained rhesus monkeys; however, we also used MaCaQuE successfully in a study with human participants (Berger et al., 2019).

Figure 1

Download asset Open asset

Both animals learned within a single first session that touching a target presented on a MaCaQuE Cue and Target box (MCT, Figure 1B) leads to a liquid reward. The computer-controlled precise timing and dosage of reward (Figure 1C) meant that we could employ MaCaQuE for positive reinforcement training (PRT) to teach both animals a memory-guided target acquisition task with instructed delay (see Materials and methods). Unlike chair-based setups, MaCaQuE allows for target placement beyond the immediate reach of the monkeys (Figure 1D,Video 1) Monkey K performed the final version of the walk-and-reach task (Figure 2A/B) with 77% correct trials on average (s.d. 9%, 19 sessions) with up to 412 correct trials per session (mean 208, s.d. 93). The sessions lasted on average 40 min (s.d. 15 min). Monkey L performed the final version of the task with 55% correct trials on average (s.d. 5%, 10 sessions) with up to 326 correct trials per session (mean 219, s.d. 55). Sessions lasted on average 65 min (s.d. 15 min). The majority of errors resulted from premature release of the start buttons prior to the go cue. Trials with properly timed movement initiation were 92% correct in monkey K and 78% correct in monkey L.

Video 1

Download asset

Figure 2

Download asset Open asset

While the animals were not physically restricted to a specific posture, the strict timing of the task encouraged them to optimize their behavior. As the MaCaQuE system makes information about MCT touches and releases available with minimal delay (<20 ms), it is possible to enforce an exact timing of the movements when solving a reaching task in the Reach Cage. Figure 2C shows the distribution of button release times and movement times towards near and far targets for the task (monkey K/L: 19/10 sessions, 3956/2194 correct trials). As a whole-body translocation is required to approach far targets, movement times were longer than for near targets in both monkeys (t-test, p<0.001). Movement time distributions were narrow (s.d. ≤ 76 ms), indicating that the monkeys optimized their behavior for consistent target acquisition. Button release time indicates the onset of the hand movement, not necessarily the whole-body movement. In monkey K, the button release times were higher for far compared to near targets (t-test, p<0.001). In contrast, button release times in monkey L were lower for far compared to near targets (p<0.001), reflecting a different behavioral strategy for movement onset (monkey K was sitting during the delay period whereas monkey L was standing).

The behavioral results as directly measured with MaCaQuE via the proximity sensors of the MCTs demonstrate that the Reach Cage is suitable to train animals on goal-directed reaching tasks with target positions not being constrained by the immediately reachable space of the animal. The temporally well-structured task performance at the same time allows behavioral and neurophysiological analyses as applied in more conventional settings.

Time-continuous 3-dimensional arm kinematics during walk-and-reach behavior

As we do not impose physical restraint, the monkeys have more freedom to move than in conventional setups. This allows for testing new behavioral paradigms such as the walk-and-reach task but also provides more freedom in how to solve the task. We used DeepLabCut (Mathis et al., 2018) for video-based motion capture and analyzed kinematics and their variability.

We measured the 3-dimensional posture of the reaching arm during the reach and walk-and-reach behavior of 2/3 sessions with a total of 469/872 successful trials in monkey K/L. Specifically, we tracked the monkeys’ headcap, left shoulder, elbow, and wrist (Videos 2–5). Figure 3A shows the side-view of the body part positions for each trial and video frame between 100 ms before button release and 100 ms after target acquisition for the reach (red) and walk-and-reach (blue) movements to the mid-left target.

Video 2

Download asset

Video 3

Download asset

Video 4

Download asset

Video 5

Download asset

Figure 3

Download asset Open asset

Within each animal, reach kinematics were highly consistent from trial to trial and from session to session. To quantify the variability in arm posture, we calculated for each target and marker separately and at corresponding times the Euclidean distance between the single-trial trajectories and the across sessions trial-averaged trajectory. Figure 3B shows the distributions of Euclidean distance averaged over time for each trial, marker, and monkey. The highest variability was seen in the wrist during walk-and-reach movements with a median of 37/46 mm and 0.75-quartile of 50/67 mm for monkeys K and L, respectively. Within a session these median deviations are 1–6 mm smaller. As a reference, the transparent front plate of the targets has a diameter of 75 mm. The center-to-center distance between neighboring targets is around 130 mm (near; shown as dashed line in the plot) and 210 mm (far). This shows that even across sessions, the arm posture during the movements towards the same target at a given time varied only by a few centimeters.

The movement patterns between monkey K (left) and monkey L (right) were different. Figure 3C shows the trial-averaged arm posture for each time point during the reach and walk-and-reach movements. Monkey K was sitting during the memory period and then used its left forelimb for walking and reaching. Monkey L was standing during the memory period and walked bipedally to the far targets. Both animals used this strategy consistently in all trials.

The kinematic analyses demonstrate that the animals not only complied with the spatial and temporal task requirements in terms of starting and endpoint acquisition but also adopted reliable repetitive behavior in terms of overall reach kinematics despite the lack of physical restraint. The animals used different behavioral strategies. However, the video-based motion capture allowed us to quantify the arm and head kinematics.

Multi-channel single unit activity can be recorded in the Reach Cage using wireless technology

The Reach Cage provides an adequate setting for studying well-isolated single neuron activity from multiple areas of the cerebral cortex of monkeys during movement planning and execution of goal-directed behavior in minimally constrained settings. We here provide simultaneous recordings from three different sensorimotor areas, including non-surface areas inside sulci, during the goal-directed memory-guided walk-and-reach task.

We chronically implanted a total of 192 electrodes in primary motor cortex (M1), dorsal premotor cortex (PMd), and the parietal reach region (PRR) in the posterior parietal cortex of both monkeys using six 32-channel floating microwire arrays (FMA) with various lengths (see Materials and methods). We recorded broadband (30 ksps per channel) neural data from all arrays simultaneously (i.e. up to 192 channels) while the monkeys performed the walk-and-reach task (Figure 4). The animals moved through the cage wearing the wireless electronics and protective cap without banging the implant into the cage equipment and performed the behavioral task as without the wireless gear.

Figure 4 with 1 supplement see all

Download asset Open asset

Figure 4—source data 1

Data loss rate differences across targets for all trials.

Two-way ANOVA table; ‘*” indicates significance with p<0.05.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig4-data1-v3.xlsx

Download elife-51322-fig4-data1-v3.xlsx

Figure 4—source data 2

Data loss rate differences across targets for trials below 5% data loss.

Two-way ANOVA table; ‘*” indicates significance with p<0.05.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig4-data2-v3.xlsx

Download elife-51322-fig4-data2-v3.xlsx

We recorded in monkey K/L 2/10 sessions from all six arrays simultaneously using two 96-channel wireless headstages. Our custom-designed implants can house both headstages and protect them and the array connectors against dirt and physical impact. The implants are designed to be used with different commercially available wireless systems, with the 2 × 96 channel digital systems presented here or with a 31- or 127-channel analog wireless system, dependent on the need of the experiment. Implant development and methodological details will be discussed below (Material and methods).

The wireless signal transmission was stable during walking movements. To quantify the stability, we calculated the rate of data loss from lost connection to the wireless system. We checked for each time point if either of the two headstages did not receive data. As a conservative measure, we only considered correctly performed trials, as in these trials it is guaranteed that the animal moved the full stretch from start to goal. The best sessions showed loss rates of 3.18%/1.03% of all time bins for monkey K/L, and worst sessions of 6.59%/6.34%, respectively. On average across sessions and monkeys, the loss rate was 3.32% (s.d. 1.7%). Data loss was spread over all targets with a slight spatial bias (Figure 4—figure supplement 1A and Figure 4—source data 1, 2 -way ANOVA position F(3, 2657)=3.48, p=0.015; position x distance F(3, 2657)=4.81, p=0.002). The spatial bias was introduced by trials with high data loss rates. When removing trials with a loss rate of above 5% there was no significant spatial bias anymore (Figure 4—figure supplement 1B and Figure 4—source data 2, 2-way ANOVA position F(3, 2657)=0.88, p=0.45; position x distance F(3, 2657)=2.36, p=0.07). From here on, we only consider correct trials with a loss rate of less than 5%. Note, walk-and-reach trials showed different loss rates from reach trials (F(3, 2657)=279.96, p<0.001); however, this does not influence further results that focus on movement direction of reach or walk-and-reach movements separately.

The wireless signal quality was stable during walking movements and allowed us to isolate single- and multi-unit activity during the walk-and-reach task. Figure 4 shows four example neurons from the frontoparietal reach network of both monkeys while performing the task. Trial-averaged spike densities (top left) show that units were modulated by task condition. All four example neurons are significantly modulated by target distance, left-to-right target position, time during the trial, and interactions of distance x position and distance x time (ANOVA p<0.05). Units A and C are mostly active during the memory period while units B and D are active during memory period and movement. Waveforms of the isolated example neurons are shown on the top right of each panel. Unfiltered broadband data of one near (red) and one far (blue) example trial are shown below. Spiking activity can be identified in the broadband signal also during the reach and walk-and-reach movement. We performed the same ANOVA for the activity in each channel of all 12 recorded sessions. Three sessions revealed task-responsive activity on all 192 channels, that is showed at least one effect in distance, position, time, or one of the interactions; across all sessions the mean number of task-responsive channels was 189 (s.d. five channels). Up to 179 channels were position-responsive, that is showing at least one effect in position or one of the interactions (mean: 162, s.d. 17 channels).

In summary, the Reach Cage proved to be suitable for addressing neuroscientific questions based on single and multi-unit recordings. Broadband wireless neural signals showed excellent spike isolation and modulation of spike frequency correlated with behavioral events.

Premotor and parietal cortex encode movement goals beyond immediate reach

The Reach Cage allows us for the first time to test the spatial encoding of movement goals at larger distances to the animal. We wanted to know whether the frontoparietal reach network encodes motor goals only within the immediate reach or also beyond. For this, we computed separately in near and far trials the performance for decoding goal direction (left vs. right) with a support vector machine (SVM) decoder based on multi-unit firing rates.

We analyzed the session with the highest number of trials for each animal to avoid biasing our results by reusing repeated measures of the same neurons on channels that showed stable signals across multiple sessions. Figure 5A shows the movement paths of the wrist (top) and head (bottom) of the animals for the reach (orange) and walk-and-reach (blue) behavior towards the different targets. Figure 5B (line plots) shows 20-fold cross validation of decoding accuracy in 300 ms time windows at 100 ms time steps. To test if there is reach goal encoding during movement planning prior to onset of movement, we analyzed the time window during the memory period starting 100 ms after target cue offset. To test if there is reach goal encoding during reaching (near) and during ongoing walking-and-reaching (far), we analyzed the 300 ms immediately before target acquisition. We performed a one-tailed permutation test to determine whether decoding accuracy is significantly above chance. In PMd and PRR, decoding is significant for both memory and movement period for reach and walk-and-reach movements (Figure 5—source data 1). In M1, decoding accuracy only reached significance for walk-and-reach movements in monkey L during the movement period. We then analyzed the population average of the firing rate modulation between the preferred and anti-preferred direction, again left vs. right (Figure 5B bar plots). We performed a one-tailed permutation test to determine significance. As in the decoding analysis, PMd and PRR show significant firing rate modulation for both memory and movement period for reach and walk-and-reach movements (Figure 5—source data 2). Here, M1 shows significant firing rate modulation during the walk-and-reach movement period for both monkeys but, as for the decoding analysis, not during the memory period.

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 5—source data 1

Test for significant SVM decoding accuracy.

One-tailed permutation test; ‘*” indicates Bonferroni corrected significance with p<0.0042.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig5-data1-v3.xlsx

Download elife-51322-fig5-data1-v3.xlsx

Figure 5—source data 2

Test for significant firing rate modulation.

One-tailed permutation test; ‘*” indicates Bonferroni corrected significance with p<0.0042.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig5-data2-v3.xlsx

Download elife-51322-fig5-data2-v3.xlsx

Figure 5—source data 3

Test for change in decoding accuracy between trials with and without passage.

Two-tailed permutation test on SVM decoding accuracy differences between with and without passage; ‘*” indicates Bonferroni corrected significance with p<0.0083.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig5-data3-v3.xlsx

Download elife-51322-fig5-data3-v3.xlsx

Figure 5—source data 4

Test for change in firing rate modulation between trials with and without passage.

Two-tailed permutation test; ‘*” indicates Bonferroni corrected significance with p<0.0083.

https://cdn.elifesciences.org/articles/51322/elife-51322-fig5-data4-v3.xlsx

Download elife-51322-fig5-data4-v3.xlsx

From the horizontal fanning out of the unconstrained movement patterns (Figure 5A), it became evident that both animals directed their walking movement towards the goal from early on in the movement. To confirm that the motor goal information decodable from PMd and PRR correlates with the reach goal location rather than initial walking movement direction, we introduced a passage in the middle of the walk-and-reach path (a transparent divider between near and far targets with a narrow opening cut out). Although movement trajectories for the different motor goal locations collapsed onto very similar initial walking directions because of the passage (Figure 5—figure supplement 1A), the decoding accuracy and firing rate modulation was not affected by this measure, that is was independent of the movement path (Figure 5 – figure supplement 1B and Figure 5—source data 3 and 4).

Taken together, the Reach Cage environment allows us to study sensorimotor neuroscience questions within an unrestrained spatial setting. Here, we show that target location information is present in premotor and parietal cortex of far-located targets beyond the immediate reach.

We introduced the Reach Cage as a novel experimental environment for sensorimotor neuroscience with physically unrestrained rhesus monkeys. As a core interactive element, we developed MaCaQuE, a new experimental control system for sensorimotor tasks in cage environments. We trained two monkeys to conduct spatially and temporally structured memory-guided reach tasks that required them to reach to targets near or far from them with a walk-and-reach movement. With MaCaQuE, we could measure button release and movement times in response to visual cues with the same if not higher temporal precision as in touch screen experiments. Using markerless video-based motion capture, we could track 3-dimensional head and multi-joint arm kinematics for reach and walk-and-reach movements. Trajectories had low spatial variability over trials, showing that monkeys perform instructed movement consistently even when no physical restraint is applied. Variations in movement pattern between task conditions or monkeys could well be quantified in detail with this motion capture approach. In parallel, we wirelessly recorded broadband neural signals of 192 channels from three brain areas (M1, PMd, and PRR) simultaneously, an approach suitable for BMI applications. Isolated single-neuron activities were clearly modulated by the task events and encoded information about the location of immediate reach targets and also of remote walk-and-reach targets. Moreover, we could identify walk-and-reach target location information in premotor and parietal cortex, but not motor cortex, during movement and even during the memory period before the movement. This suggests that premotor and parietal cortex encodes motor goals beyond immediate reach. With our Reach Cage approach, we provide an experimental environment that allows testing fully unrestrained monkeys on spatially and temporally controlled behavior. With wireless intra-cortical recordings and markerless motion capture experimental spatial configurations are possible that are not restricted to the vicinity of the animals but allow studying complex full-body movement patterns.

All data (schematics, soft- and hardware documentation) for constructing the MaCaQuE or equivalent systems is made available via GitHub: https://github.com/sensorimotorgroupdpz/MaCaQuE.

1. 1. Berger M
   2. Gail A

   (2020) Zenodo

   sensorimotorgroupdpz/MaCaQuE.

   https://doi.org/10.5281/zenodo.3685793

1. 1. Aflalo T
   2. Kellis S
   3. Klaes C
   4. Lee B
   5. Shi Y
   6. Pejsa K
   7. Shanfield K
   8. Hayes-Jackson S
   9. Aisen M
   10. Heck C
   11. Liu C
   12. Andersen RA

   (2015) Neurophysiology decoding motor imagery from the posterior parietal cortex of a tetraplegic human

   Science 348:906–910.

   https://doi.org/10.1126/science.aaa5417

   • PubMed
   • Google Scholar
2. Preprint

   1. Bala PC
   2. Eisenreich BR
   3. Bum S
   4. Yoo M
   5. Hayden BY
   6. Park HS
   7. Zimmermann J

   (2020) OpenMonkeyStudio : automated markerless pose estimation in freely moving macaques

   bioRxiv.

   https://doi.org/10.1101/2020.01.31.928861

   • Google Scholar
3. 1. Ballesta S
   2. Reymond G
   3. Pozzobon M
   4. Duhamel JR

   (2014) A real-time 3D video tracking system for monitoring primate groups

   Journal of Neuroscience Methods 234:147–152.

   https://doi.org/10.1016/j.jneumeth.2014.05.022

   • PubMed
   • Google Scholar
4. 1. Bansal AK
   2. Truccolo W
   3. Vargas-Irwin CE
   4. Donoghue JP

   (2012) Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials

   Journal of Neurophysiology 107:1337–1355.

   https://doi.org/10.1152/jn.00781.2011

   • PubMed
   • Google Scholar
5. 1. Batista AP
   2. Buneo CA
   3. Snyder LH
   4. Andersen RA

   (1999) Reach plans in eye-centered coordinates

   Science 285:257–260.

   https://doi.org/10.1126/science.285.5425.257

   • PubMed
   • Google Scholar
6. 1. Berger M
   2. Calapai A
   3. Stephan V
   4. Niessing M
   5. Burchardt L
   6. Gail A
   7. Treue S

   (2018) Standardized automated training of rhesus monkeys for neuroscience research in their housing environment

   Journal of Neurophysiology 119:796–807.

   https://doi.org/10.1152/jn.00614.2017

   • PubMed
   • Google Scholar
7. 1. Berger M
   2. Neumann P
   3. Gail A

   (2019) Peri-hand space expands beyond reach in the context of walk-and-reach movements

   Scientific Reports 9:3013.

   https://doi.org/10.1038/s41598-019-39520-8

   • PubMed
   • Google Scholar
8. Software

   1. Berger M
   2. Gail A

   (2020) sensorimotorgroupdpz/MaCaQuE

   Zenodo.

   http://doi.org/10.5281/zenodo.3685793
9. 1. Berti A
   2. Frassinetti F

   (2000) When far becomes near: remapping of space by tool use

   Journal of Cognitive Neuroscience 12:415–420.

   https://doi.org/10.1162/089892900562237

   • PubMed
   • Google Scholar
10. 1. Bjoertomt O
    2. Cowey A
    3. Walsh V

    (2002) Spatial neglect in near and far space investigated by repetitive transcranial magnetic stimulation

    Brain 125:2012–2022.

    https://doi.org/10.1093/brain/awf211

    • PubMed
    • Google Scholar
11. 1. Blanke O
    2. Slater M
    3. Serino A

    (2015) Behavioral, neural, and computational principles of bodily Self-Consciousness

    Neuron 88:145–166.

    https://doi.org/10.1016/j.neuron.2015.09.029

    • PubMed
    • Google Scholar
12. 1. Bonini L
    2. Maranesi M
    3. Livi A
    4. Fogassi L
    5. Rizzolatti G

    (2014) Space-dependent representation of objects and other's action in monkey ventral premotor grasping neurons

    Journal of Neuroscience 34:4108–4119.

    https://doi.org/10.1523/JNEUROSCI.4187-13.2014

    • PubMed
    • Google Scholar
13. 1. Botvinick M
    2. Cohen J

    (1998) Rubber hands 'feel' touch that eyes see

    Nature 391:756.

    https://doi.org/10.1038/35784

    • PubMed
    • Google Scholar
14. 1. Bouton CE
    2. Shaikhouni A
    3. Annetta NV
    4. Bockbrader MA
    5. Friedenberg DA
    6. Nielson DM
    7. Sharma G
    8. Sederberg PB
    9. Glenn BC
    10. Mysiw WJ
    11. Morgan AG
    12. Deogaonkar M
    13. Rezai AR

    (2016) Restoring cortical control of functional movement in a human with quadriplegia

    Nature 533:247–250.

    https://doi.org/10.1038/nature17435

    • PubMed
    • Google Scholar
15. 1. Brozzoli C
    2. Pavani F
    3. Urquizar C
    4. Cardinali L
    5. Farnè A

    (2009) Grasping actions remap peripersonal space

    NeuroReport 20:913–917.

    https://doi.org/10.1097/WNR.0b013e32832c0b9b

    • PubMed
    • Google Scholar
16. 1. Brozzoli C
    2. Cardinali L
    3. Pavani F
    4. Farnè A

    (2010) Action-specific remapping of peripersonal space

    Neuropsychologia 48:796–802.

    https://doi.org/10.1016/j.neuropsychologia.2009.10.009

    • PubMed
    • Google Scholar
17. 1. Buneo CA
    2. Jarvis MR
    3. Batista AP
    4. Andersen RA

    (2002) Direct visuomotor transformations for reaching

    Nature 416:632–636.

    https://doi.org/10.1038/416632a

    • PubMed
    • Google Scholar
18. 1. Buneo CA
    2. Andersen RA

    (2012) Integration of target and hand position signals in the posterior parietal cortex: effects of workspace and hand vision

    Journal of Neurophysiology 108:187–199.

    https://doi.org/10.1152/jn.00137.2011

    • PubMed
    • Google Scholar
19. 1. Caggiano V
    2. Fogassi L
    3. Rizzolatti G
    4. Thier P
    5. Casile A

    (2009) Mirror neurons differentially encode the peripersonal and extrapersonal space of monkeys

    Science 324:403–406.

    https://doi.org/10.1126/science.1166818

    • PubMed
    • Google Scholar
20. 1. Calapai A
    2. Berger M
    3. Niessing M
    4. Heisig K
    5. Brockhausen R
    6. Treue S
    7. Gail A

    (2017) A cage-based training, cognitive testing and enrichment system optimized for rhesus macaques in neuroscience research

    Behavior Research Methods 49:35–45.

    https://doi.org/10.3758/s13428-016-0707-3

    • PubMed
    • Google Scholar
21. 1. Capogrosso M
    2. Milekovic T
    3. Borton D
    4. Wagner F
    5. Moraud EM
    6. Mignardot JB
    7. Buse N
    8. Gandar J
    9. Barraud Q
    10. Xing D
    11. Rey E
    12. Duis S
    13. Jianzhong Y
    14. Ko WK
    15. Li Q
    16. Detemple P
    17. Denison T
    18. Micera S
    19. Bezard E
    20. Bloch J
    21. Courtine G

    (2016) A brain-spine interface alleviating gait deficits after spinal cord injury in primates

    Nature 539:284–288.

    https://doi.org/10.1038/nature20118

    • PubMed
    • Google Scholar
22. 1. Carmena JM

    (2013) Advances in Neuroprosthetic learning and control

    PLOS Biology 11:e1001561.

    https://doi.org/10.1371/journal.pbio.1001561

    • PubMed
    • Google Scholar
23. Book

    1. Chen X
    2. Davis J

    (2000)

    Camera Placement Considering Occlusion for Robust Motion Capture

    Stanford University.

    • Google Scholar
24. 1. Cheney PD
    2. Fetz EE

    (1980) Functional classes of primate corticomotoneuronal cells and their relation to active force

    Journal of Neurophysiology 44:773–791.

    https://doi.org/10.1152/jn.1980.44.4.773

    • PubMed
    • Google Scholar
25. 1. Chestek CA
    2. Gilja V
    3. Nuyujukian P
    4. Kier RJ
    5. Solzbacher F
    6. Ryu SI
    7. Harrison RR
    8. Shenoy KV

    (2009) HermesC: low-power wireless neural recording system for freely moving primates

    IEEE Transactions on Neural Systems and Rehabilitation Engineering 17:330–338.

    https://doi.org/10.1109/TNSRE.2009.2023293

    • PubMed
    • Google Scholar
26. 1. Christopoulos VN
    2. Bonaiuto J
    3. Kagan I
    4. Andersen RA

    (2015) Inactivation of parietal reach region affects reaching but not saccade choices in internally guided decisions

    The Journal of Neuroscience 35:11719–11728.

    https://doi.org/10.1523/JNEUROSCI.1068-15.2015

    • PubMed
    • Google Scholar
27. 1. Cisek P

    (2012) Making decisions through a distributed consensus

    Current Opinion in Neurobiology 22:927–936.

    https://doi.org/10.1016/j.conb.2012.05.007

    • Google Scholar
28. 1. Colby CL
    2. Goldberg ME

    (1999) SPACE AND ATTENTION IN PARIETAL CORTEX

    Annual Review of Neuroscience 22:319–349.

    https://doi.org/10.1146/annurev.neuro.22.1.319

    • Google Scholar
29. 1. Collinger JL
    2. Wodlinger B
    3. Downey JE
    4. Wang W
    5. Tyler-Kabara EC
    6. Weber DJ
    7. McMorland AJC
    8. Velliste M
    9. Boninger ML
    10. Schwartz AB

    (2013) High-performance neuroprosthetic control by an individual with tetraplegia

    The Lancet 381:557–564.

    https://doi.org/10.1016/S0140-6736(12)61816-9

    • Google Scholar
30. 1. Courellis HS
    2. Nummela SU
    3. Metke M
    4. Diehl GW
    5. Bussell R
    6. Cauwenberghs G
    7. Miller CT

    (2019) Spatial encoding in primate Hippocampus during free navigation

    PLOS Biology 17:e3000546.

    https://doi.org/10.1371/journal.pbio.3000546

    • PubMed
    • Google Scholar
31. 1. Courtine G
    2. Roy RR
    3. Hodgson J
    4. McKay H
    5. Raven J
    6. Zhong H
    7. Yang H
    8. Tuszynski MH
    9. Edgerton VR

    (2005) Kinematic and EMG determinants in quadrupedal locomotion of a non-human primate (Rhesus)

    Journal of Neurophysiology 93:3127–3145.

    https://doi.org/10.1152/jn.01073.2004

    • PubMed
    • Google Scholar
32. 1. Crammond DJ
    2. Kalaska JF

    (1994) Modulation of preparatory neuronal activity in dorsal premotor cortex due to stimulus-response compatibility

    Journal of Neurophysiology 71:1281–1284.

    https://doi.org/10.1152/jn.1994.71.3.1281

    • Google Scholar
33. 1. Crammond DJ
    2. Kalaska JF

    (2000) Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity

    Journal of Neurophysiology 84:986–1005.

    https://doi.org/10.1152/jn.2000.84.2.986

    • PubMed
    • Google Scholar
34. 1. Dann B
    2. Michaels JA
    3. Schaffelhofer S
    4. Scherberger H

    (2016) Uniting functional network topology and oscillations in the fronto-parietal single unit network of behaving primates

    eLife 5:e15719.

    https://doi.org/10.7554/eLife.15719

    • PubMed
    • Google Scholar
35. 1. Donchin O
    2. Gribova A
    3. Steinberg O
    4. Bergman H
    5. Vaadia E

    (1998) Primary motor cortex is involved in bimanual coordination

    Nature 395:274–278.

    https://doi.org/10.1038/26220

    • PubMed
    • Google Scholar
36. 1. Farnè A
    2. Serino A
    3. van der Stoep N
    4. Spence C
    5. Di Luca M

    (2016) Depth: the forgotten dimension multisensory Research

    Psychology 29:1–32.

    https://doi.org/10.1163/22134808-00002525

    • Google Scholar
37. 1. Fernandez-Leon JA
    2. Parajuli A
    3. Franklin R
    4. Sorenson M
    5. Felleman DJ
    6. Hansen BJ
    7. Hu M
    8. Dragoi V

    (2015) A wireless transmission neural interface system for unconstrained non-human primates

    Journal of Neural Engineering 12:056005.

    https://doi.org/10.1088/1741-2560/12/5/056005

    • PubMed
    • Google Scholar
38. 1. Foster JD
    2. Nuyujukian P
    3. Freifeld O
    4. Gao H
    5. Walker R
    6. I Ryu S
    7. H Meng T
    8. Murmann B
    9. J Black M
    10. Shenoy KV

    (2014) A freely-moving monkey treadmill model

    Journal of Neural Engineering 11:046020.

    https://doi.org/10.1088/1741-2560/11/4/046020

    • PubMed
    • Google Scholar
39. 1. Gail A
    2. Andersen RA

    (2006) Neural dynamics in monkey parietal reach region reflect context-specific sensorimotor transformations

    Journal of Neuroscience 26:9376–9384.

    https://doi.org/10.1523/JNEUROSCI.1570-06.2006

    • PubMed
    • Google Scholar
40. 1. Georgopoulos AP
    2. Schwartz AB
    3. Kettner RE

    (1986) Neuronal population coding of movement direction

    Science 233:1416–1419.

    https://doi.org/10.1126/science.3749885

    • PubMed
    • Google Scholar
41. 1. Giglia G
    2. Pia L
    3. Folegatti A
    4. Puma A
    5. Fierro B
    6. Cosentino G
    7. Berti A
    8. Brighina F

    (2015) Far space remapping by tool use: a rTMS study over the right posterior parietal cortex

    Brain Stimulation 8:795–800.

    https://doi.org/10.1016/j.brs.2015.01.412

    • PubMed
    • Google Scholar
42. 1. Gilja V
    2. Chestek CA
    3. Nuyujukian P
    4. Foster J
    5. Shenoy KV

    (2010) Autonomous head-mounted electrophysiology systems for freely behaving primates

    Current Opinion in Neurobiology 20:676–686.

    https://doi.org/10.1016/j.conb.2010.06.007

    • PubMed
    • Google Scholar
43. 1. Gilja V
    2. Pandarinath C
    3. Blabe CH
    4. Nuyujukian P
    5. Simeral JD
    6. Sarma AA
    7. Sorice BL
    8. Perge JA
    9. Jarosiewicz B
    10. Hochberg LR
    11. Shenoy KV
    12. Henderson JM

    (2015) Clinical translation of a high-performance neural prosthesis

    Nature Medicine 21:1142–1145.

    https://doi.org/10.1038/nm.3953

    • PubMed
    • Google Scholar
44. 1. Graziano MS
    2. Hu XT
    3. Gross CG

    (1997) Visuospatial properties of ventral premotor cortex

    Journal of Neurophysiology 77:2268–2292.

    https://doi.org/10.1152/jn.1997.77.5.2268

    • PubMed
    • Google Scholar
45. 1. Graziano MS
    2. Cooke DF
    3. Taylor CS

    (2000) Coding the location of the arm by sight

    Science 290:1782–1786.

    https://doi.org/10.1126/science.290.5497.1782

    • PubMed
    • Google Scholar
46. 1. Hage SR
    2. Jurgens U

    (2006) On the role of the pontine brainstem in vocal pattern generation: a telemetric Single-Unit recording study in the squirrel monkey

    Journal of Neuroscience 26:7105–7115.

    https://doi.org/10.1523/JNEUROSCI.1024-06.2006

    • Google Scholar
47. 1. Halligan PW
    2. Marshall JC

    (1991) Left neglect for near but not far space in man

    Nature 350:498–500.

    https://doi.org/10.1038/350498a0

    • PubMed
    • Google Scholar
48. 1. Hauschild M
    2. Mulliken GH
    3. Fineman I
    4. Loeb GE
    5. Andersen RA

    (2012) Cognitive signals for brain-machine interfaces in posterior parietal cortex include continuous 3D trajectory commands

    PNAS 109:17075–17080.

    https://doi.org/10.1073/pnas.1215092109

    • PubMed
    • Google Scholar
49. 1. Hazama Y
    2. Tamura R

    (2019) Effects of self-locomotion on the activity of place cells in the Hippocampus of a freely behaving monkey

    Neuroscience Letters 701:32–37.

    https://doi.org/10.1016/j.neulet.2019.02.009

    • PubMed
    • Google Scholar
50. 1. Hochberg LR
    2. Bacher D
    3. Jarosiewicz B
    4. Masse NY
    5. Simeral JD
    6. Vogel J
    7. Haddadin S
    8. Liu J
    9. Cash SS
    10. van der Smagt P
    11. Donoghue JP

    (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

    Nature 485:372–375.

    https://doi.org/10.1038/nature11076

    • PubMed
    • Google Scholar
51. 1. Holmes NP

    (2012) Does tool use extend peripersonal space? A review and re-analysis

    Experimental Brain Research 218:273–282.

    https://doi.org/10.1007/s00221-012-3042-7

    • PubMed
    • Google Scholar
52. 1. Hwang EJ
    2. Hauschild M
    3. Wilke M
    4. Andersen RA

    (2012) Inactivation of the parietal reach region causes optic ataxia, impairing reaches but not saccades

    Neuron 76:1021–1029.

    https://doi.org/10.1016/j.neuron.2012.10.030

    • PubMed
    • Google Scholar
53. 1. Iriki A
    2. Tanaka M
    3. Iwamura Y

    (1996) Coding of modified body schema during tool use by macaque postcentral neurones

    Neuroreport 7:2325–2330.

    https://doi.org/10.1097/00001756-199610020-00010

    • PubMed
    • Google Scholar
54. 1. Jackson A
    2. Mavoori J
    3. Fetz EE

    (2006) Long-term motor cortex plasticity induced by an electronic neural implant

    Nature 444:56–60.

    https://doi.org/10.1038/nature05226

    • PubMed
    • Google Scholar
55. 1. Jackson A
    2. Mavoori J
    3. Fetz EE

    (2007) Correlations between the same motor cortex cells and arm muscles during a trained task, free behavior, and natural sleep in the macaque monkey

    Journal of Neurophysiology 97:360–374.

    https://doi.org/10.1152/jn.00710.2006

    • PubMed
    • Google Scholar
56. 1. Kaufman MT
    2. Churchland MM
    3. Shenoy KV

    (2013) The roles of monkey M1 neuron classes in movement preparation and execution

    Journal of Neurophysiology 110:817–825.

    https://doi.org/10.1152/jn.00892.2011

    • PubMed
    • Google Scholar
57. 1. Klaes C
    2. Westendorff S
    3. Chakrabarti S
    4. Gail A

    (2011) Choosing goals, not rules: deciding among rule-based action plans

    Neuron 70:536–548.

    https://doi.org/10.1016/j.neuron.2011.02.053

    • PubMed
    • Google Scholar
58. 1. Kuang S
    2. Morel P
    3. Gail A

    (2016) Planning movements in visual and physical space in monkey posterior parietal cortex

    Cerebral Cortex 26:731–747.

    https://doi.org/10.1093/cercor/bhu312

    • PubMed
    • Google Scholar
59. Conference

    1. Labuguen R
    2. Bardeloza DK
    3. Negrete SB
    4. Matsumoto J
    5. Inoue K
    6. Shibata T

    (2019)

    Primate markerless pose estimation and movement analysis using DeepLabCut

    Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) And 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (IcIVPR). pp. 297–300.

    • Google Scholar
60. 1. Libey T
    2. Fetz EE

    (2017) Open-Source, low cost, Free-Behavior monitoring, and reward system for neuroscience research in Non-human primates

    Frontiers in Neuroscience 11:265.

    https://doi.org/10.3389/fnins.2017.00265

    • PubMed
    • Google Scholar
61. 1. Ludvig N
    2. Tang HM
    3. Gohil BC
    4. Botero JM

    (2004) Detecting location-specific neuronal firing rate increases in the Hippocampus of freely-moving monkeys

    Brain Research 1014:97–109.

    https://doi.org/10.1016/j.brainres.2004.03.071

    • PubMed
    • Google Scholar
62. 1. Maravita A
    2. Spence C
    3. Kennett S
    4. Driver J

    (2002) Tool-use changes multimodal spatial interactions between vision and touch in normal humans

    Cognition 83:B25–B34.

    https://doi.org/10.1016/S0010-0277(02)00003-3

    • PubMed
    • Google Scholar
63. 1. Maravita A
    2. Spence C
    3. Driver J

    (2003) Multisensory integration and the body schema: close to hand and within reach

    Current Biology 13:R531–R539.

    https://doi.org/10.1016/S0960-9822(03)00449-4

    • PubMed
    • Google Scholar
64. 1. Maravita A
    2. Iriki A

    (2004) Tools for the body (schema)

    Trends in Cognitive Sciences 8:79–86.

    https://doi.org/10.1016/j.tics.2003.12.008

    • PubMed
    • Google Scholar
65. 1. Martínez-Vázquez P
    2. Gail A

    (2018) Directed interaction between monkey premotor and posterior parietal cortex during Motor-Goal retrieval from working memory

    Cerebral Cortex 28:1866–1881.

    https://doi.org/10.1093/cercor/bhy035

    • PubMed
    • Google Scholar
66. 1. Mathis A
    2. Mamidanna P
    3. Cury KM
    4. Abe T
    5. Murthy VN
    6. Mathis MW
    7. Bethge M

    (2018) DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

    Nature Neuroscience 21:1281–1289.

    https://doi.org/10.1038/s41593-018-0209-y

    • PubMed
    • Google Scholar
67. Preprint

    1. Mathis A
    2. Yüksekgönül M
    3. Rogers B
    4. Bethge M
    5. Mathis MW

    (2019) Pretraining boosts out-of-domain robustness for pose estimation 

    arXiv.

    https://arxiv.org/abs/1909.11229

    • Google Scholar
68. 1. Moeslund TB
    2. Hilton A
    3. Krüger V

    (2006) A survey of advances in vision-based human motion capture and analysis

    Computer Vision and Image Understanding 104:90–126.

    https://doi.org/10.1016/j.cviu.2006.08.002

    • Google Scholar
69. 1. Mooshagian E
    2. Wang C
    3. Holmes CD
    4. Snyder LH

    (2018) Single units in the posterior parietal cortex encode patterns of bimanual coordination

    Cerebral Cortex 28:1549–1567.

    https://doi.org/10.1093/cercor/bhx052

    • PubMed
    • Google Scholar
70. 1. Mooshagian E
    2. Snyder LH

    (2018) Spatial eye-hand coordination during bimanual reaching is not systematically coded in either LIP or PRR

    PNAS 115:E3817–E3826.

    https://doi.org/10.1073/pnas.1718267115

    • PubMed
    • Google Scholar
71. 1. Morel P
    2. Ferrea E
    3. Taghizadeh-Sarshouri B
    4. Audí JM
    5. Ruff R
    6. Hoffmann KP
    7. Lewis S
    8. Russold M
    9. Dietl H
    10. Abu-Saleh L
    11. Schroeder D
    12. Krautschneider W
    13. Meiners T
    14. Gail A

    (2016) Long-term decoding of movement force and direction with a wireless myoelectric implant

    Journal of Neural Engineering 13:016002.

    https://doi.org/10.1088/1741-2560/13/1/016002

    • PubMed
    • Google Scholar
72. 1. Morel P

    (2018) Gramm: grammar of graphics plotting in matlab

    The Journal of Open Source Software 3:568.

    https://doi.org/10.21105/joss.00568

    • Google Scholar
73. 1. Mulliken GH
    2. Musallam S
    3. Andersen RA

    (2008) Forward estimation of movement state in posterior parietal cortex

    PNAS 105:8170–8177.

    https://doi.org/10.1073/pnas.0802602105

    • PubMed
    • Google Scholar
74. 1. Musallam S
    2. Corneil BD
    3. Greger B
    4. Scherberger H
    5. Andersen RA

    (2004) Cognitive control signals for neural prosthetics

    Science 305:258–262.

    https://doi.org/10.1126/science.1097938

    • PubMed
    • Google Scholar
75. 1. Musial PG
    2. Baker SN
    3. Gerstein GL
    4. King EA
    5. Keating JG

    (2002) Signal-to-noise ratio improvement in multiple electrode recording

    Journal of Neuroscience Methods 115:29–43.

    https://doi.org/10.1016/S0165-0270(01)00516-7

    • PubMed
    • Google Scholar
76. 1. Nakamura T
    2. Matsumoto J
    3. Nishimaru H
    4. Bretas RV
    5. Takamura Y
    6. Hori E
    7. Ono T
    8. Nishijo H

    (2016) A markerless 3D computerized motion capture system incorporating a skeleton model for monkeys

    PLOS ONE 11:e0166154.

    https://doi.org/10.1371/journal.pone.0166154

    • PubMed
    • Google Scholar
77. 1. Nath T
    2. Mathis A
    3. Chen AC
    4. Patel A
    5. Bethge M
    6. Mathis MW

    (2019) Using DeepLabCut for 3D markerless pose estimation across species and behaviors

    Nature Protocols 14:2152–2176.

    https://doi.org/10.1038/s41596-019-0176-0

    • PubMed
    • Google Scholar
78. 1. Niebergall R
    2. Khayat PS
    3. Treue S
    4. Martinez-Trujillo JC

    (2011) Multifocal attention filters targets from distracters within and beyond primate MT neurons' receptive field boundaries

    Neuron 72:1067–1079.

    https://doi.org/10.1016/j.neuron.2011.10.013

    • PubMed
    • Google Scholar
79. 1. Nummela SU
    2. Jovanovic V
    3. de la Mothe L
    4. Miller CT

    (2017) Social Context-Dependent activity in marmoset frontal cortex populations during natural conversations

    The Journal of Neuroscience 37:7036–7047.

    https://doi.org/10.1523/JNEUROSCI.0702-17.2017

    • PubMed
    • Google Scholar
80. 1. Orsborn AL
    2. Moorman HG
    3. Overduin SA
    4. Shanechi MM
    5. Dimitrov DF
    6. Carmena JM

    (2014) Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control

    Neuron 82:1380–1393.

    https://doi.org/10.1016/j.neuron.2014.04.048

    • PubMed
    • Google Scholar
81. 1. Pavani F
    2. Spence C
    3. Driver J

    (2000) Visual capture of touch: out-of-the-body experiences with rubber gloves

    Psychological Science 11:353–359.

    https://doi.org/10.1111/1467-9280.00270

    • PubMed
    • Google Scholar
82. 1. Peikon ID
    2. Fitzsimmons NA
    3. Lebedev MA
    4. Nicolelis MA

    (2009) Three-dimensional, automated, real-time video system for tracking limb motion in brain-machine interface studies

    Journal of Neuroscience Methods 180:224–233.

    https://doi.org/10.1016/j.jneumeth.2009.03.010

    • PubMed
    • Google Scholar
83. 1. Pesaran B
    2. Nelson MJ
    3. Andersen RA

    (2006) Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning

    Neuron 51:125–134.

    https://doi.org/10.1016/j.neuron.2006.05.025

    • PubMed
    • Google Scholar
84. 1. Ponce CR
    2. Genecin MP
    3. Perez-Melara G
    4. Livingstone MS

    (2016) Automated chair-training of rhesus macaques

    Journal of Neuroscience Methods 263:75–80.

    https://doi.org/10.1016/j.jneumeth.2016.01.024

    • PubMed
    • Google Scholar
85. 1. Rajangam S
    2. Tseng PH
    3. Yin A
    4. Lehew G
    5. Schwarz D
    6. Lebedev MA
    7. Nicolelis MA

    (2016) Wireless cortical Brain-Machine interface for Whole-Body navigation in primates

    Scientific Reports 6:22170.

    https://doi.org/10.1038/srep22170

    • PubMed
    • Google Scholar
86. 1. Rizzolatti G
    2. Scandolara C
    3. Matelli M
    4. Gentilucci M

    (1981) Afferent properties of periarcuate neurons in macaque monkeys. II. visual responses

    Behavioural Brain Research 2:147–163.

    https://doi.org/10.1016/0166-4328(81)90053-X

    • PubMed
    • Google Scholar
87. 1. Rizzolatti G
    2. Fadiga L
    3. Fogassi L
    4. Gallese V

    (1997) The space around Us

    Science 277:190–191.

    https://doi.org/10.1126/science.277.5323.190

    • PubMed
    • Google Scholar
88. 1. Roy S
    2. Wang X

    (2012) Wireless multi-channel single unit recording in freely moving and vocalizing primates

    Journal of Neuroscience Methods 203:28–40.

    https://doi.org/10.1016/j.jneumeth.2011.09.004

    • PubMed
    • Google Scholar
89. 1. Santhanam G
    2. Ryu SI
    3. Yu BM
    4. Afshar A
    5. Shenoy KV

    (2006) A high-performance brain-computer interface

    Nature 442:195–198.

    https://doi.org/10.1038/nature04968

    • PubMed
    • Google Scholar
90. 1. Sayegh PF
    2. Gorbet DJ
    3. Hawkins KM
    4. Hoffman KL
    5. Sergio LE

    (2017) The contribution of different cortical regions to the control of spatially decoupled Eye-Hand coordination

    Journal of Cognitive Neuroscience 29:1194–1211.

    https://doi.org/10.1162/jocn_a_01111

    • PubMed
    • Google Scholar
91. 1. Schaffelhofer S
    2. Agudelo-Toro A
    3. Scherberger H

    (2015) Decoding a wide range of hand configurations from macaque motor, premotor, and parietal cortices

    Journal of Neuroscience 35:1068–1081.

    https://doi.org/10.1523/JNEUROSCI.3594-14.2015

    • PubMed
    • Google Scholar
92. 1. Schwarz DA
    2. Lebedev MA
    3. Hanson TL
    4. Dimitrov DF
    5. Lehew G
    6. Meloy J
    7. Rajangam S
    8. Subramanian V
    9. Ifft PJ
    10. Li Z
    11. Ramakrishnan A
    12. Tate A
    13. Zhuang KZ
    14. Nicolelis MA

    (2014) Chronic, wireless recordings of large-scale brain activity in freely moving rhesus monkeys

    Nature Methods 11:670–676.

    https://doi.org/10.1038/nmeth.2936

    • PubMed
    • Google Scholar
93. 1. Serruya MD
    2. Hatsopoulos NG
    3. Paninski L
    4. Fellows MR
    5. Donoghue JP

    (2002) Brain-machine interface: instant neural control of a movement signal

    Nature 416:141–142.

    https://doi.org/10.1038/416141a

    • Google Scholar
94. Preprint

    1. Shahidi N
    2. Schrater P
    3. Wright T
    4. Pitkow X
    5. Dragoi V

    (2019) Population coding of strategic variables during foraging in freely-moving macaques

    bioRxiv.

    https://doi.org/10.1101/811992

    • Google Scholar
95. 1. Sheshadri S
    2. Dann B
    3. Hueser T
    4. Scherberger H

    (2020) 3d reconstruction toolbox for behavior tracked with multiple cameras

    Journal of Open Source Software 5:1849.

    https://doi.org/10.21105/joss.01849

    • Google Scholar
96. 1. Snyder LH
    2. Batista AP
    3. Andersen RA

    (1998) Change in motor plan, without a change in the spatial locus of attention, modulates activity in posterior parietal cortex

    Journal of Neurophysiology 79:2814–2819.

    https://doi.org/10.1152/jn.1998.79.5.2814

    • Google Scholar
97. 1. Suriya-Arunroj L
    2. Gail A

    (2019) Complementary encoding of priors in monkey frontoparietal network supports a dual process of decision-making

    eLife 8:e47581.

    https://doi.org/10.7554/eLife.47581

    • PubMed
    • Google Scholar
98. Preprint

    1. Talakoub O
    2. Sayegh PF
    3. Womelsdorf T
    4. Zinke W
    5. Fries P
    6. Lewis CM
    7. Hoffman KL

    (2019) Hippocampal and neocortical oscillations are tuned to behavioral state in freely-behaving macaques

    bioRxiv.

    https://doi.org/10.1101/552877

    • Google Scholar
99. 1. Taylor DM

    (2002) Direct cortical control of 3D neuroprosthetic devices

    Science 296:1829–1832.

    https://doi.org/10.1126/science.1070291

    • Google Scholar
100. 1. Teikari P
     2. Najjar RP
     3. Malkki H
     4. Knoblauch K
     5. Dumortier D
     6. Gronfier C
     7. Cooper HM

     (2012) An inexpensive Arduino-based LED stimulator system for vision research

     Journal of Neuroscience Methods 211:227–236.

     https://doi.org/10.1016/j.jneumeth.2012.09.012

     • PubMed
     • Google Scholar
101. 1. Velliste M
     2. Perel S
     3. Spalding MC
     4. Whitford AS
     5. Schwartz AB

     (2008) Cortical control of a prosthetic arm for self-feeding

     Nature 453:1098–1101.

     https://doi.org/10.1038/nature06996

     • PubMed
     • Google Scholar
102. 1. Vuilleumier P
     2. Valenza N
     3. Mayer E
     4. Reverdin A
     5. Landis T

     (1998) Near and far visual space in unilateral neglect

     Annals of Neurology 43:406–410.

     https://doi.org/10.1002/ana.410430324

     • PubMed
     • Google Scholar
103. 1. Wessberg J
     2. Stambaugh CR
     3. Kralik JD
     4. Beck PD
     5. Laubach M
     6. Chapin JK
     7. Kim J
     8. Biggs SJ
     9. Srinivasan MA
     10. Nicolelis MA

     (2000) Real-time prediction of hand trajectory by ensembles of cortical neurons in primates

     Nature 408:361–365.

     https://doi.org/10.1038/35042582

     • PubMed
     • Google Scholar
104. 1. Westendorff S
     2. Klaes C
     3. Gail A

     (2010) The cortical timeline for deciding on reach motor goals

     Journal of Neuroscience 30:5426–5436.

     https://doi.org/10.1523/JNEUROSCI.4628-09.2010

     • PubMed
     • Google Scholar
105. 1. Wodlinger B
     2. Downey JE
     3. Tyler-Kabara EC
     4. Schwartz AB
     5. Boninger ML
     6. Collinger JL

     (2015) Ten-dimensional anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations

     Journal of Neural Engineering 12:016011.

     https://doi.org/10.1088/1741-2560/12/1/016011

     • PubMed
     • Google Scholar
106. 1. Wong YT
     2. Fabiszak MM
     3. Novikov Y
     4. Daw ND
     5. Pesaran B

     (2016) Coherent neuronal ensembles are rapidly recruited when making a look-reach decision

     Nature Neuroscience 19:327–334.

     https://doi.org/10.1038/nn.4210

     • PubMed
     • Google Scholar
107. 1. Xu W
     2. de Carvalho F
     3. Jackson A

     (2019) Sequential neural activity in primary motor cortex during sleep

     The Journal of Neuroscience 39:3698–3712.

     https://doi.org/10.1523/JNEUROSCI.1408-18.2019

     • PubMed
     • Google Scholar
108. 1. Yin M
     2. Borton DA
     3. Komar J
     4. Agha N
     5. Lu Y
     6. Li H
     7. Laurens J
     8. Lang Y
     9. Li Q
     10. Bull C
     11. Larson L
     12. Rosler D
     13. Bezard E
     14. Courtine G
     15. Nurmikko AV

     (2014) Wireless neurosensor for full-spectrum electrophysiology recordings during free behavior

     Neuron 84:1170–1182.

     https://doi.org/10.1016/j.neuron.2014.11.010

     • PubMed
     • Google Scholar
109. 1. Zanos S
     2. Richardson AG
     3. Shupe L
     4. Miles FP
     5. Fetz EE

     (2011) The Neurochip-2: an autonomous head-fixed computer for recording and stimulating in freely behaving monkeys

     IEEE Transactions on Neural Systems and Rehabilitation Engineering 19:427–435.

     https://doi.org/10.1109/TNSRE.2011.2158007

     • PubMed
     • Google Scholar
110. 1. Zhou A
     2. Santacruz SR
     3. Johnson BC
     4. Alexandrov G
     5. Moin A
     6. Burghardt FL
     7. Rabaey JM
     8. Carmena JM
     9. Muller R

     (2019) A wireless and artefact-free 128-channel neuromodulation device for closed-loop stimulation and recording in non-human primates

     Nature Biomedical Engineering 3:15–26.

     https://doi.org/10.1038/s41551-018-0323-x

     • PubMed
     • Google Scholar

Acceptance summary:

In this manuscript, the authors present an experimental environment and approach to instruct and quantify behavior in unrestrained macaque monkeys while conducting high-bandwidth wireless recordings from multiple brain areas simultaneously. The results demonstrate how the novel approach and methodology can be used to resolve neural mechanisms of behavior in more naturalistic settings.

Decision letter after peer review:

[Editors’ note: the authors submitted for reconsideration following the decision after peer review. What follows is the decision letter after the first round of review.]

Thank you for submitting your work entitled "The Reach Cage environment for wireless neurophysiology during structured goal-directed behavior of unrestrained monkeys" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Samantha Santacruz (Reviewer #2); Andrew Jackson (Reviewer #3).

Our decision has been reached after consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that your work will not be considered further for publication in eLife at this time. Our assessment is that the revisions required may require a substantial reworking of the study to overcome what might be significant technical limitations. However, the reviewers found enough merit in this study that we would be happy to consider a new submission if you are able to adequately address the concerns raised.

In particular, the reviewers found several aspects of the work of value, such as the timeliness, the design and use of behavioral cues in the cage, and the combination of remote neurophysiological and behavioral tracking. This led to broad support for the goals and methods of the work. However, all three reviewers highlighted major weaknesses in the methodological advance presented that substantially undercut the significance of the work. The fundamental concern is that the behavioral apparatus alone seems more suited to a more specialized methods journal and it is not clear what kind of important scientific advance the wireless and motion capture components can support.

More clarification and quantification of the motion tracking performance is needed. The successful frame capture rate needs to be quantified over intervals of fixed duration and how this compares with reported performance of similar motion tracking methods that are not in-cage needs to be presented. Data for the second subject, Monkey L, also needs to be presented to give a second animal that would strengthen the claims of overall system performance. The inability to monitor more than 1 DOF is also considered a major limitation for a system meant ultimately to examine modulation of activity in an unconstrained animal, and this needs to be addressed.

The wireless recordings also need to be analyzed to quantify in detail wireless performance for the 31 and 127 channel recording setups using standard metrics, such as BER. Since the work is done in a non-ideal, highly reflective environment, we would be satisfied if the performance was close to the worst-case reported BER value of 10^-2 (taken from Yin et al., 2014, from which the current equipment is derived).

More detailed comments from the individual reviewers are provided below.

Reviewer #1:

This manuscript represents behavioral paradigms and data collection methods that are certain to become the focus of attention in the next few years. The study of the relations between single-neuron activity and highly constrained motor behaviors has largely given way in the past ten years to the study of large numbers of simultaneously recorded neurons, albeit, with largely the same behavioral paradigms that have been in place for the past several decades. The advent of the study of more natural behaviors has only more recently begun, given the attendant difficulties of designing useful behavioral paradigms, monitoring the movements themselves, and recording large number of neurons wirelessly.

This study is an initial attempt in that direction, and is described frequently as a proof-of-concept. The MaCaQuE (Macaque Cage Query Extension) component, in addition to its clever name, is the most original and important part of the methods. It combines proximity touch sensors and multi-color cue LEDs with a well-designed network communication method that interconnects the remote components with a microcontroller. The system is expandable and flexible, and forms the basis of a nice behavioral control system for eliciting innovative, more natural behaviors. I also found the flexible head-mounted chamber and connector-mounting system for neural recordings to be innovative and potentially quite useful. It is designed for an Omnetics-based headstage, but could presumably be modified without too great effort for the Cerebus connector.

Unfortunately, the cage itself appears to have been designed with too much metal for good RF transmission. While the 32-channel transmitter functioned adequately, the 128-channel version did not, suffering unacceptable artifacts and signal loss. While 32 channels will certainly be useful for many studies, it is a good bit less than the ~100 channels that have become something of the norm for multi-channel studies these days. Beyond that, passing reference to illumination levels and perhaps other restrictions seem to have severely limited motion tracking. Although they used four cameras to track a single colored marker (dye on the fur) at the monkey's wrist), the percentage of tracked frames seems to have been very low. Although this too, was described in less detail than would have been ideal, apparently one-third of trials tracked fewer than ~150ms of the trajectory ("5 data points"). These limitations of data collection strike me as being pretty substantial.

In addition to the methodological considerations reported here, this study also includes basic results for recordings from three cortical motor areas during "stretch and reach" and "walk and reach" paradigms. These results are presented in sufficiently limited form that they serve primarily to illustrate the paradigms themselves. I am sure that a fuller-length treatment of these data is planned, and anticipate that it will be of great interest. Beyond the behavioral and data collection methods described here, these kinds of experiments will also require a range of more sophisticated analytical approaches to take full advantage of the data.

Consequently, despite its potential, this paper is not adequately compelling either in term of methodological or scientific contributions. Addressing the former will be difficult, unless I have misunderstood the basic limitations of the wireless neural recordings and motion tracking. I assume the latter is a distinct possibility. There are a fair number of grammatical errors that should be addressed, not all of which I attempted to mark.

Reviewer #2:

The authors present a behavior system (the Reach Cage in combination with MaCaQuE interaction device) for performing sensorimotor tasks with nonhuman primates while wirelessly recording neural data. Overall, I find the system design to be novel and a clear demonstration of a system that may be used for the suggested purposes. My two main concerns are: (1) the motivation for the need of such a system, and (2) the limitation of kinematic tracking to one position (wrist). Regarding the former concern, I think the authors could improve their introductory discussion on how such a system is lacking and provides greater insight than, for example, using an existing commercial wireless recording system (e.g. from Blackrock) with something as simple as a cage-mounted touchscreen (essentially porting standard experimental setups with head-fixed animals to a the homecage environment). Regarding the latter concern, the authors do address this point late in the paper (Discussion section) but unconvincingly. There are sleeves/bodysuits that could be used with LED markers for tracking multiple positions that subjects do tolerate. The authors should at least make an argument regarding potential ways they could envision future revisions of their system to allow for tracking of more than one point. Additionally, I have minor concerns regarding the statistical analysis of single-unit data (present in Figure 4). For the analysis of Example B, statistical tests are performed separately for the two time points (cue onset and before go cue) and for the two distances of targets (near and far). It is more appropriate to do a combined statistical analysis, such as a two-way ANOVA with firing rate as the dependent variable, and timepoint (2 levels) and distances (2 levels) are independent variables. Similarly, for example D, a combined statistical analysis should be performed for the distances (near and far) and sides (left and right).

Reviewer #3:

This methods paper by Berger and Gail describes an experimental set-up for studying goal-directed reaching in unrestrained non-human primates with associated wireless neural recording. This work is timely, since the use of chronically-implanted electrodes and wireless recording is allowing neuroscientific experiments to be conducted in less constrained environments. However, the lack of constraints brings both advantages (behaviours may be more naturalistic than conventional restrained tasks such as centre-out reaching) and disadvantages (behaviours are less controlled and therefore data is less amenable to common analysis techniques such as trial averaging). The approach here appears to be a reasonable trade-off between the two, in that movements are unrestrained but trained/cued so as to be fairly stereotyped. I can certainly envisage some scientific questions that would be amenable to such an approach, although with the caveat that such stereotyped behaviours may not be only marginally more naturalistic than the traditional tasks. I also see this work as valuable as a stepping stone to studying fully unconstrained, free behaviour, although in this regard it may be important to incorporate more detailed motion capture in order to make sense of the recordings.

The neural recording uses a commercially available system, and only a couple of exemplar neurons are reported, so the principle novelty of this work comprises a description of the behavioural set-up, task design and kinematic recordings. The system comprises a number of cue/target boxes placed within an arena, with camera-based tracking of movement, and the operation of this is well described in the paper. However, it would be useful to include more description of how easily the animals were trained on this task (e.g. how many sessions, how long did it take for consistent behaviour to emerge). The authors should also assess how stereotyped was the behaviour from trial-to-trial vs. session-to-session. The collection of motion kinematics seems to be fairly limited – only the position of the wrist is tracked via red paint markings, in only one of the animals. Why was this not done for the other animal? No attempt is made to characterise the accuracy of this tracking – it appears that quite a large region of the wrist was painted so I can envisage that determining the centre location from multiple cameras may be quite dependent on arm/hand orientation etc. It would also be interesting to discuss whether other colours on, for example, the elbow and shoulder could allow a more detailed reconstruction of limb kinematics.

The modular design of the headpiece is nice. However, please add a scale bar to Figure 5C and also give dimensions for the total implant height. It is not clear whether the wireless recording system is battery- or inductively-powered. If the former, please give details of battery capacity, life-time, weight etc. If the latter, please include details of how power coils were positioned etc.

[Editors’ note: further revisions were suggested prior to acceptance, as described below.]

Thank you for submitting your article "The Reach Cage environment for wireless neural recordings during goal-directed behavior of unrestrained monkeys" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Samantha R Santacruz (Reviewer #1).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

The authors present an updated version of their behavior system (the Reach Cage in combination with MaCaQuE) which has been demonstrated in a sensorimotor task ("walk-and-reach" task) with nonhuman primates while wirelessly recording neural data as a proof-of-principle of the overall functionality of the system. The primary strength of this work, is perceived by the reviewers to be the reduced constraints on behavior (towards naturalistic settings) while still affording systematic study via controlled tasks and quantification. Previous concerns regarding the motivation for the study and limitations of kinematic tracking have been addressed with a marked significant improvement in the system. However, since the individual pieces of the system have been previously demonstrated (e.g. wireless recordings, in-cage behavior), whether the combined effort represents a novel contribution in a way that will facilitate new experiment needs more direct support. Moreover, more detailed analyses of the behavioral and neurophysiological data are needed.

Essential revisions:

1) The direction of the work (unconstrained task electrophysiology exploration) is of great interest, but the significance of this manuscript toward those goals is still not sufficiently clear. A fundamental concern is that this content seems more appropriate for a specialized methods journal. Although the authors have addressed in their rebuttal and through revisions in their paper the potential for scientific impact of this system, this does not address the point that ultimately they are working in an area of technology/methodology that has been previously reported. Claimed novelty is not correct: other groups have recorded neural data and behavior wirelessly from custom cage designs (Powell, 2017, J. Neuroscience Meth., for example). The authors also claim in the Introduction that goal-directed behaviors that involve walking have not previously been possible, but then cite Yin. Et al., 2014 and Capogrosso et al., 2016 which both involve walking behaviors and are goal (treat) directed walking behavior.

In revision, a deeper comparison to the current manuscript results is needed. First, the MaCaQue system was already published (Berger, 2019). That paper describes the behavioral collection applied to humans. The current report presents a combination of neural data collection with that behavior, and in monkey, which is interesting, but no novel behavioral or neuroscience results are provided. Please describe clearly what is unique in the system from the human version. Please present (e.g., via a table) a comparison of benchmarks across previous reports not limited to the following:

Roy and Wang, 2012; Chestek et al., 2009; Yin et al., 2014; Foster et al., 2014; Schwarz et al., 2014; Capogrosso et al., 2016.

2) The work is overly descriptive, and at some points, misleading. The neuroscience and electrophysiology leave substantial questions about the interpretation of the results and could be described more clearly. For example, the authors appear to present conclusions based on 4 neurons (out of 192) and almost no population analysis is completed (trajectories are not analysis in and of themselves). While statistical tests on those neurons is appropriate, the results do not substantiate the claims that 192 channels are recorded and convey information about goal-directed behavior.

3) Quantification of behavioral variability is also limited. Movement analysis does not describe the kinematics that are said to be collected. For example, acquisition times are given, but not variability in kinematic variables. Additional documentation of the data analyzed for the behavioral variability (Figure 3) is needed. Please report analysis of variability in movement kinematics. How many trials and behavioral sessions are presented? At what point in behavioral training are these data from? Claims of consistency/variability are difficult to assess without these details.

4) Wireless coverage rates merits more careful quantification. Wireless transmission efficacy can often vary across the cage volume due to non-uniform receiver coverage. An analysis of data loss rates that quantifies volume of the cage spanned and presence or absence of spatial trends in where data loss occurs would be significantly more compelling than aggregate data loss rates. Drop-out with strong spatial bias would hamper and complicate task analyses. This is important for the authors to quantify because their other analyses highlight that animals exhibit relatively stereotyped behaviors during tasks (Figure 3). Are the low data loss rates because the animals occupy a restricted volume of the cage during task behavior? Some quantification of coverage volume will also be valuable for reporting and demonstrating the tool's capabilities.

5) Claims for BMI utility require more careful elaboration. The authors argue their system, with real-time neural data recording and behavioral control, would be useful for BMI studies. While this is true, their system as currently designed has constraints on the possible BMI experiments that could be done which should be elaborated. Their behavioral task system, with discrete targets, would have limited applications to discrete decoding behaviors. Elaboration on specific utility and limitations/future directions is needed.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for submitting your article "Wireless recording from unrestrained monkeys reveals motor goal encoding beyond immediate reach in frontoparietal cortex" for consideration by eLife. Your revised article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Samantha R Santacruz (Reviewer #1).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission to clear up a few remaining points.

We would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). Specifically, we are asking editors to accept without delay manuscripts, like yours, that they judge can stand as eLife papers without additional data, even if they feel that they would make the manuscript stronger. Thus the revisions requested below only address clarity and presentation.

Summary:

The authors present a cohesive and compelling piece of work 1) demonstrating the utility of their in-cage behavioral system and wireless recordings, and 2) providing novel findings enabled by their work. The reviewers find the manuscript is greatly strengthened by the new analyses and re-framing to fully place their work in context. The combination of in-cage, untethered neural recording with well-controlled sensorimotor tasks is both powerful and a niche that has not been addressed to-date. the manuscript is suitable for publication in eLife subject to revisions.

Essential revisions:

1) The authors state in the rebuttal "We now added an analysis on all 12 recorded session and test for each of the 192 channels with an ANOVA if the activity is task ( = time, target distance or position) modulated and if it is target position modulated (Results section), which was not the scope of the analysis in the previous version of the manuscript. We demonstrate that there are sessions that show 192 task modulated channels and provide statistics on the population activity in the newly written part of the Results section." I believe this quoted text actually refers to: "movement. Of all twelve recorded sessions three sessions revealed task responsive activity on all 192 channels, i.e. showed at least one effect in distance, position, time or one of the interactions; across all sessions the mean number of task-responsive channels was 189 (s.d. 5 channels). Up to 179 channels were position responsive, i.e. showing at least one effect in position or one of the interactions (mean: 162, s.d. 17 channels)." It is unclear how this analysis was performed. Was "task responsive activity" determined from another ANOVA? Please clarify.

2) In the section "Premotor and parietal cortex encode movement goals beyond immediate reach". Here the authors perform SVM classification using neural spiking activity to decode reach to either a near or far target. This analysis is problematic in that they determine significant decoding accuracy by comparing performance to that of a classifier using activity prior to cue onset and also assume chance levels of 50% (based on plots in Figure 5B). It would be more appropriate to compare accuracy to shuffled data in order to determine true chance levels (particularly if the near and far trials aren't balanced) and significance. Please perform the permutation-based test of significance.

3) The authors state: "Moreover, we could decode walk-and-reach target location information from premotor and parietal cortex, but not motor cortex, during movement and even during the memory period before the movement. This suggests that premotor and parietal cortex encodes motor goals beyond immediate reach." A decoding model and encoding model are complimentary, but not the same. High decoding accuracy from the SVM simply means that the population-level neural activity during these timepoints is separable enough in a high dimensional space. This may be due to many factors. To more completely demonstrate that premotor and parietal cortex encode motor goals beyond immediate reach, an encoding model (e.g. regression or GLM) describing how neural activity co-varies with the motor goals would be valuable.

[Editors’ note: the authors resubmitted a revised version of the paper for consideration. What follows is the authors’ response to the first round of review.]

[…]

The fundamental concern is that the behavioral apparatus alone seems more suited to a more specialized methods journal and it is not clear what kind of important scientific advance the wireless and motion capture components can support.

We are not aware of previous work studying large-scale neural network activity in freely moving monkeys trained on goal-directed naturalistic walk-and-reach movements that are temporally and spatially structured under the control of the experimenter. We provide new wireless neurophysiology and motion capture data. We show broadband intracortical recordings from 192 channel at 30 ksps from three brain areas simultaneously. Motion capture is now fully markerless and provides 3D tracking of four body-parts namely head, left wrist, elbow and shoulder. We show with an additional analysis that single-trial neural dynamics of multiple areas can be analyzed and correlated with detailed multi-joint kinematics of on unrestrained behavior. Real-time read-out of the neural data makes the setting suitable for BMI applications. Details are described below.

More clarification and quantification of the motion tracking performance is needed. The successful frame capture rate needs to be quantified over intervals of fixed duration and how this compares with reported performance of similar motion tracking methods that are not in-cage needs to be presented. Data for the second subject, Monkey L, also needs to be presented to give a second animal that would strengthen the claims of overall system performance. The inability to monitor more than 1 DOF is also considered a major limitation for a system meant ultimately to examine modulation of activity in an unconstrained animal, and this needs to be addressed.

Now, we use a different approach for markerless motion tracking based on a deep neural network. We present new data from both monkeys performing a memory-guided walk-and-reach task. The head, left wrist, elbow and shoulder are tracked in 3D during performance of the walk-and reach task. We increased the framerate to 60 Hz and quantified the successful frame capture rate after outlier rejection (97.58% – 99.95%).

The main goal of the presented research approach is to provide an experimental setting that allows studying structured and goal-oriented behavior beyond what is possible in a chair-based setup, for instance walk-and-reach behavior towards targets outside the immediate peripersonal space. We tried to make our rational clearer throughout the manuscript.

The wireless recordings also need to be analyzed to quantify in detail wireless performance for the 31 and 127 channel recording setups using standard metrics, such as BER. Since the work is done in a non-ideal, highly reflective environment, we would be satisfied if the performance was close to the worst-case reported BER value of 10^-2 (taken from Yin et al., 2014, from which the current equipment is derived).

We now use 2x 96-channel wireless headstages simultaneously to records from all 192 electrodes at 30 ksps sampling rate at the same time. Our former wireless system was not derived from Yin et al., however, our current system distinctly is. Since we do not have direct access to the digital data stream of this proprietary system, we cannot perform BER measures as shown by Yin et al. or other publications about wireless systems. Also, we do not believe that a standard BER measure is suitable to judge the relevant performance, since it is a property of the system itself and measured under optimal conditions for wireless transmission. Here, we report an experimental environment and not a wireless recording system. Thus, we report transmission stability, i.e. the fraction of time points for which we received data (around 96.68%). This is an equivalent metric to BER, in our application, as instead of measuring the quality of the wireless transmission signal itself, we measure the ability to receive the signal within our recording environment during a recording session. We are not aware of corresponding stability measures for comparable setups in which monkeys relocate themselves. This includes Yin et al., who provide data of monkeys sleeping or walking on a treadmill. In both cases, the head of the monkey, and consequently the wireless transmitter, stays approximately in the same position, which is technically less challenging.

[Editors’ note: what follows is the authors’ response to the second round of review.]

Essential revisions:

1) The direction of the work (unconstrained task electrophysiology exploration) is of great interest, but the significance of this manuscript toward those goals is still not sufficiently clear. A fundamental concern is that this content seems more appropriate for a specialized methods journal. Although the authors have addressed in their rebuttal and through revisions in their paper the potential for scientific impact of this system, this does not address the point that ultimately they are working in an area of technology/methodology that has been previously reported. Claimed novelty is not correct: other groups have recorded neural data and behavior wirelessly from custom cage designs (Powell, 2017, J. Neuroscience Meth., for example). The authors also claim in the Introduction that goal-directed behaviors that involve walking have not previously been possible, but then cite Yin. et al., 2014 and Capogrosso et Al., 2016, which both involve walking behaviors and are goal (treat) directed walking behavior.

Our manuscript is not primarily about wireless transmission, it is about a conceptual approach in cognitive and sensorimotor neuroscience bearing on, among other things, wireless recordings. In this sense the study of Powell et al., 2017, which presents an RF transparent cage, i.e. equipment for wireless recording, has a very different scope. We present an experimental environment and approach for which precisely guided and quantifiable spatially and temporally instructed behavior is possible with unrestrained macaques while conducting high-bandwidth wireless recordings from multiple brain areas simultaneously. The combined features of the Reach Cage allow experimental designs that former neurophysiology studies/environments with unrestrained monkeys did not provide:

– Precise computer-controlled timing of temporal instructions (programmable color illumination of targets) to the animal and real-time control of the experimental flow based on the animals registered behavior (touch-sensitivity of the targets).

– As a consequence, investigation of motor planning is possible and demonstrated here. No other study known to us dealt with motor planning in unrestrained monkeys.

– Multiple distributed, light identifiable and instructable reach goals that are independent of the food source (important in cognitive neuroscience) and can be configured variably in number and position.

– As a consequence, a defined starting position with the ability to pause the animal’s movement allows to provide visual cues at a defined locations relative to the body, necessary for motor-cognitive aspects of sensorimotor science. For example, trained goal directed behavior led to a distinct number of behavioral categories (here movements to 8 different targets) and repetitive trials as previously possible only in more conventional chair-seated paradigms (see also response to comment 3). We are not aware of a study showing that pure training of monkeys without any physical restraint leads to such low trial-to-trail variability in freely moving animals.

We have revised the Introduction and Discussion to make the idea behind our study more clear. We have added an overview of previous studies, Supplementary file 1, as suggested below.

As requested by the editor’s summary, we provide more detail on our neurophysiology data as a proof-of-concept. For this, we replaced the last subsection in the Results section with a new analysis of the neurophysiology data that shows that premotor and parietal cortex encode motor goals beyond immediate reach during planning and locomotion. We are not aware of existing experimental environments or published data that would show this or would have been able to show this.

In revision, a deeper comparison to the current manuscript results is needed. First, the MaCaQue system was already published (Berger, 2019). That paper describes the behavioral collection applied to humans. The current report presents a combination of neural data collection with that behavior, and in monkey, which is interesting, but no novel behavioral or neuroscience results are provided. Please describe clearly what is unique in the system from the human version.

We are surprised by this argument given eLife’s support for preprints and find it in fact a highly problematic argument. The MaCaQuE system was first published as the preprint of this manuscript. In Berger et al., 2019, we cited the preprint accordingly. We will cite this manuscript, not Bergeret al.,2019, when referring to the MaCaQuE System, since only here we present an extensive description of the MaCaQuE system. Since the current manuscript was already in the pipeline at eLife and available as preprint, we think this is the only appropriate way of dealing with such situation. Since the publication history can easily be tracked in the archive system, the genesis is transparent. Regarding novelty, Berger et al., 2019, is all about a human psychophysics result showing congruency effects in the interference of cross-modal sensory integration with spatial distractor stimuli, not about the MaCaQuE system and not about non-human primates, as evident from the paper. The novelty here is explained in our response to the above comment, for which MaCaQuE is an important but not the sole component. Specifically, the novelty here is directed towards studies with non-human primates. The current and previous studies have a completely independent scope.

We find the argument highly problematic since it effectively means that throughout the review process, during which authors – like in our case – invest massively to follow reviewer requests for significant modifications and sometimes have to wait many months to get feedback on their submission, authors would not be allowed to make reference to their published preprints in other studies. This defeats the purpose of preprints and in our view contradicts eLife’s policy.

Please present (e.g., via a table) a comparison of benchmarks across previous reports not limited to the following:

Roy and Wang, 2012; Chestek et al., 2009; Yin et al., 2014; Foster et al., 2014; Schwarz et al., 2014; Capogrosso et al., 2016.

We present this table as Supplementary file 1.

2) The work is overly descriptive, and at some points, misleading. The neuroscience and electrophysiology leave substantial questions about the interpretation of the results and could be described more clearly. For example, the authors appear to present conclusions based on 4 neurons (out of 192) and almost no population analysis is completed (trajectories are not analysis in and of themselves). While statistical tests on those neurons is appropriate, the results do not substantiate the claims that 192 channels are recorded and convey information about goal-directed behavior.

Since the focus of the manuscript, based on the reviewer suggestions, has now partly shifted towards more neuroscientific content, we also adapted the style and it should be less descriptive. The methods-oriented parts, which were originally more in the foreground since we submitted to the “Tools and Resources” rubric of the journal, remain descriptive. The example units are shown to demonstrate that it is possible to record clearly modulated, well-isolated single-unit activity in much detail, with modulations of the neural responses following closely the precisely controlled different stages of the trials over time and at the same time being selective for the spatial parameters of the task. We now added an analysis on all 12 recorded session and test for each of the 192 channels with an ANOVA if the activity is task ( = time, target distance or position) modulated and if it is target position modulated (Results section), which was not the scope of the analysis in the previous version of the manuscript. We demonstrate that there are sessions that show 192 task modulated channels and provide statistics on the population activity in the newly written part of the Results section.

3) Quantification of behavioral variability is also limited. Movement analysis does not describe the kinematics that are said to be collected. For example, acquisition times are given, but not variability in kinematic variables. Additional documentation of the data analyzed for the behavioral variability (Figure 3) is needed. Please report analysis of variability in movement kinematics. How many trials and behavioral sessions are presented? At what point in behavioral training are these data from? Claims of consistency/variability are difficult to assess without these details.

We do report an analysis of variability of movement kinematics (Figure 3B and third paragraph in this subsection). This is the across time points averaged Euclidean distance for each trial to a trial-averaged trajectory. We show that trials deviate only few centimeters from the averaged trajectory. Even for the most variable marker (wrist) the averaged Euclidean distance is below 67 mm for 75% of the trials.

We now added more sessions to this analysis. Now the analysis is based on 2 (monkey K) and 3 sessions (monkey L) with a total of 469 and 872, respectively, successful trials. We added this information in the Results section and Figure 3.

All data presented in the manuscript is after behavioral training. We added this information in the Materials and methods section.

4) Wireless coverage rates merits more careful quantification. Wireless transmission efficacy can often vary across the cage volume due to non-uniform receiver coverage. An analysis of data loss rates that quantifies volume of the cage spanned and presence or absence of spatial trends in where data loss occurs would be significantly more compelling than aggregate data loss rates. Drop-out with strong spatial bias would hamper and complicate task analyses. This is important for the authors to quantify because their other analyses highlight that animals exhibit relatively stereotyped behaviors during tasks (Figure 3).

We added an analysis of data loss rates across targets (Results section and Figure 4—figure supplement 1). While the raw data shows some dependency of data loss rates on the target position in the range of few percent, we did not observe significant differences between trials towards different target positions in the left-right dimension after applying our 5% criterion for rejecting poor trials. We do see a difference between reach and walk-and-reach movements; however, this does not affect our results since these quantify left-right selectivity for reach and walk-and-reach movements independently.

Are the low data loss rates because the animals occupy a restricted volume of the cage during task behavior? Some quantification of coverage volume will also be valuable for reporting and demonstrating the tool's capabilities.

Since monkeys performed a spatially and temporally highly structured behavior during the recording sessions, we cannot draw conclusions on the coverage beyond the space covered in Figure 4—figure supplement 1.

5) Claims for BMI utility require more careful elaboration. The authors argue their system, with real-time neural data recording and behavioral control, would be useful for BMI studies. While this is true, their system as currently designed has constraints on the possible BMI experiments that could be done which should be elaborated. Their behavioral task system, with discrete targets, would have limited applications to discrete decoding behaviors. Elaboration on specific utility and limitations/future directions is needed.

We thank the reviewer for pointing this out. We used the opportunity to rewrite the corresponding Discussion section. On the one hand, we are specifically interested in the control of discrete events, e.g. for smart home applications, for which spatial selectivity/invariance regarding the subjects position in space is important. On the other hand, robustness of prosthetic control during performance of other movements in parallel, like walking, is critical for applications in partially impaired patients (e.g. arm amputees). The new neural analyses show that target location information can be decoded during walking, as a first step to show that information for BMIs can be harnessed from the neural activity recorded in the Reach Cage. We now elaborate on this in the Discussion section, but since BMI application is not the main focus of this study, we removed the paragraph from the Introduction.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Essential revisions:

1) The authors state in the rebuttal "We now added an analysis on all 12 recorded session and test for each of the 192 channels with an ANOVA if the activity is task ( = time, target distance or position) modulated and if it is target position modulated (Results section), which was not the scope of the analysis in the previous version of the manuscript. We demonstrate that there are sessions that show 192 task modulated channels and provide statistics on the population activity in the newly written part of the Results section." I believe this quoted text actually refers to: "movement. Of all twelve recorded sessions three sessions revealed task responsive activity on all 192 channels, i.e. showed at least one effect in distance, position, time or one of the interactions; across all sessions the mean number of task-responsive channels was 189 (s.d. 5 channels). Up to 179 channels were position responsive, i.e. showing at least one effect in position or one of the interactions (mean: 162, s.d. 17 channels)." It is unclear how this analysis was performed. Was "task responsive activity" determined from another ANOVA? Please clarify.

We understand why this paragraph is unclear and added an explanatory sentence: “We performed the same ANOVA for the activity in each channel of all twelve recorded sessions.“

2) In the section "Premotor and parietal cortex encode movement goals beyond immediate reach". Here the authors perform SVM classification using neural spiking activity to decode reach to either a near or far target. This analysis is problematic in that they determine significant decoding accuracy by comparing performance to that of a classifier using activity prior to cue onset and also assume chance levels of 50% (based on plots in Figure 5B). It would be more appropriate to compare accuracy to shuffled data in order to determine true chance levels (particularly if the near and far trials aren't balanced) and significance. Please perform the permutation-based test of significance.

We replaced the statistics on the decoding analysis with a permutation-based test of significance for our main analysis (Results, Figure 5 and Figure 5—source data 1) and for our supplementary analysis (Results, Figure 5—figure supplement 1 and Figure 5—source data 3). The permutation-based test revealed the same results as the test in the previous version of the manuscript, apart from now one additional significant decoding accuracy in M1 of monkey L during movement (but not within the memory period) of walk-and-reach movements. All previously reported effects are still valid. This minimally different result does not change the conclusions of our study. We revised the Materials and methods section accordingly.

3) The authors state: "Moreover, we could decode walk-and-reach target location information from premotor and parietal cortex, but not motor cortex, during movement and even during the memory period before the movement. This suggests that premotor and parietal cortex encodes motor goals beyond immediate reach." A decoding model and encoding model are complimentary, but not the same. High decoding accuracy from the SVM simply means that the population-level neural activity during these timepoints is separable enough in a high dimensional space. This may be due to many factors. To more completely demonstrate that premotor and parietal cortex encode motor goals beyond immediate reach, an encoding model (e.g. regression or GLM) describing how neural activity co-varies with the motor goals would be valuable.

We added an encoding (“tuning”) analysis. As we are interested in population and not single unit effects, we calculated the population average of the firing rate modulation, i.e. absolute difference between average firing rate for left and right trials and then averaged over all units after aligning to the preferred side (“population tuning”). As for the decoding analysis, we calculated a permutation-based test to test for significant encoding (Figure 5 and Figure 5—source data 2) or significant differences between passage and open trials (Figure 5—figure supplement 1 and Figure 5—source data 4). We describe this accordingly in the Results and Materials and methods. The results for the encoding analysis do not differ from the decoding analyses, hence the conclusions of the study remain the same.

Author details

1. Michael Berger

   1. Cognitive Neuroscience Laboratory, German Primate Center – Leibniz-Institute for Primate Research, Goettingen, Germany
   2. Faculty of Biology and Psychology, University of Goettingen, Goettingen, Germany

   Contribution

   Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology

   For correspondence

   mberger@dpz.eu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7239-1675

2. Naubahar Shahryar Agha

   Cognitive Neuroscience Laboratory, German Primate Center – Leibniz-Institute for Primate Research, Goettingen, Germany

   Contribution

   Investigation, Methodology

   Competing interests

   No competing interests declared

3. Alexander Gail

   1. Cognitive Neuroscience Laboratory, German Primate Center – Leibniz-Institute for Primate Research, Goettingen, Germany
   2. Faculty of Biology and Psychology, University of Goettingen, Goettingen, Germany
   3. Leibniz-ScienceCampus Primate Cognition, Goettingen, Germany
   4. Bernstein Center for Computational Neuroscience, Goettingen, Germany

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Methodology, Project administration

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-1165-4646

• 4,244

  Page views

• 549

  Downloads

• 21

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.