Sensorimotor Transformation: The hand that ‘sees’ to grasp

  1. Kenneth F Valyear  Is a corresponding author
  1. Bangor University, United Kingdom

How does the hand know what the eyes see? When we reach to grasp an object, our hand shapes to match the object's size, shape, and orientation. How does the brain translate visual information into motor commands that control the hand?

Now, in eLife, thanks to the work of Stefan Schaffelhofer and Hansjörg Scherberger of the German Primate Center in Göttingen, our understanding of this fundamental process has been significantly advanced (Schaffelhofer and Scherberger, 2016). We can think about the information that is represented in the activity of cells as a ‘code’. For the first time, cells that code for the visual properties of objects are distinguished from those that code for how the hand is moved to grasp.

The approach used by Schaffelhofer and Scherberger records the hand movements and the activity of brain cells in monkeys while they view and grasp objects of different shapes and sizes. Recordings are taken from three brain areas known to be important for grasping – the anterior intraparietal (AIP) area, the ventral premotor area F5 and the primary motor hand area M1 (Figure 1).

A schematic representation of the brain areas implicated in the transformation of visual-to-motor information during grasping.

The cortical surface of the macaque monkey is shown. The cortical surface is defined at the gray-white matter boundary and has been partially inflated to reveal regions within the sulci (the grooves on the brain’s surface) while preserving a sense of curvature. AIP = anterior intraparietal area, F5 = ventral premotor area, M1 = primary motor hand area. White lines indicate sulci. IPS = intraparietal sulcus, STS = superior temporal sulcus, CS = central sulcus, AS = arcuate sulcus, LS = lunate sulcus. LH = left hemisphere. The monkey MRI data on which the reconstruction is based was provided by Stefan Everling.

The set of objects used in the study elicits a wide range of different hand postures. Critically, some objects look different but are grasped similarly, while others look identical but are grasped in different ways. This approach allows for cells that represent visual object properties (visual-object encoding) to be distinguished from those that represent how the hand is moved during grasping (motor-grasp encoding).

The results reveal predominately visual-object encoding in area AIP and motor-grasp encoding in areas F5 and M1. The cells in area AIP respond strongly during object viewing, and these responses clearly reflect object shape. Conversely, area F5 responds only weakly during the viewing period, and its activity strongly reflects hand movements during grasping.

It is particularly informative that when the objects are visually distinct but are grasped similarly, each object initially causes a distinct response in area AIP during the viewing period. These responses become more similar over time and before the start of a movement. This is consistent with a change from a visual-object to a motor-grasp encoding scheme. Area AIP also responds differently to visually identical objects that are grasped differently, which is also consistent with a motor-grasp encoding scheme.

Despite these aspects of their results, Schaffelhofer (who is also at Rockefeller University) and Scherberger (who is also at the University of Göttingen) maintain that altogether their data more strongly support a visual-object encoding account of AIP activity. They suggest that area AIP represents the visual features of objects that are relevant for grasping.

According to this account, over time AIP activity reflects a narrowing of action possibilities, honing in on the object features that will be essential for the upcoming grasp. This explains why responses to different objects that are grasped similarly become increasingly similar during planning, and why grasping the same object in different ways elicits distinct responses.

The findings also reveal that areas AIP and F5 briefly show common encoding when viewing objects, suggesting that these areas share information during this time. Speculatively, feedback from F5 may help to narrow the range of possible hand actions (specified visually in area AIP) to a single set of grasp points on the target object.

The results of Schaffelhofer and Scherberger also provide compelling evidence for the role of area F5 in driving the activity of the primary motor area M1. Response encoding during grasping becomes remarkably similar between areas F5 and M1, and F5 responses show earlier onsets. These data complement and extend previous results (Umilta et al., 2007; Spinks et al., 2008).

Altogether the new findings suggest the following model. Area AIP represents visual information about the features of objects. Together with area F5, area AIP then ‘flags’ those features that are most relevant for the intended actions, and the two areas collaborate to transform this information into the sensory and motor parameters that control the hand during grasping. Finally, area F5 signals this information to area M1, and ultimately the information reaches the spinal cord for controlling the hand and finger muscles.

This model can now be further tested. For example, the model predicts that area AIP will respond robustly and variably to objects that can be grasped in many different ways. The model also suggests that the more difficult it is to visually identify the parts of an object that will permit a stable grasp, the more rigorously AIP will respond.

It would also be interesting to see how the interplay between areas AIP and F5 unfolds when grasping the same object for different purposes (Marteniuk et al., 1987; Ansuini et al., 2006). Does the activity in area AIP first represent all the visual features of the object, as the current results of Schaffelhofer and Scherberger suggest? Or are the responses in area AIP adjusted to reflect those features of the object that are most relevant to the specific action that is intended?


Article and author information

Author details

  1. Kenneth F Valyear

    School of Psychology, Bangor University, Bangor, United Kingdom
    For correspondence
    Competing interests
    The author declares that no competing interests exist.

Publication history

  1. Version of Record published: July 29, 2016 (version 1)


© 2016, Valyear

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.


  • 1,465
    Page views
  • 99
  • 0

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Kenneth F Valyear
Sensorimotor Transformation: The hand that ‘sees’ to grasp
eLife 5:e18887.

Further reading

    1. Neuroscience
    Niraja Ramesh, Marc Escher ... Stephan J Sigrist
    Research Article

    We still face fundamental gaps in understanding how molecular plastic changes of synapses intersect with circuit operation to define behavioral states. Here we show that an antagonism between two conserved regulatory proteins, Spinophilin (Spn) and Syd-1, controls presynaptic long-term plasticity and the maintenance of olfactory memories in Drosophila. While Spn mutants could not trigger nanoscopic active zone remodeling under homeostatic challenge and failed to stably potentiate neurotransmitter release, concomitant reduction of Syd-1 rescued all these deficits. The Spn/Syd-1 antagonism converged on active zone close F-actin, and genetic or acute pharmacological depolymerization of F-actin rescued the Spn deficits by allowing access to synaptic vesicle release sites. Within the intrinsic mushroom body neurons, the Spn/Syd-1 antagonism specifically controlled olfactory memory stabilization but not initial learning. Thus, this evolutionarily conserved protein complex controls behaviorally relevant presynaptic long-term plasticity, also observed in the mammalian brain but still enigmatic concerning its molecular mechanisms and behavioral relevance.

    1. Computational and Systems Biology
    2. Neuroscience
    Huu Hoang, Shinichiro Tsutsumi ... Keisuke Toyama
    Research Article Updated

    Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing ‘Go/No-go’ reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.