Perception as a closed-loop convergence process
- Weizmann Institute of Science, Israel
Figures
Figure 1
Possible perceptual schemes.
(A) An open-loop scheme (in the motor-sensory sense) – perception begins with an interaction (uni- or bi-directional) between the object and the sensory organ (an eye in this illustration) and ends somewhere in the brain where a relevant neuronal representation (NR) is formed. (B) A closed-loop scheme (in the motor-sensory sense) – perception is a circular process, with no starting or ending points, which contains the sensory organ.
Figure 2
Active sensing.
(A) Ocular scanning of a scene.The trajectory of a human subject’s gaze (of one eye) during free viewing of an image presented on a computer screen is depicted. “Drift” points to the slow eye movements scanning a region of interest during a fixational pause. “Saccade” points to a rapid saccadic eye movement moving the gaze from one fixational pause to another. Section duration: 60 s; sampling: 240 Hz. Courtesy of Moshe Fried and Amos Arieli. (B) Manual scanning of a surface. The trajectory of a human subject’s hand, while palpating a virtual surface with a varying density of elevated dots (black), is depicted. The surface was mimicked via a tactile computer mouse system (VTPlayer; VirTouch, Jerusalem) whose two 4x4 pin arrays, which were touched constantly with the index and middle fingers of the right hand, reflected the spatial details of the virtual surface according to mouse location. Section duration: 152 s; sampling: 125 Hz. Courtesy of Avraham Saig and Amos Arieli. (C) Facial scanning of an arena. The trajectory of the snout of a rat, exploring an arena using sniffing and touch, is depicted. Section duration: 828 s; sampling: 25 Hz. Courtesy of Ben Mitchinson, Chris J. Martin, Robyn A. Grant and Tony. J. Prescott; see (Mitchinson et al., 2007). (D) Local vibrissal scanning. The trajectory of a point near the middle of whisker C1 of a rat, exploring a region of an arena, is depicted. All whiskers except row C were trimmed on both sides of the snout. Section duration: 1.5 s; sampling: 500 Hz. Courtesy of Tess Oram, Noy Barak and Dudi Deutsch. (E) Sensory granularity. Left, a sample of retinal photoreceptors array of the human foveal area (from Curcio et al., 1987). Middle, a schematic illustration of the organization of one type of mechanoreceptor (rapidly adapting) under the skin of the human fingertip. Right, whiskers array: left, the array of whiskers across the right snout of a rat, courtesy of Sebastian Haidarliu; right, a schematic illustration of a whisker’s follicle containing hundreds of mechanoreceptors, courtesy of Satomi Ebara.
Figure 3
An example of MS-Contingency in vibrissal touch.
A schematic illustration of morphological coding of object location (Bagdasarian et al., 2013) is depicted. The motor-sensory phase plane describes the combinations of values of a motor (θp: push angle, maximal change in whisker angle from contact onset) and sensory (k, whisker base curvature) variables when a whisker actively contacts an object at various locations. The locations are defined by their coordinates in the horizontal plane (inset): three azimuth coordinates (Lθ = [p1, p2, p3]) and three radial coordinates (Lr = [60%, 75%, 90%] of whisker length) are depicted and coded by colors. Note that neither of the two variables provide unambiguous coding of object location by itself; for example, k around .02 mm-1 codes for both ~[p2, 60%] and ~[p1, 90%]. In contrast, the contingency between the motor and sensory variables provides unique coding of both Lr and Lθ (see equations).
Figure 4
Anatomy and perceptual schemes of a sensory modality.
(A) Closed-loop motor-sensory-motor (MSM) connections of the vibrissal system.A schematic diagram of the most relevant connections, through which sensory activities feed motor circuits at various levels, is depicted; efference copies are not explicitly depicted. Oval circles indicate brain regions [BPN, brainstem premotor nuclei (arbitrarily divided into two oval circles); BG, basal ganglia; Cer, cerebellum; FN, facial nucleus; MCx, motor cortex; POm, posteromedial thalamic nucleus; RN, red nucleus; SC, superior colliculus; SI, primary somatosensory cortex; SII, secondary somatosensory cortex; TG, trigeminal ganglion; TN, trigeminal brainstem nuclei; VL, ventrolateral thalamic nucleus; VPM, ventroposteromedial thalamic nucleus; ZI, zona incerta]. Black curves connecting brain regions indicate anatomical connections. Arrows indicate the direction of information flow between brain regions. Connections not labeled with arrows are reciprocal (for more details see Bosman et al., 2011; Diamond et al., 2008; Kleinfeld et al., 2006). Three examples of individual MSM-loops are illustrated by green (a brainstem loop), blue (a thalamic loop) and red (a cortical loop); the primary efferents (FN to muscles) and afferents (follicle to TN) may or may not be common to different pathways. Modified from (Ahissar and Knutsen, 2008; Ahissar et al., 2015b). Inset, top view of the head and whiskers of a rat performing a bilateral localization task. (B) An MSM-loop (left) activates and senses the same organ. Sensory-motor arcs (right), which sense one organ and activate another, are not discussed in this paper. (C) Inclusion in an MSM-loop. Re-afferent loops (green) are always closed and thus can be considered as constantly ‘perceiving’ their organs. Ex-afferent loops (magenta) are normally open (dotted). An ex-afferent loop is closed (solid) only when the sensory organ interacts with the object (right); neither object presence alone (left) nor sensor movement alone (middle) close the loop.
Figure 5
CLP dynamics.
(A) The dynamics of perception of an individual feature by an individual MSM-loop follows a convergence pattern.The loop starts converging towards its steady-state (in which state perception is complete and “direct”) upon the first interaction with the object, whether active or passive (e.g., a flashed stimulus). The confidence of perceiving feature j (Cj) gradually increases during convergence. The loop may quit the process when Cj becomes larger than a certain internal threshold (Cd) or upon an internal or external perturbation. (B) The relationships between physical and perceptual time during CLP convergence are presented via a spiral metaphor, in which the physical time can be measured along the spiral, and the perceptual time can be measured across the spiral, e.g., by counting the number of activations of a given point along the loop. A steady-state can be reached at some point along the process.
Figure 6
Synthesis of closed-loop perception in a robotic setup.
(A) A sketch of the MSM-loop model template.m, motor variable; SO, sensory organ; s, sensory variable; n, neuronal variable; h, f, g, transfer functions; u the environment dynamics. The arrows depict the direction of information flow within the loop. (B) The SYCLOP robotic platform. A sketch of the robot with its different components: Pan-Tilt control unit (PTCU, only the pan axis was used here) (1), DVS camera (2), and desktop computer (3). The computer sends commands to the PTCU which controls the camera’s rotations in the azimuth (θ) and elevation (ε) axes. The DVS camera sends visual ‘on’ and ‘off’ events to the computer. (C-F) Implementation of a specific contrast perceiving CLP model (see text). (C,D) n1, the integrated difference between ‘ON’ and ‘OFF’ events, and ω, sensor angular velocity along the pan axis, (C and D, respectively) as a function of time in two different runs of the CLP algorithm, one facing a contrast of 0.9 (red) and one facing a contrast of 0.5 (green). n1 is scaled in units of 1000 events. (E) System’s trajectories in the 2D n1-ω state space. Same data as in C and D. (F) Example of emergent smooth-pursuit like behavior when using a moving edge as a stimulus. The trajectory of the system in the n1-ω plane (gray line) overlaid on a heat map where the color of each segment corresponds to the amount of time in seconds the system spent within this segment. The smooth pursuit periods are represented by the white and light red squares. While in a smooth pursuit, the camera was moving with a constant angular velocity – smoothly tracking the edge.
Figure 7
Functional connectivity and essential elements of perceptual schemes.
The essential elements in each scheme are indicated by solid curves and blue titles.MSM, motor-sensory-motor; MS, motor-sensory; S, sensory; NR, neuronal representation; green curves, re-afferent related pathways; magenta curves, ex-afferent related pathways. Note that re-afferent related pathways can form closed-loops with their sensory organs also in OLP schemes (dashed curves). Arrows indicate optional whisker (black) or object (magenta) movement; solid arrows indicate movements that are essential for perception; in the sOLP scheme none of the movements is essential in itself, but it is essential that at least one of them will occur in order to activate the receptors. Appropriate experimental paradigms are indicated by green titles; CLP and msOLP schemes can be studied only via active sensing paradigms.The minimal sets for invariant representations (IvRs) of external features, i.e., the components that must be included in any IvR according to each perceptual scheme, are marked by the bluish ellipses. sOLP: internal, sensory only NRs. msOLP: sensory NRs + motor-object-sensory contingencies. CLP: entire motor-object-sensory-motor loops.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Perception as a closed-loop convergence process
eLife 5:e12830.
https://doi.org/10.7554/eLife.12830
