Existing continuous attractor network models and the biological systems where they are found.

A. Left, schematic representation of a spatially embedded set of neurons and their connections. The neural connectivity constrains the patterns of neural co-activation, thus determining the dimensionality and topology of neural activity in the state space. Center, schematic representation of neural activity states, in this case forming a continuous manifold in state space. Right, schematic representation of the states of a (latent) variable in the external world. B,C,D. Examples of integrator circuits. Top row, integration in the oculomotor system. Center row, head direction system. Bottom row shows the grid cell system. B. Schematic representation of CAN models architecture for line, ring and torus attractors. C. Schematic illustration of the continuous manifolds of fixed points predicted and found to exist in the corresponding circuits, adapted from published work [15, 14, 34, 47, 35, 3, 48, 18, 46, 44]. D. Schematic illustration of variable manifolds.

CAN construction and activity manifolds.

A.Left, neural lattice 𝒫 for the Plane (top) and Torus (bottom) attractor networks. Black circles indicate the location of an example neuron, shades of green represent distance from other points on the lattice. Bottom right, inhibitory connectivity strength between the example neurons and all other points on the neural lattice. Middle inset, three examples of valid connectivity kernel functions k. B. Neural manifold in state space (top,right) and activity patterns on the neural lattice 𝒫 (top,left). Bottom row shows three activity patterns with bumps at different locations corresponding to different points on the activity manifold 𝒩.

Quasiperiodic Attractor Networks for Path Integration.

A. Schematic representation of a desired 2D spherical set of fixed points in state space and corresponding connectivity on 𝒫. B. Example activity bump plotted on the neural manifold 𝒫. C. Schematic illustration of Killing vector fields for the sphere manifold, left, and resulting offset connectivity weights on 𝒫, right. D. Schematic illustration of the QAN approach to velocity integration. Left two panels, relationship between changes in the variable on ℳ and on the neural 𝒩 manifold, and associated tangent vectors. Center, each QAN receives a velocity-dependent input based on the tangent vectors at left projected onto its Killing fields, and the activity of all networks is combined. Right: this results in a trajectory in the state-space 𝒩, which corresponds to velocity integration of inputs from ℳ.

Stationary states and manifold topologies of the MADE CANs

A. Desired population activity manifold topology for CANs constructed with MADE for several manifolds (from top to bottom): line, ring, plane, cylinder, torus, sphere, Mobius band and Klein bottle. B. Distance functions over the neural lattice 𝒫 for selected example neurons. C. Low dimensional embedding of the neural activity manifold 𝒩. D. Betti number and persistent homology bar code for each CAN’s neural population states (in 𝒩). E. Left: Activity of one example neuron over 𝒩 (low dimensional embedding). Right: Stationary population activity states form localized bumps on the neural lattice 𝒫.

Dimensionality and attractor dynamics of the MADE CANs.

A, Left, tangent planes approach to computing the intrinsic manifold dimension (schematic) of 𝒩. Right, estimated tangent space dimension for each manifold, which estimates the low intrinsic dimensionality of the CAN networks. B Cumulative manifold variance explained by global PCA analysis: the slow saturation of the curves shows that the linear (embedding) dimension of the manifolds can be large. C Numerical simulations to probe attractor dynamics. Inset: activity manifold, perturbation vector (black) and on-manifold (red) and off-manifold (blue) components of the perturbation. Main plot: Time-varying distance from the starting point in the off-manifold and along-manifold dimensions.

Numerical simulations of path integration performance with MADE path integrators.

A. Tuning curves of single example neurons as a function of the external (latent) variable x. Insets show the manifold topologies of the external variable (red) and neural population states (blue): these pairings might be of identical manifolds, or e.g. a 2D Euclidean manifold in x could be mapped to a cylinder or torus, etc. in the neural population states. B. Example input trajectory (red) and decoded trajectory from the neural population response (blue). C. Decoding error across multiple simulations for various external-neural manifold pairs. Decoding error is shown as percentage of trajectory length over ℳ. Colored boxes show the interquartile range, white lines the mean, circles outliers and vertical lines the 95th percentile confidence interval. D. Same as B but for torus attractors with varying amounts of noise. E. Left: Killing and non-Killing weight offsets for the torus (top) and sphere (bottom). Right: Same as C for integrators correctly constructed with Killing weight offsets, and with the non-Killing weight offsets from the left. F. Same as C for Möbius to Möbius (left) and cylinder to Möbius mappings (right).

Kernel function parameters

Torus CAN activity manifold (top) and persistence diagram (bottom) for varying noise intensity levels (columns).