RatInABox, a toolkit for modelling locomotion and neuronal activity in continuous environments

  1. Tom M George  Is a corresponding author
  2. Mehul Rastogi
  3. William de Cothi
  4. Claudia Clopath
  5. Kimberly Stachenfeld
  6. Caswell Barry
  1. Sainsbury Wellcome Centre, University College London, United Kingdom
  2. Department of Cell and Developmental Biology, University College London, United Kingdom
  3. Department of Bioengineering, Imperial College London, United Kingdom
  4. Google DeepMind, United Kingdom
  5. Columbia University, United States
6 figures, 1 table and 1 additional file


RatInABox is a flexible toolkit for simulating locomotion and neural data in complex continuous environments.

(a) One minute of motion in a 2D Environment with a wall. By default the Agent follows a physically realistic random motion model fitted to experimental data. (b) Premade neuron models include the most commonly observed position/velocity selective cells types (6 of which are displayed here). Users can also build more complex cell classes based on these primitives. Receptive fields interact appropriately with walls and boundary conditions. (c) As the Agent explores the Environment, Neurons generate neural data. This can be extracted for downstream analysis or visualised using in-built plotting functions. Solid lines show firing rates, and dots show sampled spikes. (d) One minute of random motion in a 1D environment with solid boundary conditions. (e) Users can easily construct complex Environments by defining boundaries and placing walls, holes and objects. Six example Environments, some chosen to replicate classic experimental set-ups, are shown here.

The RatInABox random motion model closely matches features of real rat locomotion.

(a) An example 5-min trajectory from the Sargolini et al., 2006. dataset. Linear velocity (Rayleigh fit) and rotational velocity (Gaussian fit) histograms and the temporal autocorrelations (exponential fit) of their time series’. (b) A sampled 5-min trajectory from the RatInABox motion model with parameters matched to the Sargolini data. (c) Figure reproduced from Figure 8D in Satoh et al., 2011 showing 10 min of open-field exploration. ‘Thigmotaxis’ is the tendency of rodents to over-explore near boundaries/walls and has been linked to anxiety. (d) RatInABox replicates the tendency of agents to over-explore walls and corners, flexibly controlled with a ‘thigmotaxis’ parameter. (e) Histogram of the area-normalised time spent in annuli at increasing distances, d, from the wall. RatInABox and real data are closely matched in their tendency to over-explore locations near walls without getting too close.

Advanced features and computational efficiency analysis.

(a) Low temporal-resolution trajectory data (2 Hz) imported into RatInABox is upsampled (‘augmented’) using cubic spline interpolation. The resulting trajectory is a close match to the ground truth trajectory (Sargolini et al., 2006) from which the low resolution data was sampled. (b) Movement can be controlled by a user-provided ‘drift velocity’ enabling arbitrarily complex motion trajectories to be generated. Here, we demonstrate how circular motion can be achieved by setting a drift velocity (grey arrows) which is tangential to the vector from the centre of the Environment to the Agent’s position. (c) Egocentric VectorCells can be arranged to tile the Agent’s field of view, providing an efficient encoding of what an Agent can ‘see’. Here, two Agents explore an Environment containing walls and an object. Agent-1 (purple) is endowed with three populations of Boundary- (grey), Object- (red), and Agent- (green) selective field of view VectorCells. Each circle represents a cell, its position (in the head-centred reference frame of the Agent) corresponds to its angular and distance preferences and its shading denotes its current firing rate. The lower panel shows the firing rate of five example cells from each population over time. (d) A Neurons class containing a feed forward neural network learns, from data collect online over a period of 300 min, to approximate a complex target receptive field from a set of grid cell inputs. This demonstrates how learning processes can be incorporated and modelled into RatInABox. (e) RatInABox used in a simple reinforcement learning example. A policy iteration technique converges onto an optimal value function (heatmap) and policy (trajectories) for an Environment where a reward is hidden behind a wall. State encoding, policy control and the Environment are handled naturally by RatInABox. (f) Compute times for common RatInABox (purple) and non-RatInABox (red) operations on a consumer grade CPU. Updating the random motion model and calculating boundary vector cell firing rates is slower than place or grid cells (note log-scale) but comparable, or faster than, size-matched non-RatInABox operations. Inset shows how the total update time (random motion model and place cell update) scales with the number of place cells.

Appendix 1—figure 1
RatInABox used for a simple neural decoding experiment.

(a) Training (5 min) and testing (1 min) trajectories are sampled in a 1 m square environment containing a small barrier. (b) The firing rates of a population of Ncells=20 cells, taken over the training trajectory, are used to fit a Gaussian Process regressor model estimating position. This decoder is then used to decode position from firing rates on the the unseen testing dataset. Top row shows receptive field for 4 of the 20 cells, bottom row shows decoding estimate (coloured dots) against ground truth (black dots). The process is carried out independently for populations of place cells (left), grid cells (middle) and boundary vector cells (right). (c) Average decoding error against number of cells, note log scale. Error region shows the standard error in the mean over 15 random seeds. A jupyter script demonstrating this experiment is given in the codebase GitHub repository.

Appendix 1—figure 2
RatInABox used in a simple reinforcement learning project.

(a) A schematic of the 1 layer linear network. Using a simple model-free policy iteration algorithm the Agent, initially moving under a random motion policy, learns to approach an optimal policy for finding a reward behind a wall. The policy iteration algorithm alternates between (left) calculating the value function using temporally continuous TD learning and (right) using this to define an improved policy by setting the drift velocity of the Agent to be proportional to the gradient of the value function (a roughly continuous analog for the ϵ-greedy algorithm). (b) 1000 PlaceCells act as a continuous feature basis for learning the value function. (c) The reward is also a (top-hat) PlaceCell, hidden behind the obstructing wall. (d) A ValueNeuron (a bespoke Neurons subclass defined for this demonstration) estimates the policy value function as a linear combination of the basis features (heatmap) and improves this using TD learning. After learning the Agent is able to accurately navigate around the wall towards the reward (middle). Because PlaceCells in RatInABox are continuous and interact adaptively with the Environment when a small gap is opened in the wall place fields corresponding to place cells near this gap automatically bleed through it, and therefore so does the value function. This allows the Agent to find a shortcut to the reward with zero additional training. A jupyter script replicating this project is given in the demos folder GitHub repository.

Author response image 1


Table 1
Default values, keys and allowed ranges for RatInABox parameters.

* This parameter is passed as a kwarg to Agent.update() function, not in the input dictionary. ** This parameter is passed as a kwarg to FeedForwardLayer.add_input() when an input layer is being attached, not in the input dictionary.

ParameterKeyDescription (unit)DefaultAcceptable range
DdimensionalityDimensionality of Environment."2D"["1D","2D"]
Boundary conditionsboundary_conditionsDetermines behaviour of Agent and PlaceCells at the room boundaries."solid"["solid", "periodic"]
Scale, sscaleSize of the environment (m).1.0R+
Aspect ratio, aaspectAspect ratio for rectangular 2D Environments; width = sa, height = s.1.0R+
dxdxDiscretisation length used for plotting rate maps (m).0.01R+
WallswallsA list of internal walls (not the perimeter walls) which will be added inside the Environment. More typically, walls will instead be added with the Env.add_wall() API (m).[]Nwalls×2×2-array/list
BoundaryboundaryInitialise non-rectangular Environments by passing in this list of coordinates bounding the outer perimeter (m).NoneNcorners×2-array/list
HolesholesAdd multiple holes into the Environment by passing in a list of lists, each internal list contains coordinates (min 3) bounding the hole (m).NoneNholes×3×2-array/list
ObjectswallsA list of objects inside the Environment. More typically, objects will instead be added with the Env.add_object() API (m).[]Nobjects×2-array/list
dtdtTime discretisation step size (s).0.01R+
τvspeed_coherence_timeTimescale over which speed (1D or 2D) decoheres under random motion (s).0.7R+
σv (2D) μv (1D)speed_mean2D: Scale Rayleigh distribution scale parameter for random motion in 2D. 1D: Normal distribution mean for random motion in 1D (ms-1).0.082D: R+ 1D: R
σvspeed_stdNormal distribution standard deviation for random motion in 1D (ms-1).0.08R+
τωrotational_velocity_coherence_timeRotational velocity decoherence timescale under random motion (s).0.08R+
σωrotational_velocity_stdRotational velocity Normal distribution standard deviation (rad s-1).2π/3R+
λthigthigmotaxisThigmotaxis parameter.0.50<λthig<1
dwallwall_repel_distanceWall range of influence (m).0.1R+
swalls_repel_strengthHow strongth walls repel the Agent. 0=no wall repulsion.1.0R0+
kdrift_to_random_ strength_ratio*How much motion is dominated by the drift velocity (if present) relative to random motion.1.0R0+
nnNumber of neurons.10Z+
fmaxmax_frMaximum firing rate, see code for applicable cell types (Hz).1.0R
fminmin_frMinimum firing rate, see code for applicable cell types (Hz).0.0fmin<fmax
σηnoise_stdStandard deviation of OU noise added to firing rates (Hz).0.0R+
τηnoise_coherence_timeTimescale of OU noise added to firing rates (s).0.5R+
NamenameA name which can be used to identify a Neurons class."Neurons"Any string
TypedescriptionPlace cell firing function."gaussian"["gaussian", "gaussian_threshold", "diff_of_gaussians", "top_hat", "one_hot"]
wiwidthsPlace cell width parameter; can be specified by a single number (all cells have same width), or an array (each cell has different width) (m).0.2R+
{xiPC}place_cell_centresPlace cell locations. If None, place cells are randomly scattered (m).NoneNone or array of positions (length n)
Wall geometrywall_geometryHow place cells interact with walls."geodesic"["geodesic", "line_of_sight", "euclidean"]
λigridscaleGrid scales (m), or parameters for grid scale sampling distribution.(0.5,1)array-like or tuple
λi-distgridscale_distributionThe distribution from which grid scales are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution_sampler() for list
θiorientationOrientations (rad), or parameters for orientation sampling distribution.(0,2π)array-like or tuple
θi-distorientation_distributionThe distribution from which orientations are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution_sampler() for list
ϕiphase_offsetPhase offsets (rad), or parameters for phase offset sampling distribution.(0,2π)array-like or tuple
ϕi-distphase_offset_distributionThe distribution from which phase offsets are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution_sampler() for list
TypedescriptionGrid cell firing function."three_rectified_cosines"["three_rectified_cosines", "three_shifted_cosines"]
Reference framereference_frameWhether receptive fields are defined in allo- or egocentric coordinate frames"allocentric"["allocentric", "egocentric"]
Arrangement protocolcell_arrangementHow receptive fields are arranged in the environment."random"["random", "uniform_manifold", "diverging_manifold", function()]
dituning_distanceTuning distances (m), or parameters for tuning distance sampling distribution.(0.0,0.3)array-like or tuple
di-disttuning_distance_distributionThe distribution from which tuning distances are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution _sampler() for list
σd,isigma_distanceDistance tuning widths (m), or parameters for distance tuning widths distribution. (By default these give ξ and β)(0.08,12)array-like or tuple
σd,i-distsigma_distance_distributionThe distribution from which distance tuning widths are sampled, if they aren’t manually provided as an array/list. "diverging" is an exception where distance tuning widths are an increasing linear function of tuning distance."diverging"see utils.distribution _sampler() for list
ϕituning_angleTuning angles (), or parameters for tuning angle sampling distribution (degrees).(0.0,360.0)array-like or tuple
ϕi-disttuning_angle_distributionThe distribution from which tuning angles are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution_sampler() for list
σϕ,isigma_angleAngular tuning widths (), or parameters for angular tuning widths distribution (degrees).(10,30)array-like or tuple
σϕ,i-distsigma_angle_distributionThe distribution from which angular tuning widths are sampled, if they aren’t manually provided as an array/list."uniform"see utils.distribution_sampler() for list
dθdthetaSize of angular integration step (°).2.00<dθ<<360
tiobject_tuning_typeTuning type for object vectors, if "random" each OVC has preference for a random object type present in the environment"random""random" or any-int or arrray-like
wall-behaviourwalls_occludeWhether walls occlude objects behind them.Truebool
Other agent, jOther_AgentThe ratinabox.Agent which these cells are selective for.Noneratinabox.Agent
wall-behaviourwalls_occludeWhether walls occlude Agents behind them.Truebool
FieldOfView[X]s() for [X] [BVC,OVC,AVC]
rfovdistance_rangeRadial extent of the field-of-view (m).[0.02,0.4]List of two distances
θfovangle_rangeAngular range of the field-of-view (°).[0,75]List of two angles
δfov0spatial_resolutionResolution of the inner-most row of vector cells (m)0.02
βbetaInverse gradient for how quickly receptie fields increase with distance (for "diverging_manifold" only)5R+
Arrangement protocolcell_arrangementHow the field-of-view receptive fields are constructed"diverging_manifold"["diverging_manifold", "uniform_manifold"]
{Lj}j=1Ninput_layersA list of Neurons classes which are upstream inputs to this layer.[]N-list of Neurons for N1
Activation functionactivation_functionEither a dictionary containing parameters of premade activation functions in utils.activate() or a user-define python function for bespoke activation function.{"activation": "linear"}See utils.activate() for full list
gw_init_scale**Scale of random weight initialisation.1.0R+
bibiasesBiases, one per neuron (optional).[0,....,0]Rn
{Lj}j=1Ninput_layersA list of Neurons classes which are upstream inputs to this layer.[]A list of Neurons
NNNeuralNetworkModuleThe internal neural network function which maps inputs to outputs. If None a default ReLU networ kwith two-hidden layers of size 20 will be used.NoneAny torch.nn.module
llengthscaleLengthscale of the Gaussian process kernel (m).0.1R+
Wall geometrywall_geometryHow distances are calculated and therefore how these cells interact with walls."geodesic"["geodesic", "line_of_sight", "euclidean"]
νθtheta_freqThe theta frequency (Hz).10.0R+
κθkappaThe phase precession breadth parameter.1.0R+
βθbetaThe phase precession fraction.0.50.0<β<1.0

Additional files

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Tom M George
  2. Mehul Rastogi
  3. William de Cothi
  4. Claudia Clopath
  5. Kimberly Stachenfeld
  6. Caswell Barry
RatInABox, a toolkit for modelling locomotion and neuronal activity in continuous environments
eLife 13:e85274.