Introduction

Consider a simple experiment where a single stimulus is presented for a brief moment followed by an unfilled delay interval. With the passage of time, the memory for the event is preserved. According to philosophers (James, 1890; Husserl, 1966; Bergson, 1910), the identity of the stimulus in memory is unchanged, but the memory takes on a new character with the passage of time. In the words of Husserl (1966), with the passage of time, “points of temporal duration recede, as points of a stationary object in space recede when I ‘go away from the object’.” Experimental data from cognitive psychology is consistent with this introspection; participants can separately judge the occurrence and relative time of different stimuli (Hacker, 1980; Hintzman, 2010).

The great mathematical physicist Hermann Weyl introduced the argument above (with analogous requirements for space) as part of an axiomatic derivation of general relativity. Viewed in the light of contemporary discussions about artificial intelligence, we might say that Weyl required a compositional representation of empirical content and time. Given a way to describe empirical content—a what—it must be possible to describe every possible what at every possible when, and vice versa.

The way the early visual system conjunctively codes for what and where information provides a template for how the brain could code for what and when information. Through-out the early visual system there are neurons with similar sensitivity to patterns of light, but with receptive fields in different locations over retinal coordinates. For instance, simple cells respond best to patterns of light with a particular orientation—a what—at a particular location—a where. The pattern of activity of simple cells over all spatial locations gives a conjunctive representation of what is where in the visual field.

Like position in the visual field, physical time is also an ordered continuous dimension. In much the same we might notice that object A is located to the left of object B, we can also remember if event A happened before or after event B. This paper pursues the implications of assuming a conjunctive representation that places events a what on an internal timeline-—-a when—as a fundamental property of working memory for the recent past.

We show that neuronal populations with receptive fields of neurons as a a product of a stimulus term and a temporal term satisfies the requirement of compositionality of what and when (Machens, Romo, & Brody, 2010). Such conjunctive codes of what × when make it straightforward to write out covariance matrices in simple experiments, enabling a description of population dynamics in low dimensional spaces like those widely used in neuroscience research. We then specify a particular choice for temporal basis functions inspired by work in theoretical neuroscience (Shankar & Howard, 2013), cognitive psychology (Howard, Shankar, Aue, & Criss, 2015) and deep networks (Jacques, Tiganj, Sarkar, Howard, & Sederberg, 2022). When projected onto a linear space, these Laplace Neural Manifolds for time generate predictions that resemble empirical results from monkey cortex (Murray et al., 2017; Cueva et al., 2020). Finally, to illustrate that these equations can be instantiated in biological networks, we sketch a continuous attractor neural network model that conjunctively codes for what × when, with temporal basis functions chosen as a Laplace Neural Manifold.

Theoretical Considerations

This paper looks into compositional representations and their implications and implementations in neuroscience. We show that a compositional working memory, coding for what happened when, requires that neurons must have conjunctive receptive fields, decomposable as functions of stimulus and time. Placing minimal restraints on the form of the receptive fields, we show that we can further write the population co-variance as a tensor product of the stimulus (Σ_what) and time (Σ_when) covariances, respectively.

We simulate these populations for simple working memory tasks, where a task variable, distributed either on a ring or on a line, must be remembered for a short amount of time. Depending on our choice of temporal basis set, we observe, through linear dimensionality reduction techniques like PCA, neurally observed traits like stable subspaces and rotational dynamics.

Finally, since the covariance matrix of the population decomposes into Σ_what and Σ_when - and the dimensionality of Σ_what is completely fixed by the task and choice of stimulus RFs, we observe that the subspace spanned by Σ_when controls the dimensionality of the covariance matrix. Measuring the dimensionality of neural trajectories as a function of time should reveal the density of basis functions over the continuous dimension of time. As a natural consequence of smooth basis functions, we see that the dimensionality of Σ_when can grow without bound as a function of the elapsed time, but the growth can decelerate.

Conjunctive receptive fields can support a compositional working memory

Let us assume that we prepare an experiment in which one of several stimuli x, y, z is presented at t = 0 for many trials. We counterbalance the number of presentations of each stimulus and perform all other experimental controls. We record the firing rate over a population of neurons m that reflects the state of a memory with finite duration (Maass, Natschläger, & Markram, 2002). We choose the time between trials to be long enough that we can ignore any carryover from previous trials. Let us describe the population vector expected at a time t after presentation of a particular stimulus as m (, t).

We operationalize Weyl’s requirement that the empirical content of the memory—the what—be unchanged with the passage of time by requiring that x be linearly decodable without knowing t. It is acceptable that the accuracy of decoding changes with the passage of time. However, in order for the “empirical content” to remain fixed we require that the relationships between all pairs of stimuli in the decoding space are preserved at all time points (Fig. 1b).

Figure 1.

Just like the relationships in the what representations should stay independent of the duration when they were experienced, the when representations should also be independent of the identity of the stimulus. The relationships between pairs of time intervals should thus be preserved independent of the stimuli used to mark those intervals (Fig. 1c). If three stimuli are presented in succession (x, y, z) such that the second one follows the first one by a delay of τ₁ and the third one follows the second one by a delay of τ₂, the time interval between the first and third stimuli (τ₃) is just the sum of the first two delays (τ₃ = τ₁ + τ₂). This relationship between the intervals would not change even if the order of the stimuli themselves are shuffled (y, z, x).

This requirement leads to the conclusion that neurons in m (, t) have conjunctive receptive fields as a function of what × when. To see this, let us refer to the column of a linear decoder pointing in the direction of x as x and define m_x(, t) = x′m (, t) and m_y(, t) = y′m (, t). It is acceptable that the magnitude of m_x (, t) and m_y (, t) change as a function of time as information recedes into the past. However for the “empirical content” to be the same, we require that the relationships between all stimuli be constant. This requires that be constant as a function of time for all pairs of stimuli x, y and all stimuli that could be presented . For each cell in m projecting into x with a particular time course, there must be another cell projecting into y with the same time course, up to multiplication by a constant.

We can thus subscript the cells in the population vector m with two indices i and j

where a_ij is just a normalization factor that we will set to 1 for simplicity. Note that this decomposition of m (, t) also satisfies the other constraint to the effect that we can also linearly decode the when, t, without knowing the what, .

Conjunctive, mixed selective receptive fields can create a compositional representation of what and when. We can understand g_i () as a set of receptive fields over whatever dimensions span the stimulus space and h_j(t) as a set of temporal receptive fields.

This requirement would not be compatible with many forms of memory, some of which are widely used in neuroscience. For instance, suppose that m (, t) were maintained as a function of time with an RNN initialized as m (, t = 0) = I() and that then evolved as . Being able to decompose m (, t) as Eq. 1 requires that R be decomposable as

and that I(x) provides the same input to each part of the space spanned by M_when for each stimulus. This is a very strong constraint on recurrent connectivity that reflects a specific structural inductive bias. This choice is not typically made in computational neuroscience network models.

Total covariance Σ decouples into Σ_what and Σ_when

To understand the behavior of neural data, linear dimensional reduction techniques is often used to visualize population vectors as neural trajectories in time. This requires knowledge of the population covariance matrix of the neural data. If the activity of populations of neurons can be decomposed into products of what × when receptive fields, as in Eq. 1, then this results in straightforward expressions for the population covariance matrices.

Lets start with a compositional representation of what happened when, with neurons obeying

where the activity Φ of the neuron (indexed by a what index i and a when index j) describes the receptive field when stimulus is a time τ in the past. If a stimulus is presented at time t = 0, then at time t, the stimulus is τ = t seconds in the past. The function g describes the neuron’s tuning curve over the stimulus dimension, with the preferred stimulus x_i, and the function h describes the temporal receptive field of the neuron, with a time sensitivity parameter τ_j. The stimulus and time preferences can be indexed separately, since the choice of one has no bearing on the other.

Let us calculate the general covariance matrix for populations that obey (3) for some generic set of stimuli indexed by x. We are not restrained to any specific choice of receptive fields for g and h, but only make the assumption that they are normalized over the stimulus and time space respectively

This allows us to write the expectation of the activity over time as a pure function of stimulus, , and the expectation of the activity over stimulus as a pure function of experimental time .

For Σ_when, the covariance over time of and , we get

Note that the first term can be written as 1_ik H_jl, where H is a symmetric matrix which encodes the expectation (over time) of the product of h(τ, τ_j) and h(τ, τ_l).

For Σ_what, the covariance over stimulus of and , we get

Note that the first term can be written as G_ik 1_jl where G is a symmetric matrix which encodes the expectation (over stimuli) of the product of g(, x_i) and g(, x_k).

The overall covariance Σ_ijkl is calculated over both stimuli and time and can be written as

where the second term on the left appears as the expectation of the activity over both stimulus and time and is unity since g and h are normalized appropriately. For the first term, the sum and integral can be separated since g and h are pure functions of stimulus and time. Recognizing that the first term is just a product of the matrix elements G_ik and H_jl, we can generalize this to

We can thus write the overall covariance matrix (Σ) as a Kronecker product

Thus, we see that the overall covariance matrix Σ (computed over both time and stimulus dimensions) neatly decomposes into the tensor product of the covariance matrices for stimulus (Σ_what) and time (Σ_when) respectively, due to the conjunctive coding designed into the Laplace neural manifolds.

Low-dimensional projections of What × When conjunctive representations

A comparison with experimental data requires us to specify the form of the receptive fields for what and when information. For concreteness, we study reasonable choices of temporal receptive fields and examine working memory population dynamics during simple delay experiments that have been used to study cortical populations in monkeys. The choices of stimulus receptive fields are appropriate to the stimuli used in each of two tasks. The temporal receptive fields are chosen consistent with two broad classes - each choice forms a basis set over the time dimension, and they are provably related to one another by a linear operator. We will see that these two closely related choices of temporal basis functions lead to qualitatively different population dynamics as a function of time when projected onto a low-dimensional space.

Laplace Neural Manifolds: Two forms of temporal basis functions

We choose two forms of receptive fields, both of which form a basis set over the continuous dimension of time. One set of basis functions is chosen such that each cell fires in a circumscribed region of time. As a “what” recedes into the past, cells tiling the time axis fire sequentially, much like so-called time cells (Fig. 2f) that have been observed in the hippocampus and elsewhere (Pastalkova, Itskov, Amarasingham, & Buzsaki, 2008; Jin, Fujii, & Graybiel, 2009; Mac-Donald, Lepage, Eden, & Eichenbaum, 2011). Although cells that sequentially activate in time are widely observed in the brain (Tiganj et al., 2018; Akhlaghpour et al., 2016; Parker et al., 2022; Subramanian & Smith, 2024), these are not the only choice of basis functions.

Figure 2.

In addition, we consider monotonic receptive fields in which individual neurons decay as a function of time, but with different time constants (Fig. 2e) (Tsao et al., 2018; Bright et al., 2020; Zuo et al., 2024; Cao et al., 2024; Atanas et al., 2023). Note that when projected into principal component space, decaying firing is indistinguishable from ramping firing.

Exponential temporal receptive fields for Laplace transform

For exponential receptive fields, we set the function h as

where the constant of proportionality is set to ensure the normalization of h specified in Eq. 4. Comparison of these receptive fields with the definition of the Laplace transform shows that at time t this population is closely related to the Laplace transform with real coefficients, of a delta function a time τ in the past. This justifies the claim that h_F is a basis set if τ_j is effectively continuous. We refer to the population with temporal receptive fields as a Laplace population and label this population as F (Fig. 3c).

Figure 3.

The inverse space: Circumscribed temporal receptive fields

To construct circumscribed temporal receptive fields, we use gamma functions (Tank & Hopfield, 1987; De Vries & Principe, 1992) parameterized by a variable k

This receptive field is a product of a power law that grows with τ/τ_j and an exponential that decays with τ/τ_j. With this choice, h(τ, τ_j) goes to zero as τ/τ_j → 0 and also as τ/τ_j → ∞. In between there is a single peak at a time that depends on the choice of τ_j. Choosing a new value of τ_j rescales the function. Thus a population with different values of τ_j will fire sequentially as a triggering stimulus enters the past (Fig. 3d).

It can be shown that the choice of temporal receptive fields in Eq. 10 leads to a deep relationship with neurons with receptive fields described by Eq. 9. Whereas a population of neurons with exponential receptive fields and a variety of time constants encode the Laplace transform with real coefficients, a population of neurons with receptive fields obeying Eq. 10 approximate the inverse Laplace transform, computed using the Post approximation with coefficient k (Shankar & Howard, 2012). This means that the two populations are related to one another via a linear transformation that can readily be computed (Shankar & Howard, 2013). For this reason, we refer to a population with receptive fields chosen as Eq. 10 as an Inverse Laplace Space and label the activity across the population as .

Distributions of time constants for log time

Each Laplace neuron has a time constant τ_j that dictates how fast its activity decays, while each Inverse Laplace neuron has a time constant τ_j which dictates the position of its temporal receptive field. It remains to choose the distribution of time constants τ_j for the populations. Rather than a basis set uniformly tiling τ, we select time constants so that the basis functions tile log τ. This choice has theoretical advantages (Wei & Stocker, 2012; Piantadosi, 2016; Howard & Shankar, 2018) and is in agreement with some neural data (Cao, Bladon, Charczynski, Hasselmo, & Howard, 2022; Guo, Huson, Macosko, & Regehr, 2021).

The distribution of the time constants τ_j is instrumental in defining how the basis functions sample time. In this paper, the τ_j are distributed geometrically. The time constants for the temporal receptive fields are chosen such that the nth time constant is given by

for some positive constant c. If we rearrange the terms a bit, we can express n(τ), the number of cells that tile the time axis from 0 to τ

The number of cells spanning an interval τ thus grows logarithmically with τ. This is also equivalent to saying that the time constants evenly tile log time

(Fig. 3b,c) shows the activity of cells with two kinds of temporal receptive fields firing as a function of experimental time, and as a function of log time (Fig. 3d,e). We will refer to these paired sets of basis functions with time constants chosen as in Eq. 11 as a Laplace Neural Manifold (Daniels & Howard, 2025; Howard, Esfahani, Le, & Sederberg, 2024).

‘What’ Representations

The experimental data reported in Murray et al. (2017) used simple memory tasks using two different types of to-be-remembered stimuli. The oculomotor delayed response (ODR) task requires a monkey to remember the angle of a visual stimulus for a later saccade. The vibrotactile delayed discrimination (VDD) task requires a monkey to remember the frequency of a vibrotactile stimulus for a short time and then compare it to the frequency of a test stimulus. We model the stimulus receptive fields g(x_i, ) in these two experiments as a ring describing the possible angles and a line describing log frequency of the vibrations respectively (Fig. 3a,b).

For the ODR task (Ring), the stimulus specificity of each cell is specified by a symmetric tuning curve function g(θ_i, ), where θ_i is preferred angle of cell i and is the stimulus presented in the experiment. We choose the tuning curves g to follow the von Mises distribution, which serves as a close approximation to the wrapped normal distribution over a circle, normalized approximately. Cells are chosen so that their preferred directions θ_i tile the stimulus space uniformly, and simulated for eight stimuli conditions evenly distributed on the ring (Fig. 3a).

For the VDD task (Line), the stimulus dimension is along a line, and ten stimulus conditions are chosen which tile this frequency space evenly from a minimum frequency of f_min = 10 Hz to a maximum frequency f_max = 30 Hz. The tuning curves coding this space are chosen to be decaying and ramping cells, clamped at these minimum and maximum frequencies, respectively, with rate constants 1/f_i distributed geometrically, such that f_i tile the frequencies between f_min and f_max logarithmically (Fig. 3b), akin to the choice of time constants for ‘When’ representations.

Visualizing conjunctive representations

Fig. 3g-i provides a way to visualize the activity over the network for stimuli chosen on a ring at different time points. Each stimulus presented at the beginning of a trial can be described by an angle. As the stimulus recedes into the past, the “true past” that would be recorded by a perfect observer with, say, a video camera is describable as a point on a cylinder. Fig. 3g provides a cartoon of this “true past” at three moments in external time as a function of log τ. In this depiction, the present is closest to the viewer.

Fig. 3h shows the pattern of activity over the conjunctive what × when representation with Laplace temporal receptive fields as a function of temporal index n. Fig. 3i shows the conjunctive representation with Inverse Laplace temporal receptive fields displayed in the same way. For the Laplace representation, the angle of the stimulus controls the angular location of the edge. As time passes and the stimulus recedes into the remembered past, the edge moves as a function of log τ. The Inverse Laplace space has the same properties, except that the remembered time is represented as a bump of activity. Memory for different kinds of stimuli would require a different structure along the stimulus directions.

Dynamics of conjunctive representations in low-dimensional linear projections

We study the population dynamics of conjunctive representations of what × when with Laplace and Inverse Laplace temporal receptive fields in both ODR and VDD tasks. Analyzing state-space population trajectories of primate PFC neurons, Murray et al. (2017) found stimulus-specific subspaces with persistent ‘What’ representations that held stable even as individual neurons exhibited heterogeneous temporal dynamics. We employ a similar methodology to look at the projections of population-level activity, using Principal Component Analysis (PCA) to account for variance, separately over the stimulus and time space, to generate neural trajectories φ(, t).

Projecting neural trajectories onto stimulus and time Principal Components

To calculate PCA over stimuli, we compute the stimulus covariance Σ_what using the time-averaged delay activity , and use eigendecomposition to extract the first two principal axes and . The projection of the mean-subtracted population activity onto the two principal axes gives us the stimulus principal components

This gives us the first two axes of our neural trajectories. The z axis is constructed to be orthogonal to the stimulus subspace defined by the stimulus axes , which captures the largest component of time variance in the neural activity. To define this, we calculate the time covariance Σ_when over the stimulus-averaged activity , and extract the first principal axis v_t,1. We then orthogonalize this axis to the stimulus subspace by subtracting the stimulus principal axes from it and normalizing

We now project the mean subtracted population activity onto the orthogonalized time axis to get the z axis of the neural trajectories

There now exist neural trajectories φ corresponding to each stimulus condition , which are constructed to have two axes which captures stimulus variance, and a z axis that captures time variance (Fig. 4).

Figure 4.

Laplace cells, resembling temporal context cells, show stimulus-specific stable subspaces

Laplace cells, which have exponentially decaying temporal receptive fields, generate a covariance matrix with smoothly decaying activation along the diagonal (from bottom left to top right), which is the direction of maximum variance (Fig. 4b). The first principal component picks this up to show a monotonically varying trend as well (Suppl. Fig. S1, c2).

When the neural activity of Laplace cells is projected onto this smoothly and monotonically varying temporal principal component (z axis), the stimulus representation thus remains stable, with only a relative scaling of the relative distances between the trajectories for different stimuli. Thus, for both ODR (Fig. 4a) and VDD (Fig. 4c) tasks. Although the populations show strong temporal dynamics and show a heterogeneity of timescales, taken together, the population-wide activity seems to encode a stable representation of the stimulus space.

Inverse Laplace cells, resembling time cells, show rotational dynamics

Inverse Laplace neurons, which have sequential receptive fields tiling a one-dimensional time-line, generate a mostly diagonal covariance matrix (Fig. 4i), decaying monotonically from bottom left to top right. The biggest principal components pick this diagonal direction to explain the most variance, and to tile this diagonal line end up being oscillating sinusoids (Suppl. Fig. S1, d2) with a decaying envelope. When the neural activity is projected onto this oscillating component (z-axis), the trajectories end up rotating in the low-dimensional space.

The population trajectories for both ODR (Fig. 4h) and VDD (Fig. 4j) tasks thus show rotational dynamics, although the activity of the neurons themselves do not intrinsically have a oscillatory component. Sequential activity in the brain has also previously been documented to produce artefactual rotational dynamics in linearly projected low-dimensional neural trajectories (Lebedev et al., 2019; Shinn, 2023).

Even though both populations share the same ‘what’ representation, their activity, when projected onto the stimulus and time principal components, paints very different pictures of their low-dimensional dynamics. This happens due to the choice of temporal basis functions, even though the Laplace and Inverse Laplace representations are simply related by a linear transformation. Both temporal representations are high-rank - as evidenced by the variance in their covariance matrix Σ_when, explained by their principal components falling off smoothly (as seen in Suppl. Fig. S1, c3 and d3). However, the nature of their receptive fields (ramping vs. sequences) gives rise to qualitatively different principal components for time and thus different linear low-dimensional dynamics like stable subspaces or oscillatory dynamics.

Dimensionality of neural trajectories grows with the distribution of time constants

In this section, we examine the dimensionality of the neural space spanned by the population trajectories of neurons with these temporal basis functions. For the choices of temporal basis functions chosen here the dimensionality spanned by the network out to a particular time T depends only on the distribution of time constants. We compute the rank of the overall covariance matrix calculated using times from 0 to T as a measure of dimensionality of neural trajectories.

The rank of the total covariance grows with Σ_when

We have seen in Eq. 8 that the total covariance can be expressed as a tensor product of Σ_when and Σ_what. Because Σ_what is not a function of time, all of the time dependence of the covariance matrix must be carried by Σ_when. This implies that the growth of linear dimensionality of the space should be the same for different kinds of stimuli maintained in working memory. Starting from Eq. 8, using inequalities describing ranks of matrix sums and products, it is possible to establish lower bounds on the rank of the total covariance

We can see that the rank of the the total covariance goes up with the rank of Σ_when, as the rank of Σ_what remains fixed once we choose a particular experimental paradigm and the shape and number of receptive fields tiling the stimulus space. Thus, to determine the dimensionality of the neural representation as a function of the total recording time T, we only need to assess the rank of Σ_when estimated from an experiment with the first T seconds of the delay.

Before stating the result, let us provide an intuition about how the rank of Σ_when should change as a function of T. Consider a pair of Laplace cells (Fig. 5a) with different time constants. If their time constants are less than T, they will have some covariance and contribute to the rank of the covariance matrix. However, consider pairs of cells that both have time constants much greater than T. These cells would both be firing maximally throughout the interval 0 ≤ t ≤ T. The covariance estimated up to time T between these cells is zero. Thus, the pair of cells with time constants much greater than T would not affect the rank of covariance matrix. A similar argument can be made for the Inverse Laplace cells (Fig. 5b).

Figure 5.

Geometrically spaced time-constants result in logarithmic growth in dimensionality of Σ_when with recording time T.

As T changes, the number of cells that contribute to the rank of the covariance matrix goes up like the density of the basis n(T), as defined in Eq. 12. Thus, a logarithmic distribution of time constants, as in Eq. 11, implies that the linear dimensionality of the population measured from 0 to T should grow like log T. Fig. 5c,d show this argument corroborated numerically as the rank of Σ_when, computed for both Laplace and Inverse Laplace neurons grows logarithmically as a function of T. This matches the empirical trend of growth of the cumulative dimensionality with time (Fig. 5e) recorded from cortical regions when performing WM tasks (Cueva et al., 2020).

For generally high recording times, the dimensionality of neural trajectories thus approaches full rank for both Laplace and Inverse Laplace representations. These neurons show nonlinear mixed selectivity, which has previously been shown to be a signature of high-dimensional representations and enables a wide combination of linear readouts (Fusi, Miller, & Rigotti, 2016). Just like neurons in the monkey PFC show high shattering dimensionality (Posani, Wang, Muscinelli, Paninski, & Fusi, 2024; Bernardi et al., 2020) which allows a linear readout to disambiguate different stimulus experimental conditions effectively, Laplace and Inverse Laplace neuronal populations have a high-rank temporal covariance matrix, which allows a wide range of subsets of the continuum of time intervals to be linearly decoded.

Neural Circuit Model for Compositional Working Memory of What × When

A compositional representation of items in time requires conjunctive representations of what × when (Eq. 1). With particular choices for temporal receptive fields (Eqs. 9 and 10) we find a good correspondence with a range of neural data (e.g., Figures 4 and 5). This raises the question of how neural circuits could come to have receptive fields that obey these high-level constraints. One possibility is to simply have an RNN that obeys Eq. 2, with temporal recurrent weights chosen to give appropriate temporal receptive fields (see Liu and Howard (2020)). However, a linear RNN would not be robust to perturbations. To address this question, we introduce a continuous attractor neural network (CANN) to implement the Laplace Neural Manifold. The neurons in this CANN will exhibit conjunctive what × when receptive fields as specified by Eq. 1, with temporal receptive fields given by Eqs. 9 and 10 and logarithmic distribution of time constants as given by Eq. 11.

If the temporal receptive fields across neurons differ by a single parameter that can be mapped onto time, then it is possible to describe the time of a past occurrence of a single stimulus by translating a pattern of activity along a population. As a thought experiment, imagine that we had built a CANN that maintains a memory of the time of past events by constructing a bump of activity that moves at constant velocity along the population as a function of time (W. Zhang, Wu, & Wu, 2022). In this case, the activity of individual neurons would rise and then fall as the bump approaches and then leaves their location along the population. The center of each neuron’s temporal receptive field would depend only on its location along the population. If instead of a bump attractor, the network exhibited an edge across the population that moved at constant velocity, then instead of circumscribed temporal receptive fields, we would find monotonic temporal receptive fields.

The temporal receptive fields described by Eqs. 9 and 10 differ from the receptive fields in our thought experiment. Let us return to a bump attractor where the bump moves at a constant rate. In that case, the center of the temporal receptive fields would vary systematically across neurons, but their width in time would be unaffected. In contrast, in the temporal receptive fields used above (Eqs. 9 and 10) the shape of the temporal receptive fields depend only on τ/τ_j. The temporal receptive fields of different neurons are thus rescaled versions of one another, rather than translated versions of one another. Rescaling time translates log time for the simple reason that log ax = log a + log x. By choosing logarithmic time constants, Eq. 11, we can exploit the fact that changing τ simply translates the representation of a single event over the population (see Fig. 3b,c, bottom). For Laplace neurons, the activity over the network as a function of n takes the form of an edge; for Inverse Laplace neurons, the activity takes the form of a bump. In both cases, local connections favor nearby neurons to be in the same state. The bump network implementing the Inverse Laplace temporal receptive fields works as a standard local excitation/global inhibition CANN. In the edge network, the two ends of the network are clamped to be in an up and down states, so that an edge appears in between. The edge can appear at any location such that the local connections are far from the clamped ends of the population. In contrast to the hypothetical CANN in which the bump moves at a constant rate, Laplace Neural Manifolds require the edge/bump to move at a velocity that decreases as 1/τ (Daniels & Howard, 2025).

In this section, we flesh out this idea to build a CANN that implements a conjunctive working memory using a Laplace Neural Manifold focusing on the ODR task from the previous section. In this task, the monkey must remember the angle of a visual stimulus, allowing a straightforward model for the representation of “what” using a ring attractor CANN (Fig. 6a), a topic that has been extensively studied for decades (Amari, 1977; K. Zhang, 1996; Redish, Elga, & Touretzky, 1996; Kim, Rouault, Druckmann, & Jayaraman, 2017). We couple this to an edge/bump attractor to implement temporal receptive fields (Fig. 6b). In this model, paired CANNs with edge solutions and bump solutions interact with one another. Feedback between the two networks causes the edge/bump complex to move at a velocity that goes down with the location of the edge/bump, thus implementing logarithmically compressed temporal receptive fields. Coupling these two ideas together (Fig. 6c) we can construct a single CANN that maintains a compositional representation of what happened when.

Figure 6.

In this model a stimulus, characterized by an angle describing its location around the circle is presented at time zero. This forms a bump of activity along the ‘what’ dimension of the cylinder, with an edge/bump at one edge of the cylinder. Feedback from the bump moves the edge from one moment to the next. The strength of this connection decreases as a function of position along the network, causing the edge/bump to move more slowly as time progresses (Fig. 6d,e).

Separate CANNs for what and when information

Before describing the combined network, for expository purposes, we first step through the properties of the CANNs for what and when in isolation.

Ring attractor tracks stimulus space ‘What’

To build a ring attractor to maintain information about the location of the visual stimulus, we assume that the neural interactions are Gaussian: , i.e., we impose strong local excitation and weak global excitation over the ring attractor network. The input to each neuron saturates under global activity-dependent inhibition

given the inhibition strength κ and neural density ρ. The dynamics of the synaptic input r(x, t) is governed by:

We have chosen the time constant of the dynamics to be 1 and assume it is much faster than the functional time constants of the receptive fields. When the parameters are chosen properly the ring attractor network can maintain a stable “activity bump” centered around the initial input z: , where a is the width of Gaussian interaction. The constant of proportionality r₀ depends on the parameters of the network, including the value of the inhibition parameter κ. This bump can shift around the ring in response to changes in stimulus identity. The stimulus identity is thus encoded by the manifold coordinate .

Paired edge-bump attractors track temporal space ‘When’

Previous work has shown it is possible to build a continuous attractor model for Laplace/inverse representations of a delta function (Daniels & Howard, 2025) under the assumption that the time constants {τ_n} form a geometric series. In this neural network the relative firing rate r_E(τ_i, t) of all neurons is governed by the dynamics:

where W_EE is the recurrent matrix within the neural population, and ϕ is the nonlinear transfer function ϕ(x) = tanh(gx) with gain g. Again, we assume the time constant of this dynamical system to be 1 and much less than the functional time constants τ_i. By carefully choosing the recurrent matrix WEE and the gating g, the network shows monotonic exponential receptive fields if we provide external input to clamp two edges of the network such that at i = 1 and i = N r_E are constrained to be −1 and 1 respectively. With less careful tuning, any solution that gives an edge that moves at the appropriate speed will give non-exponential monotonic receptive fields (Daniels & Howard, 2025).

To cause the edge to move at a decreasing rate as a function of time we provide a dynamical input to the edge attractor. This input comes via another neuronal population that forms a bump attractor in register with the edge attractor:

where and W_BB is chosen order for the neural network to maintain continuous bumpshaped states. There is a natural one-to-one correspondence between the edge attractor and the bump attractor as the corresponding units i are associated with the same time constant τ_i.

The bump attractor receives input from the edge attractor forming a bump at the position of the edge. Meanwhile, the bump network provides input to the edge attractor to stimulate it to move with a desired speed . By customizing the speed function ν(t) the edge position grows logarithmically in time:

where denotes the edge position at time t, and , t₀ are free parameters that determine the initial condition of the edge. The edge/bump attractors exhibit dynamics that correspond to the temporal receptive fields in Eqs. 9 and 10 as follows:

Thus this edge/bump CANN can implement the temporal receptive fields hypothesized for Laplace Neural Manifolds.

Ring × edge-bump attractors track What × When

The entire model is composed of a series of ring attractors that together form a 2-D manifold that can be visualized as a cylinder. Each neuron in this 2-D manifold is indexed by i and j, indicating its preferred orientation x_i on the ring and time constant τ_j. There are connections between the neurons in each ring of the cylinder, but in this simple model there are no connections between the neurons across different rings. The analytical solution of this continuous ring attractor shows that the height of the “activity bump” of a particular ring depends on the inhibition strength κ:

To generate conjunctive what × when receptive fields, each cylinder is controlled by an edge/bump “spine” that controls the magnitude of the inhibition κ of the ring attractor for each value of τ_j at each moment. To obtain conjunctive what × when receptive fields, the inhibition strength at the ring at τ_j depends on the activation of the spine edge-bump attractor, r_E/r_B. The simulations in Fig. 6d and 6e use inhibition chosen as:

with S ∈ {E, B}. The parameter values in Fig. 6d and e are a_E = −9.60, b_E = 10.71, a_B = −18.10, b_B = 20.31. These values were chosen such that r₀(k) lies within [0,1] for the edge attractor network as τ varies.

Encoding of What × When. Combining previous results, we are able to write out the dynamics of Laplace/Inverse Laplace neural manifolds:

Fig. 6d and 6e show results from this network choosing a specific starting position . Neurons in our 2-D manifold simultaneously encode both the ‘what’ and ‘when’ information. The identity of the stimulus can be readily decoded by observing the center of the bump across the rings, while the timing of the stimulus can be inferred from the location of edge/bump along the ‘when’ axis. This simple circuit model is sufficient to generate conjunctive temporal receptive fields with properties consistent with the properties desired for h_F and .

Discussion

This paper explores the implications of a compositional neural representation of what happened when in working memory. Neurons in a population with such a compositional representation should exhibit conjunctive receptive fields for what and when, as has been reported in many brain regions (Tiganj et al., 2018; Terada, Sakurai, Nakahara, & Fujisawa, 2017; Taxidis et al., 2020). This property makes it straight-forward to write out closed form expressions for the covariance matrix of this population, which in turn allows us to work out the dynamics of the population when studied using standard linear dimensionality reduction techniques. The low-dimensional dynamics depends dramatically on the choice of temporal basis functions, even when the basis functions are related to one another by a simple linear transformation. With conjunctive receptive fields, the dimensionality of the space spanned by the the population is controlled by the rank of the temporal covariance matrix, allowing the density of basis functions to be directly assessed. We show that a logarithmic tiling of time, as proposed by work in cognitive psychology (Fechner, 1860/1912; Stevens, 1957; Murdock, 1960; Luce & Suppes, 2002; Brown, Neath, & Chater, 2007; Balsam & Gallistel, 2009) and supported by evidence from neuroscience (Cao et al., 2022; Guo et al., 2021) provides a reasonable approximation of empirical data. Finally, we sketch out a circuit model using continuous attractor neural networks that exhibits conjunctive what × when receptive fields when a single item is present in working memory.

Network Models in Neuroscience

Early attractor models used persistent activity of neurons to encode a task variable in working memory. The current paper builds on and extends network models that attempt to reconcile stable task representations with heterogeneous temporal dynamics (Druckmann & Chklovskii, 2012; Murray et al., 2017). In those papers, the requirement on temporal dynamics is to leave some stimulus-coding variable invariant to the passage of time. For instance, Druckmann and Chklovskii (2012) required that the summed activity over neurons coding for a particular stimulus is constant. The inverse Laplace cells (Eq. 10) in the current paper can satisfy that requirement with appropriate normalization; because the temporal receptive fields decay monotonically, the Laplace cells (Eq. 9) do not satisfy this requirement.

More broadly, the strategy in those papers is to keep the stimulus representation within a “null space” of the network dynamics. As long as the changes in individual neuronal activity occur along directions in the high-dimensional activity space that do not affect the decoded stimulus representation, the stimulus can be linearly decoded. For instance Murray et al. (2017) used a circuit model that has a mnemonic coding subspace where the input stimulus activates a stable representation, and a non-coding subspace which exhibits temporal dynamics that are orthogonal to the coding subspace.

The major distinction between those previous models and the present paper is that our approach is designed to construct a particular temporal representation whereas previous papers did not place an emphasis on any particular form of temporal dynamics. For instance Cueva et al. (2020) used a low-rank RNN (Beiran, Meirhaeghe, Sohn, Jazayeri, & Ostojic, 2023) (see also Beiran et al. (2023)), to allow temporal dynamics along with a stable stimulus representation. The present approach starts from the prior that memory for the passage of time requires a continuous distribution of time constants, effectively tiling the time axis. If the time constants of a network must be trained by the requirements of a particular task, this leaves the memory less able to express temporal information in different tasks, or even in situations where there is no explicit demand for timing information. For instance, we are aware of the passage of time even when there is no explicit requirement to make a response (Husserl, 1966). Commitment to a timeline of what happened when leaves the memory able to effectively decode any what at any when.

Attractor circuits as a mechanism for slow functional time constants

The proposed circuit model using CANNs provides an existence proof that long functional time constants need not be a consequence of intrinsic time constants of individual neurons (Fransén, Alonso, & Hasselmo, 2002; Fransén, Tahvildari, Egorov, Hasselmo, & Alonso, 2006; Loewenstein & Sompolinsky, 2003; Tiganj, Hasselmo, & Howard, 2015; Saber Marouf et al., 2025) nor of statistical interactions between modes in a nearly-random RNN (Dahmen, Grün, Diesmann, & Helias, 2019; Helias & Dahmen, 2020). Ságodi, Martín-Sánchez, Sokół, and Park (2024) described conditions under which an RNN forms a line attractor (see also Can and Krishnamurthy (2024); Krishnamurthy, Can, and Schwab (2022)). Our results require further that line attractors should be formed for all ‘what’s that can exist in memory. In order to get basis functions that are evenly spaced over log time, the network must have degenerate eigenvalues in geometric series and eigenvectors that are translated versions of one another (Liu & Howard, 2020). Different temporal basis functions, which are apparently both observed in the mammalian brain, also require distinct forms of line attractors. Thus the cognitive requirement for compositional representations of what happened when provide strong high-level constraints on physical models of working memory maintenance.

Cognitive psychology of working memory

In this paper we considered working memory for retention of a single item in continuous time. Cognitive psychologists have studied considerably more complex working memory tasks involving multiple stimuli that are to be remembered. Time per se has little effect on either visual working memory performance (Shin, Zou, & Ma, 2017; Kahana & Sekuler, 2002), nor on verbal working memory performance (Baddeley & Hitch, 1977). However, there is widespread evidence that the amount of information that can be maintained in working memory in a highly veridical manner is finite. For many decades the dominant view has been that working memory maintains a discrete number of items in a discrete number of “slots” (Atkinson & Shiffrin, 1968; Baddeley & Hitch, 1974; Luck & Vogel, 1997). However, more recently, cognitive psychologists’ understanding of the nature of working memory has become much more subtle (Ma, Husain, & Bays, 2014; Brady, Störmer, & Alvarez, 2016; Schurgin, Wixted, & Brady, 2020). In the words of Ma et al. (2014) “working memory might better be conceptualized as a limited resource that is distributed flexibly among all items to be maintained in memory.”

It has long been understood that recurrent attractor neural networks provide a natural means to understand capacity limitations (Grossberg, 1969, 1978). Mutual inhibition controls the number of attractors, or bumps, that can be simultaneously sustained by recurrent activation. One can readily imagine extending the model described in Fig. 6 to allow multiple bumps to coexist. Given a particular structure of the “what” network, depending on how inhibition flows across the network, one can control capacity within a time point (as is typically done in visual working memory experiments) or across time points (as is typically done in verbal working memory experiments. In light of recent experimental work, much care should be taken in allowing stimulus features to cooperate and compete in working memory.

The wiring requirements necessary to extend this approach such that all possible stimuli can be maintained in working memory through continuous time seem daunting. For instance, what if the task also required memory for the color of the stimuli as well as their location? Or the orientation and color of a bar at an angual location? Or the identity of a letter at an angular location? The circuit would have to either be extremely complicated a priori or be able to be dynamically configured for task-relevant information.

Perhaps recent work on gated RNNs provides a way for these continuous attractor networks to dynamically form in response to task demands (Krishnamurthy et al., 2022; Can & Krishnamurthy, 2024). In this way, gated input would dynamically specify the features that can be maintained in a continuous attractor network for what information.

Time and space and number

One could have easily generalized the compositionality argument for what × when to what × spatial position. Indeed, Weyl (1922) used precisely the same argument for space in developing an axiomatic development of space-time in general relativity. The requirement of compositional representations for what and when leads naturally to conjunctive neural codes.

In addition to physical space, one might have made analogous arguments for any number of continuous variables. Conjunctive codes for what × when is thus a special case of mixed selectivity, which has been proposed as a general property for neural codes (Rigotti et al., 2013) and has been observed for many different variables in neural representations (Mante, Sussillo, Shenoy, & Newsome, 2013; Ward, Tan, & Grenfell-Essam, 2010; Johnston, Palmer, & Freedman, 2020; Dang, Jaffe, Qi, & Constantinidis, 2021; Wallach, Melanson, Longtin, & Maler, 2022). Conjunctive receptive fields are also closely related to gain fields which were proposed for coordinate transformations and efficient function learning (Pouget & Sejnowski, 1997; Salinas & Abbott, 2001).

Numerosity is a variable that can be composed (in principle) with any kind of object. Mammals (Gallistel & Gelman, 1992; Dehaene & Brannon, 2011) and other animals (Kirschhock & Nieder, 2023) are equipped with a number sense. This number sense appears to be computed over a neural scale that maps receptive fields onto the log of numerosity (Nieder & Miller, 2003; Dehaene, 2003; Nieder & Dehaene, 2009), presumably using a set of basis functions distributed over a logarithmic number line.

Theoretically-driven neural data analysis techniques

The appearance of population dynamics projected onto a low-dimensional PCA space changed dramatically depending on how we chose temporal basis functions (Fig. 4). Even though the temporal basis functions are simply related by a linear transformation, one might have concluded that they reflect very different coding schemes if one only looked at the PCA results. For instance, one may have concluded that the Laplace representation shows a stable subspace with persistent firing (Murray et al., 2017; Constantinidis et al., 2018) whereas the inverse space shows sustained delay-period coding without persistent firing (King & Dehaene, 2014; Lundqvist, Herman, & Miller, 2018).

This is another example that extreme caution should be exercised in using linear dimensionality reduction techniques in neuroscience (Lebedev et al., 2019; De & Chaudhuri, 2023; Shinn, 2023). Jazayeri and Ostojic (2021) write that “… without concrete computational hypotheses, it could be extremely challenging to interpret measures of dimensionality.” If neural codes use basis functions tiling continuous variables out in the world, then the linear dimensionality of the neural code is in principle unbounded (Manley et al., 2024). Depending on the choice of basis functions, the principle components of the population may, or may not be readily interpretable, even if we know a priori the continuous variables that are being coded by the population.

A number of well-studied non-linear dimensionality reduction techniques exist (Belkin & Niyogi, 2003; Kohli, Cloninger, & Mishne, 2021; Tenenbaum, de Silva, & Langford, 2000; Roweis & Saul, 2000; McInnes, Healy, & Melville, 2018; Van der Maaten & Hinton, 2008). Although still vastly outnumbered by data analyses using linear dimensionality reduction techniques, some neuroscience work has made use of non-linear dimensionality reduction (Chaudhuri, Gerçek, Pandey, Peyrache, & Fiete, 2019; Nieh et al., 2021) (see Langdon, Genkin, & Engel, 2023 for a review). Resolving the empirical question of the generality of compositional codes using conjunctive basis functions will require careful experimentation and development of new data analysis tools.

Given a hypothesis about relevant variables, it should be possible to distinguish if the decomposition of the covariance matrix as asserted in Eq. 8 is valid. This would establish that a conjunctive code exists. Proximity of the parameters describing the receptive fields, e.g., s_i, , n, provide a natural metric for proximity along the population manifold. However, these parameters need to be recovered from the data, preferably without a strong prior on the form of the functions describing the receptive fields.