Many biological systems can be thought of as networks in which a large number of elements, called ‘nodes’, are connected to each other. The brain, for example, is a network of interconnected neurons, and the changing activity patterns of this network underlie our experience of the world around us. Within the brain, different parts can process information at different speeds: sensory areas of the brain respond rapidly to the current environment, while the cognitive areas of the brain, involved in complex thought processes, are able to gather information over longer periods of time. However, it has been largely unknown what properties of a network allow different regions to process information over different timescales, and how variations in structural properties translate into differences in the timescales over which parts of a network can operate.

Now Chaudhuri et al. have addressed these issues using a simple but ubiquitous class of networks called linear networks. The activity of a linear network can be broken down into simpler patterns called eigenvectors that can be combined to predict the responses of the whole network. If these eigenvectors ‘map’ to different parts of the network, this could explain how distinct regions process information on different timescales.

Chaudhuri et al. developed a mathematical theory to predict what properties would cause such eigenvectors to be separated from each other and applied it to networks with architectures that resemble the wiring of the brain. This revealed that gradients in the connectivity across the network, such that nodes share more properties with neighboring nodes than distant nodes, combined with random differences in the strength of inter-node connections, are general motifs that give rise to such separated activity patterns. Intriguingly, such gradients and randomness are both common features of biological systems.

A major challenge in the study of neural circuits, and complex networks more generally, is understanding the relationship between network structure and patterns of activity or possible functions this structure can subserve (Strogatz, 2001; Newman, 2003; Honey et al., 2010; Sporns, 2011). A number of neural networks show a diversity of time constants, namely different nodes (single neurons or local neural groups) in the network display dynamical activity that changes on different timescales. For instance, in the mammalian brain, long integrative timescales of neurons in the frontal cortex (Romo et al., 1999; Wang, 2001; Wang, 2010) are in striking contrast with rapid transient responses of neurons in a primary sensory area (Benucci et al., 2009). Furthermore, even within a local circuit, a diversity of timescales may coexist across a heterogeneous neural population. Notable recent examples include the timescales of reward integration in the macaque cortex (Bernacchia et al., 2011), and the decay of neural firing rates in the zebrafish (Miri et al., 2011) and macaque oculomotor integrators (Joshua et al., 2013). While several models have been proposed, general structural principles that enable a network to show a diversity of timescales are lacking.

Studies of the cortex have revealed that neural connectivity decays rapidly with distance (Holmgren et al., 2003; Markov et al., 2011; Perin et al., 2011; Levy and Reyes, 2012; Markov et al., 2014; Ercsey-Ravasz et al., 2013) as does the magnitude of correlations in neural activity (Constantinidis and Goldman-Rakic, 2002; Smith and Kohn, 2008; Komiyama et al., 2010). This characteristic is apparent on multiple scales: in the cerebral cortex of the macaque monkey, both the number of connections between neurons in a given area and those between neurons across different brain areas decay rapidly with distance (Markov et al., 2011, 2014). Intuitively, local connectivity may suggest that the timescales of network activity are localized, by which we mean that nodes that respond with a certain timescale are contained within a particular region of the network. Such a network would show patterns of activity with different temporal dynamics in disparate regions. Surprisingly, this is not always true and, as we show, additional conditions are required for localized structure to translate into localized temporal dynamics.

We study this structure–function relationship for linear networks of interacting nodes. Linear networks are used to model a variety of physical and biological networks, especially those where inter-node interactions are weighted (Newman, 2010). Most dynamical systems can be linearized around a point of interest, and so linear networks generically emerge when studying the response of nonlinear networks to small perturbations (Strogatz, 1994; Newman, 2010). Moreover, for many neurons the dependence of firing rate on input is approximately threshold-linear over a wide range (Ahmed et al., 1998; Ermentrout, 1998; Wang, 1998; Chance et al., 2002), and linear networks are common models for the dynamics of neural circuits (Dayan and Abbott, 2001; Shriki et al., 2003; Vogels et al., 2005; Rajan and Abbott, 2006; Ganguli et al., 2008; Ganguli et al., 2008; Murphy and Miller, 2009; Miri et al., 2011).

The activity of a linear network is determined by a set of characteristic patterns, called eigenvectors (Rugh, 1995). Each eigenvector specifies the relative activation of the various nodes. For example, in one eigenvector the first node could show twice as much activity as the second node and four times as much activity as the third node, and so on. The activity of the network is the weighted sum of contributions from the eigenvectors. The weight (or amplitude) of each eigenvector changes over time with a timescale determined by the eigenvalue corresponding to the eigenvector. The network architecture determines the eigenvectors and eigenvalues, while the input sets the amplitudes with which the various eigenvectors are activated. In Figure 1, we illustrate this decomposition in a simple schematic network with three eigenvectors whose amplitudes change on a fast, intermediate and slow timescale respectively.

Figure 1

Download asset Open asset

In general, the eigenvectors are poorly segregated from each other: each node participates significantly in multiple eigenvectors and each eigenvector is spread out across multiple nodes (Trefethen and Embree, 2005). Consequently, timescales are not segregated, and a large number of timescales are shared across nodes. Furthermore, if the timescales have largely different values, certain eigenvectors are more persistent than others and dominate the nodes at which they are present. If these slow timescales are spread across multiple nodes, they dominate the network activity and the nodes will show very similar temporal dynamics. This further limits the diversity of network computation.

In this paper, we begin by observing that rapidly-decaying connectivity by itself is insufficient to give rise to localized eigenvectors. We then examine conditions on the network-coupling matrix that allow localized eigenvectors to emerge and build a framework to calculate their shapes. We illustrate our methods with simple examples of neural dynamics. Our examples are drawn from Neuroscience, but our results should be more broadly applicable for understanding network dynamics and the relationship between the structure and function of complex systems.

We study linear neural networks endowed with a connection matrix W (j,k) (‘Methods’, Equation 9), which denotes the weight of connection from node k to node j. For a network with N nodes, the matrix W has N eigenvectors and N corresponding eigenvalues. The time constant associated with the eigenvector v_λ is $1/\mathfrak{R}\mathfrak{e}\left(-\lambda \right)$, where λ is the corresponding eigenvalue (‘Methods’, Equation 11). This time constant is present at all nodes where the eigenvector has non-zero magnitude. We say an eigenvector is delocalized if its components are significantly different from 0 for most nodes. In this case, the corresponding timescale is spread across the entire network. On the other hand, if an eigenvector is localized then v_λ (j) ≈ 0 except for a restricted subset of spatially contiguous nodes, and the timescale $1/\mathfrak{R}\mathfrak{e}\left(-\lambda \right)$ is confined to a region of the network. If most or all of the eigenvectors are localized, then different nodes show separated timescales in their dynamical response to external stimulation.

Note that even if the eigenvectors are localized, a large proportion of network nodes could respond to a given input, but they would do so with disparate temporal dynamics. Conversely, even if the eigenvectors are delocalized, a given input could still drive some nodes much more strongly than others. However, the temporal dynamics of the response will be very similar at the various nodes even if the magnitudes are different.

Consider a network with nodes arranged in a ring, as shown in the top panel of Figure 2A. The connection strength between nodes decays with distance according to

where, l_c is set to be 1 node so that the connectivity is sharply localized spatially. In Figure 2B we plot the absolute values and real parts of three sample eigenvectors. The behavior is typical of all eigenvectors: despite the local connectivity they are maximally delocalized and each node contributes with the same relative weight to each eigenvector (its absolute value is constant, while its real and imaginary parts oscillate across the network). As shown in Figure 2C, the timescales of decay are very similar across nodes.

Figure 2

Download asset Open asset

As known from the theory of discrete Fourier transforms, such delocalized eigenvectors are generically seen if the connectivity is translationally invariant, meaning that the connectivity profile is the same around each node (see mathematical appendix [Supplementary file 1], Section 1 or standard references on linear algebra or solid-state physics [Ashcroft and Mermin, 1976]). In this case the jth component of the eigenvector v_λ is

where, ω/2π is the oscillation frequency (which depends on λ) and i is the imaginary unit (i² = −1). Thus local connectivity is insufficient to produce localized eigenvectors.

We developed a theoretical approach that enables us to test network architectures that yield localized eigenvectors. Although in general it is not possible to analytically calculate all timescales (eigenvalues) of a generic matrix, the theory allows us to predict which timescales would be localized and which would be shared. For the localized timescales, it yields a functional form for the shape of the corresponding localized eigenvectors. Finally, the theory shows how changing network parameters promotes or hinders localization. For a further discussion of these issues, see Section 2 of the mathematical appendix (Supplementary file 1).

For a given local connectivity, W (j,k), we postulate the existence of an eigenvector v_λ that is well localized around some position, j₀, defined as its center. We then solve for the detailed shape (functional form) of our putative eigenvector and test whether this shape is consistent with our prior assumption on v_λ. If so, this is a valid solution for a localized eigenvector.

Specifically, if v_λ is localized around j₀ then v_λ (k) is small when $\left|k-j_0\right|$ is large. We combine this with the requirement of local connectivity, which implies that W (j,k) is small when $\left|j-k\right|$ is large, and expand W and v_λ to first-order in $\left|k-j_0\right|$ and $\left|j-k\right|$ respectively. With this approximation, we solve for v_λ across all nodes and find (‘Methods’ and mathematical appendix [Supplementary file 1], Section 2)

The eigenvector is a modulated Gaussian function, centered at j₀. The characteristic width is α, such that a small α corresponds to a sharply localized eigenvector. Note that j₀ and ω depend on the particular timescale (or eigenvalue, λ) being considered and hence, in general, α² will depend on the timescale under consideration. For v_λ to be localized, the real part of α² must be positive when evaluated at the corresponding timescale. In this case, v_λ is consistent with our prior assumption, and we accept it as a meaningful solution.

Our theory gives the dependence of the eigenvector width on network parameters and on the corresponding timescale. In particular, α depends inversely on the degree of local heterogeneity in the network, so that greater heterogeneity leads to more tightly localized eigenvectors (see appendix [Supplementary file 1], Section 2). ω is a frequency term that allows v_λ to oscillate across nodes, as in Equation 1. As shown later, the method is general and a second-order expansion can be used when the first-order expansion breaks down. In that case the eigenvector shape is no longer Gaussian.

We now apply this theory to models of neural dynamics in the mammalian cerebral cortex. We use connectivity that decays exponentially with distance (Markov et al., 2011, 2014; Ercsey-Ravasz et al., 2013) but our analysis applies to other forms of local connectivity.

Localization in a network with a gradient of local connectivity

Our first model architecture is motivated by observations that as one progresses from sensory to prefrontal areas in the primate brain, neurons receive an increasing number of excitatory connections from their neighbors (Wang, 2001; Elston, 2007; Wang, 2008). We model a chain of nodes (i.e., neurons, networks of neurons or cortical areas) with connectivity that decays exponentially with distance. In addition, we introduce a gradient of excitatory self-couplings along the chain to account for the increase in local excitation.

The network is shown in Figure 3A and the coupling matrix W is given by

(3) $W\left(j,k\right)=\{\begin{array}{cc}{\mu }_0+{\text{Δ}}_{r}j& \text{for} j=k (\text{self}-\text{coupling})\\ {\mu }_f{e}^{-\left(j-k\right)/l_c}& \text{for} j>k (\text{feedforward}\hspace{0.17em}\text{connections})\\ {\mu }_b{e}^{\left(j-k\right)/l_c}& \text{for} j<k (\text{feedback}\hspace{0.17em}\text{connections})\end{array}.$

Figure 3 with 1 supplement see all

Download asset Open asset

The self-coupling includes a leakage term (μ₀ < 0) and a recurrent excitation term that increases along the chain with a slope Δ_r. Nodes higher in the network thus have stronger self-coupling. Connection strengths have a decay length l_c. μ_f scales the overall strength of feedforward connections (i.e., connections from early to late nodes in the chain) while μ_b scales the strength of feedback connections. In general we set μ_f > μ_b.

If the gradient of self-coupling (Δ_r) is strong enough, some of the eigenvectors of the network will be localized. As the gradient becomes steeper this region of localization expands. Our theory predicts which eigenvectors will be localized and how this region expands as the gradient becomes steeper (Figure 3—figure supplement 1).

By applying the theory sketched in the previous section (and developed in detail in the appendix [Supplementary file 1]), we find that the value of the eigenvector width for the localized eigenvectors (α in Equation 2) is equal to (see Section 3 of Supplementary file 1)

(4) ${\alpha }^2=\frac{{\mu }_f-{\mu }_b}{2{\text{Δ}}_r\left(1+\text{cosh}\left(\frac{1}{l_c}\right)\right)}.$

This equation asserts that α² is inversely proportional to the gradient of local connectivity, Δ_r, so that a steeper gradient leads to sharper localization, and α² increases with increasing connectivity decay length, l_c. Note that in this case the eigenvector width is independent of the location of the eigenvector (or the particular timescale).

In Figure 3B, we plot sample eigenvectors for a network with a weak gradient, where localized and delocalized eigenvectors coexist. We also plot the analytical prediction for the localized eigenvectors, which fits well with the numerical simulation results. For more details on this network see Figure 3—figure supplement 1. In Figure 3C, we plot sample eigenvectors for a network with a strong enough gradient that all eigenvectors are localized. As shown in Figure 3D, all the remaining eigenvectors of this network are localized. In Figure 3E, we plot the decay of this network’s activity from a uniform initial condition; as predicted from the structure of the eigenvectors, decay time constants increase up the chain.

With a strong gradient of self-coupling, Equation 4 holds for all eigenvectors except those at the end of the chain, where edge effects change the shape of the eigenvectors. These eigenvectors are still localized, at the boundary, but are no longer Gaussian and appear to be better described as modulated exponentials. Equation 4 also predicts that eigenvectors become more localized as feedforward and feedback connection strengths approach each other. This is counter-intuitive, since increasing feedback strength should couple nodes more tightly. Numerically, this prediction is confirmed only when μ_f − μ_b is not close to 0. As seen in Figure 4, when μ_f − μ_b is small, the eigenvector is no longer Gaussian and instead shows multiple peaks. Strengthening the feedback connections leads to the emergence of ripples in the slower modes that modulate the activity of the earlier, faster nodes. While the first-order approximation of the shape of v_λ breaks down in this regime, Equation 4 is locally valid in that the largest peak sharpens with increasing symmetry, as seen in Figure 4B.

Figure 4

Download asset Open asset

We extend our expansion to second-order in v_λ (appendix [Supplementary file 1], Sections 5 & 6) to predict that the eigenvector is given by

(5) ${\mathit{v}}_{\mathit{\lambda }}\left(j\right)=e^{{\beta }_2\left(j-j_0\right)}\text{Ai}\left(\frac{{\beta }_1\left(j-j_0\right)+{\beta }_2^2}{{\beta }_1^{2/3}}\right)e^{i\omega j}$

with

(6) ${\beta }_1=\frac{{\text{Δ}}_r\text{csch}{\left(\frac{1}{2l_c}\right)}^4\text{sinh}{\left(\frac{1}{l_c}\right)}^3}{\left({\mu }_f+{\mu }_b\right)}\hspace{1em}\text{and}\hspace{1em}{\beta }_2=\frac{\left({\mu }_f-{\mu }_b\right)\text{coth}\left(\frac{1}{2l_c}\right)}{\left({\mu }_f+{\mu }_b\right)}$

where, Ai is the first Airy function (Olver, 2010). The eigenvector is the product of an exponential and an Airy function and this product is localized when the exponential is steep (Figure 4A). The steepness of the exponential depends on μ_f − μ_b. When this difference is small the exponential is shallow and the trailing edge of the product is poorly localized. Figure 4B shows that this functional form accurately predicts the results from numerical simulations, except when the eigenvector is almost completely delocalized.

These results reveal that an asymmetry in the strength of feedforward and feedback projections can play an important role in segregation of timescales in biological systems.

The second-order expansion demonstrates that the approach is general and can be extended as needed. While the first-order expansion in v_λ generically gives rise to modulated Gaussians, the functional form of the eigenvectors from a second-order expansion depends on the connectivity (appendix [Supplementary file 1], Section 5) and, in general, the asymptotic decay is slower than that of a Gaussian.

Localization in a network with a gradient of connectivity range

The previous architecture was a chain of nodes with identical inter-node connectivity but varying local connectivity. We now consider a contrasting architecture: a chain with no self-coupling but with a location-dependent bias in inter-node connectivity. We build this model motivated by the intuitive notion that nodes near the input end of a network send mostly feedforward projections, while nodes near the output send mostly feedback projections. The network architecture is shown in Figure 5A.

Figure 5

Download asset Open asset

Connectivity decays exponentially, as in the previous example, but the decay length depends on position. Moving along the chain, feedforward decay length decreases while feedback decay length increases:

(7) $W\left(j,k\right)=\{\begin{array}{cc}{\mu }_0& \text{for} j=k (\text{self}-\text{coupling})\\ {\mu }_f{e}^{-\left(f_0+f_{1}k\right)\left(j-k\right)}& \text{for} j>k (\text{feedforward}\hspace{0.17em}\text{connections})\\ {\mu }_b{e}^{\left(b_0-b_{1}k\right)\left(j-k\right)}& \text{for} j<k (\text{feedback}\hspace{0.17em}\text{connections})\end{array}$

The parameters f₀, f₁, b₀, and b₁ control the location-dependence in decay length, μ₀ is the leakage term, and μ_f and μ_b set the maximum strength of feedforward and feedback projections. We also add a small amount of randomness to the connection strengths.

As before we calculate the eigenvector width, α. In this case, for a wide range of the parameters in Equation 7, α² is positive and approximately constant for all eigenvectors. Therefore, all eigenvectors are localized and have approximately the same width (appendix [Supplementary file 1], Section 4). Four eigenvectors are plotted in Figure 5B along with theoretical predictions. Figure 5C shows all of the eigenvectors on a heat map and demonstrates that all are localized. The fastest and slowest timescales are localized to the earlier nodes while the intermediate timescales are localized towards the end of the chain. The earlier nodes thus show a combination of very fast and very slow time courses, whereas the later nodes display dynamics with an intermediate range of timescales. Such dynamics present a salient feature of networks with opposing gradients in their connectivity profile. In Figure 5D, we plot the decay of network activity from a uniform initial condition; note the contrast between nodes early and late in the chain.

While the eigenvectors are all localized, different eigenvectors tend to cluster their centers near similar locations. Near those locations, nodes may participate in multiple eigenvectors, implying that time constants are not well segregated. This is a consequence of the architecture: nodes towards the edges of the chain project most strongly towards the center, so that small perturbations at either end of the chain are strongly propagated inward. The narrow spread of centers (the overlap of multiple eigenvectors) reduces the segregation of timescales that is one benefit of localization. We find that adding a small amount of randomness to the system spreads out the eigenvector centers without significantly changing the shape. This approach is more robust than fine-tuning parameters to maximally spread the centers, and seems reasonable in light of the heterogeneity intrinsic to biological systems (Raser and O’Shea, 2005; Barbour et al., 2007). Upon adding randomness, most eigenvectors remain Gaussian while a minority are localized but lose their Gaussian shape.

The significant overlap of the eigenvectors means that the eigenvectors are far from orthogonal to each other. Such matrices, called non-normal matrices, can show a number of interesting transient effects (Trefethen and Embree, 2005; Goldman, 2009; Murphy and Miller, 2009). In particular we note that the dynamics of our example network show significant initial growth before decaying, as visible in the scale of Figure 5D.

Randomness and diversity

As observed in the last section, the heterogeneity intrinsic to biological systems can play a beneficial role in computation. Indeed, sufficient randomness in local node properties has been shown to give localized eigenvectors in models of physical systems with nearest-neighbor connectivity, and the transition from delocalized to localized eigenvectors has been suggested as a model of the transition from a conducting to an insulating medium (Anderson, 1958; Abou-Chacra et al., 1973; Lee, 1985). A similar mechanism should apply in biological systems. We numerically explore eigenvector localization in a network with exponentially-decaying connectivity and randomly distributed self-couplings.

The network connection matrix is given by

(8) $W\left(j,k\right)=\{\begin{array}{cc}{\mu }_0+\mathcal{N}\left(0,{\sigma }^2\right)& \text{for} j=k\\ {\mu }_c{e}^{-\left|j-k\right|/l_c}& \text{for} j\ne k\end{array},$

where, $\mathcal{N}\left(0,{\sigma }^2\right)$ is drawn from a normal distribution with mean zero and variance σ².

As σ² increases, the network shows a transition to localization. This transition is increasingly sharp and occurs at lower values of σ as the network gets larger. Figure 6 shows a network with sufficient randomness for the eigenvectors to localize, with sample eigenvectors shown in Figure 6B. These show a variety of shapes and are no longer well described by Gaussians. Importantly, there is no longer a relationship between the location of an eigenvector and the timescale it corresponds to (Figure 6C). Thus while each timescale is localized, a variety of timescales are present in each region of the network, and each node will show a random mixture of timescales. This is in contrast to our previous examples, which have a spatially continuous distribution of time constants. The random distribution of time constants is also observed in the decay from a uniform initial conditions, as shown in Figure 6D.

Figure 6

Download asset Open asset

Local connectivity is insufficient to create localized temporal patterns of activity in linear networks. A network with sharply localized but translationally invariant connectivity has delocalized eigenvectors. This implies that distant nodes in the network have similar temporal activity, since they share the timescales of their dynamics. Breaking the invariance can give rise to localized eigenvectors, and we study conditions that allow this. We develop a theory to predict the shapes of localized eigenvectors and our theory generalizes to describe eigenvectors that are only partially localized and show multiple peaks. A major finding of this study is the identification of two network architectures, with either a gradient of local connectivity or a gradient of long-distance connection length, that give rise to activity patterns with localized timescales.

Our approach to eigenvector localization is partly based on Trefethen and Embree (2005); Trefethen and Chapman (2004). The authors study perturbations of translationally invariant matrices and determine conditions under which eigenvectors are localized in the large-N limit. We additionally assume that the connectivity is local, since we are interested in matrices that describe connectivity of biological networks. This allows us to calculate explicit functional forms for the eigenvectors.

We stress that the temporal aspect of the network dynamics should not be confused with selectivity across space in a neural network. Even if temporal patterns are localized, a large proportion of network nodes may be active in response to a given input, albeit with distinct temporal dynamics. Conversely, even if temporal patterns are delocalized, nodes show similar dynamics yet may still be highly selective to different inputs and any stimulus could primarily activate only a small fraction of nodes in the network.

Our results are particularly relevant to understanding networks that need to perform computations requiring a wide spread of timescales. In general, input along a fast eigenvector decays exponentially faster than input along a slow eigenvector. To see this, consider a network with a fast and a slow timescale ($1/\left|{\lambda }_{fast}\right|$ and $1/\left|{\lambda }_{slow}\right|$), and having initial condition with components a_fast and a_slow along the fast and the slow eigenvectors respectively. As shown in Equation 11, the network activity will evolve as $a_{fast}{e}^{-\left|{\lambda }_{fast}\right|t}+a_{slow}{e}^{-\left|{\lambda }_{slow}\right|t}$. For a node to show a significant fast timescale in the presence of a slower, more persistent timescale, the contribution of this slow timescale to the node must be small. This can happen in two ways, corresponding to the terms of Equation 10. If the input contributes little to the slower eigenvectors then their amplitudes will be small at all nodes. This requires fine-tuned input (exponentially smaller along the slow eigenvectors) and means that the slow timescales do not contribute significantly to any node. Alternately, as in the architectures we propose, the slow eigenvectors could be exponentially smaller at certain nodes; these nodes will then show fast timescales for most inputs, with a small slow component.

The architecture with a gradient of local connectivity (Figure 3) may explain some observations in the larval zebrafish oculomotor system (Miri et al., 2011). The authors observed a wide variation in the time constants of decay of firing activity across neurons, with more distant neurons showing a greater difference in time constants. They proposed a model characterized by a chain of nodes with linearly-decaying connectivity and a gradient of connection strengths, and found that different nodes in the model showed different timescales. Furthermore, the introduction of asymmetry to connectivity (with feedback connections weaker than feedforward connections) enhanced the diversity of timescales. This effect of asymmetry was also seen in an extension of the model to the macaque monkey oculomotor integrator (Joshua et al., 2013). Our work explains why such architectures allow for a diversity of timescales, and we predict that such gradients and asymmetry should be seen experimentally.

With a gradient of local connections, time constants increase monotonically along the network chain. By contrast, with a gradient of connectivity length (Figure 5), the relationship between timescales and eigenvector position is lawful but non-monotonic, as a consequence of the existence of two gradients (feedforward connectivity decreases while feedback increases along the chain). The small amount of randomness added to this system helps segregate the timescales across the network, while only mildly affecting the continuous dependence of eigenvector position on timescale. This suggests that randomness may contribute to a diversity of timescales.

The connection between structural randomness and localization is well known in physical systems (Anderson, 1958; Abou-Chacra et al., 1973; Lee, 1985). We applied this idea to a biological context (Figure 6), and showed that localization can indeed emerge from sufficiently random node properties. However, in this case nearby eigenvectors do not correspond to similar timescales. A given timescale is localized to a particular region of the network but a similar timescale could be localized at a distant region and, conversely, a much shorter or longer timescale could be localized in the same part of the network. Thus, the timescales shown by a particular node are a random sample of the timescales of the network.

Chemical gradients are common in biological systems, especially during development (Wolpert, 2011), and structural randomness and local heterogeneity are ubiquitous. We predict that biological systems could show localized activity patterns due to either of these mechanisms or a combination of the two. Furthermore, local randomness can enhance localization that emerges from gradients or long-range spatial fluctuations in local properties. We have focused on localization that yields a smooth relationship between timescale and eigenvector position; such networks are well-placed to integrate information at different timescales. However, it seems plausible that biological networks have evolved to take advantage of randomness-induced localization, and it would be interesting to explore the computational implications of such localization. It could also be fruitful to explore localization from spatially correlated randomness.

An influential view of complexity is that a complex network combines segregation and integration: individual nodes and clusters of nodes show different behaviors and subserve different functions; these behaviors, however, emerge from network interactions and the computations depend on the flow of information through the network (Tononi and Edelman, 1998). The localized activity patterns we find are one way to construct such a network. Each node participates strongly in a few timescales and weakly in the others, but the shape and timescales of the activity patterns emerge from the network topology as a whole and information can flow from one node to another. Moreover, as shown in Figure 7, adding a small number of long-range strong links to local connectivity, as in small-world networks (Watts and Strogatz, 1998), causes a few eigenvectors to delocalize while leaving most localized. This is a possible mechanism to integrate computations while preserving segregated activity, and is an interesting direction for future research.

Figure 7

Download asset Open asset

We study the activity of a linear network of coupled units, which will be called ‘nodes’. These represent neurons or populations of neurons. The activity of the jth node, ϕ_j (t), is determined by interactions with the other nodes in the network and by external inputs. It obeys the following equation:

where W (j,k) is the connection strength from node k to node j of the network and I_j is the external input to the jth node. W (j,j) is the self-coupling of the jth node and typically includes a leakage term. Note that the intrinsic timescale of node j is absorbed into the matrix W.

By solving Equation 9, ϕ_j (t) can be expressed in terms of the eigenvectors of the connection matrix W, yielding

(Rugh, 1995). Here, λ indexes the eigenvalues of W, and v_λ (j) is the jth component of the eigenvector corresponding to λ. These are independent of the input. A_λ (t) is the time-dependent amplitude of the eigenvector v_λ and depends on the input, which determines to what extent different eigenvectors are activated. If the real parts of the eigenvalues are negative then the network is stable and, in the absence of input, A_λ (t) decays exponentially with a characteristic time of $1/\mathfrak{R}\mathfrak{e}\left(-\lambda \right)$.

A_λ (t) consists of the sum of contributions from the initial condition and the input, so that Equation 10 can be written as

${\tilde{a}}_{\lambda }$ and ${\tilde{I}}_{\lambda }$ are the coefficients for the initial condition and the input, respectively, represented in the coordinate system of the eigenvectors. In a stable network, each node forgets its initial condition and simultaneously integrates input with the same set of time constants.

In this work, we examine different classes of the connection matrix W, with the constraint that connectivity is primarily local, and we identify conditions under which its eigenvectors are localized in the network in such a way that different nodes (or different parts of the network) exhibit disparate timescales.

1. 1. Abou-Chacra R
   2. Thouless DJ
   3. Anderson PW

   (1973) A selfconsistent theory of localization

   Journal of Physics C Solid State Physics 6:1734–1752.

   https://doi.org/10.1088/0022-3719/6/10/009

   • Google Scholar
2. 1. Ahmed B
   2. Anderson JC
   3. Douglas RJ
   4. Martin KA
   5. Whitteridge D

   (1998) Estimates of the net excitatory currents evoked by visual stimulation of identified neurons in cat visual cortex

   Cerebral Cortex 8:462–476.

   https://doi.org/10.1093/cercor/8.5.462

   • Google Scholar
3. 1. Anderson PW

   (1958) Absence of diffusion in certain random lattices

   Phys Rev 109:1492–1505.

   https://doi.org/10.1103/PhysRev.109.1492

   • Google Scholar
4. Book

   1. Ashcroft NW
   2. Mermin DN

   (1976)

   Solid state physics

   New York: Rinehart and Winston.

   • Google Scholar
5. 1. Barbour B
   2. Brunel N
   3. Hakim V
   4. Nadal JP

   (2007) What can we learn from synaptic weight distributions?

   Trends in Neurosciences 30:622–629.

   https://doi.org/10.1016/j.tins.2007.09.005

   • Google Scholar
6. 1. Benucci A
   2. Ringach DL
   3. Carandini M

   (2009) Coding of stimulus sequences by population responses in visual cortex

   Nature Neuroscience 12:1317–1324.

   https://doi.org/10.1038/nn.2398

   • Google Scholar
7. 1. Bernacchia A
   2. Seo H
   3. Lee D
   4. Wang X-J

   (2011) A reservoir of time constants for memory traces in cortical neurons

   Nature Neuroscience 14:366–372.

   https://doi.org/10.1038/nn.2752

   • Google Scholar
8. 1. Chance FS
   2. Abbott LF
   3. Reyes AD

   (2002) Gain modulation from background synaptic input

   Neuron 35:773–782.

   https://doi.org/10.1016/S0896-6273(02)00820-6

   • Google Scholar
9. 1. Constantinidis C
   2. Goldman-Rakic PS

   (2002) Correlated discharges among putative pyramidal neurons and interneurons in the primate prefrontal cortex

   Journal of Neurophysiology 88:3487–3497.

   https://doi.org/10.1152/jn.00188.2002

   • Google Scholar
10. Book

    1. Dayan P
    2. Abbott LF

    (2001)

    Theoretical neuroscience

    Cambridge: The MIT Press.

    • Google Scholar
11. Book

    1. Elston GN

    (2007)

    Specialization of the neocortical pyramidal cell during primate evolution

    In: Kass JH, Preuss TM, editors. Evolution of nervous systems: a comprehensive reference. New York: Elsevier. pp. 191–242.

    • Google Scholar
12. 1. Ercsey-Ravasz M
    2. Markov NT
    3. Lamy C
    4. Van Essen DC
    5. Knoblauch K
    6. Toroczkai Z
    7. Kennedy H

    (2013) A predictive network model of cerebral cortical connectivity based on a distance rule

    Neuron 80:184–197.

    https://doi.org/10.1016/j.neuron.2013.07.036

    • Google Scholar
13. 1. Ermentrout B

    (1998) Linearization of F-I curves by adaptation

    Neural Computation 10:1721–1729.

    https://doi.org/10.1162/089976698300017106

    • Google Scholar
14. 1. Ganguli S
    2. Bisley JW
    3. Roitman JD
    4. Shadlen MN
    5. Goldberg ME
    6. Miller KD

    (2008) One-dimensional dynamics of attention and decision making in LIP

    Neuron 58:15–25.

    https://doi.org/10.1016/j.neuron.2008.01.038

    • Google Scholar
15. 1. Ganguli S
    2. Huh D
    3. Sompolinsky H

    (2008) Memory traces in dynamical systems

    Proceedings of the National Academy of Sciences of the United States of America 105:18970–18975.

    https://doi.org/10.1073/pnas.0804451105

    • Google Scholar
16. 1. Goldman MS

    (2009) Memory without feedback in a neural network

    Neuron 61:621–634.

    https://doi.org/10.1016/j.neuron.2008.12.012

    • Google Scholar
17. 1. Holmgren C
    2. Harkany T
    3. Svennenfors B
    4. Zilberter Y

    (2003) Pyramidal cell communication within local networks in layer 2/3 of rat neocortex

    Journal of Physiology (London) 551:139–153.

    https://doi.org/10.1113/jphysiol.2003.044784

    • Google Scholar
18. 1. Honey CJ
    2. Thivierge JP
    3. Sporns O

    (2010) Can structure predict function in the human brain?

    NeuroImage 52:766–776.

    https://doi.org/10.1016/j.neuroimage.2010.01.071

    • Google Scholar
19. 1. Joshua M
    2. Medina JF
    3. Lisberger SG

    (2013) Diversity of neural responses in the brainstem during smooth pursuit eye movements constrains the circuit mechanisms of neural integration

    Journal of Neuroscience 33:6633–6647.

    https://doi.org/10.1523/JNEUROSCI.3732-12.2013

    • Google Scholar
20. 1. Komiyama T
    2. Sato TR
    3. O’Connor DH
    4. Zhang YX
    5. Huber D
    6. Hooks BM
    7. Gabitto M
    8. Svoboda K

    (2010) Learning-related fine-scale specificity imaged in motor cortex circuits of behaving mice

    Nature 464:1182–1186.

    https://doi.org/10.1038/nature08897

    • Google Scholar
21. 1. Lee PA

    (1985) Disordered electronic systems

    Reviews of Modern Physics 57:287–337.

    https://doi.org/10.1103/RevModPhys.57.287

    • Google Scholar
22. 1. Levy RB
    2. Reyes AD

    (2012) Spatial profile of excitatory and inhibitory synaptic connectivity in mouse primary auditory cortex

    Journal of Neuroscience 32:5609–5619.

    https://doi.org/10.1523/JNEUROSCI.5158-11.2012

    • Google Scholar
23. 1. Markov NT
    2. Ercsey-Ravasz MM
    3. Ribeiro Gomes AR
    4. Lamy C
    5. Magrou L
    6. Vezoli J
    7. Misery P
    8. Falchier A
    9. Quilodran R
    10. Gariel MA
    11. Sallet J
    12. Gamanut R
    13. Huissoud C
    14. Clavagnier S
    15. Giroud P
    16. Sappey-Marinier D
    17. Barone P
    18. Dehay C
    19. Toroczkai Z
    20. Knoblauch K
    21. Van Essen DC
    22. Kennedy H

    (2014) A weighted and directed interareal connectivity matrix for macaque cerebral cortex

    Cerebral Cortex 24:17–36.

    https://doi.org/10.1093/cercor/bhs270

    • Google Scholar
24. 1. Markov NT
    2. Misery P
    3. Falchier A
    4. Lamy C
    5. Vezoli J
    6. Quilodran R
    7. Gariel MA
    8. Giroud P
    9. Ercsey-Ravasz M
    10. Pilaz LJ
    11. Huissoud C
    12. Barone P
    13. Dehay C
    14. Toroczkai Z
    15. Van Essen DC
    16. Kennedy H
    17. Knoblauch K

    (2011) Weight consistency specifies regularities of macaque cortical networks

    Cerebral Cortex 21:1254–1272.

    https://doi.org/10.1093/cercor/bhq201

    • Google Scholar
25. 1. Miri A
    2. Daie K
    3. Arrenberg AB
    4. Baier H
    5. Aksay E
    6. Tank DW

    (2011) Spatial gradients and multidimensional dynamics in a neural integrator circuit

    Nature Neuroscience 14:1150–1159.

    https://doi.org/10.1038/nn.2888

    • Google Scholar
26. 1. Murphy BK
    2. Miller KD

    (2009) Balanced amplification: a new mechanism of selective amplification of neural activity patterns

    Neuron 61:635–648.

    https://doi.org/10.1016/j.neuron.2009.02.005

    • Google Scholar
27. 1. Newman MEJ

    (2003) The structure and function of complex networks

    SIAM Review 45:167–256.

    https://doi.org/10.1137/S003614450342480

    • Google Scholar
28. 1. Newman MEJ

    (2010)

    Networks: an introduction

    New York, Oxford University Press.

    • Google Scholar
29. Book

    1. Olver FWJ

    (2010)

    Chapter 9 airy and related functions

    In: Olver FWJ, Lozier DW, Boisvert RF, Clark CW, editors. NIST handbook of mathematical functions. New York: Cambridge University Press. pp. 193–214.

    • Google Scholar
30. 1. Perin R
    2. Berger TK
    3. Markram H

    (2011) A synaptic organizing principle for cortical neuronal groups

    Proceedings of the National Academy of Sciences of the United States of America 108:5419–5424.

    https://doi.org/10.1073/pnas.1016051108

    • Google Scholar
31. 1. Rajan K
    2. Abbott LF

    (2006) Eigenvalue spectra of random matrices for neural networks

    Physical Review Letters 97:188104.

    https://doi.org/10.1103/PhysRevLett.97.188104

    • Google Scholar
32. 1. Raser JM
    2. O’Shea EK

    (2005) Noise in gene expression: origins, consequences, and control

    Science 309:2010–2013.

    https://doi.org/10.1126/science.1105891

    • Google Scholar
33. 1. Romo R
    2. Brody CD
    3. Hernandez A
    4. Lemus L

    (1999) Neuronal correlates of parametric working memory in the prefrontal cortex

    Nature 399:470–473.

    https://doi.org/10.1038/20939

    • Google Scholar
34. Book

    1. Rugh WJ

    (1995)

    Linear system theory (2nd edition)

    New Jersey: Prentice Hall.

    • Google Scholar
35. 1. Shriki O
    2. Hansel D
    3. Sompolinsky H

    (2003) Rate models for conductance-based cortical neuronal networks

    Neural Computation 15:1809–1841.

    https://doi.org/10.1162/08997660360675053

    • Google Scholar
36. 1. Smith MA
    2. Kohn A

    (2008) Spatial and temporal scales of neuronal correlation in primary visual cortex

    Journal of Neuroscience 28:12591–12603.

    https://doi.org/10.1523/JNEUROSCI.2929-08.2008

    • Google Scholar
37. 1. Sporns O

    (2011) The non-random brain: efficiency, economy, and complex dynamics

    Frontiers in Computational Neuroscience 5:5.

    https://doi.org/10.3389/fncom.2011.00005

    • Google Scholar
38. Book

    1. Strogatz SH

    (1994)

    Nonlinear dynamics and chaos

    New York: Perseus Books Publishing.

    • Google Scholar
39. 1. Strogatz SH

    (2001) Exploring complex networks

    Nature 410:268–276.

    https://doi.org/10.1038/35065725

    • Google Scholar
40. 1. Tononi G
    2. Edelman GM

    (1998) Consciousness and complexity

    Science 282:1846–1851.

    https://doi.org/10.1126/science.282.5395.1846

    • Google Scholar
41. 1. Trefethen LN
    2. Chapman SJ

    (2004) Wave packet pseudomodes of twisted Toeplitz matrices

    Communications on Pure and Applied Mathematics 57:1233–1264.

    https://doi.org/10.1002/cpa.20034

    • Google Scholar
42. Book

    1. Trefethen LN
    2. Embree M

    (2005)

    Spectra and Pseudospectra: the behavior of Nonnormal matrices and Operators

    Princeton: Princeton University Press.

    • Google Scholar
43. 1. Vogels TP
    2. Rajan K
    3. Abbott LF

    (2005) Neural network dynamics

    Annual Review of Neuroscience 28:357–376.

    https://doi.org/10.1146/annurev.neuro.28.061604.135637

    • Google Scholar
44. 1. Wang X-J

    (1998)

    Calcium coding and adaptive temporal computation in cortical pyramidal neurons

    Journal of Neurophysiology 79:1549–1566.

    • Google Scholar
45. 1. Wang X-J

    (2001) Synaptic reverberation underlying mnemonic persistent activity

    Trends in Neurosciences 24:455–463.

    https://doi.org/10.1016/S0166-2236(00)01868-3

    • Google Scholar
46. 1. Wang X-J

    (2008) Decision making in recurrent neuronal circuits

    Neuron 60:215–234.

    https://doi.org/10.1016/j.neuron.2008.09.034

    • Google Scholar
47. Book

    1. Wang X-J

    (2010)

    Prefrontal cortex

    In: Shepherd GM, Grillner S, editors. Handbook of brain microcircuits. New York: Oxford University Press. pp. 46–56.

    • Google Scholar
48. 1. Watts DJ
    2. Strogatz SH

    (1998) Collective dynamics of ‘small-world’ networks

    Nature 393:440–442.

    https://doi.org/10.1038/30918

    • Google Scholar
49. 1. Wolpert L

    (2011) Positional information and patterning revisited

    Journal of Theoretical Biology 269:359–365.

    https://doi.org/10.1016/j.jtbi.2010.10.034

    • Google Scholar

Author details

1. Rishidev Chaudhuri

   1. Department of Applied Mathematics, Yale University, New Haven, United States
   2. Department of Neurobiology, Yale University, New Haven, United States

   Contribution

   RC, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

2. Alberto Bernacchia

   School of Engineering and Science, Jacobs University Bremen, Bremen, Germany

   Contribution

   AB, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

3. Xiao-Jing Wang

   1. Department of Neurobiology, Yale University, New Haven, United States
   2. Center for Neural Science, New York University, New York, United States

   Contribution

   X-JW, Conception and design, Analysis and interpretation of data, Drafting or revising the article

   For correspondence

   xjwang@nyu.edu

   Competing interests

   The authors declare that no competing interests exist.

• 2,157

  views

• 415

  downloads

• 60

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.