Maximally informative foraging by Caenorhabditis elegans
Abstract
Animals have evolved intricate search strategies to find new sources of food. Here, we analyze a complex food seeking behavior in the nematode Caenorhabditis elegans (C. elegans) to derive a general theory describing different searches. We show that C. elegans, like many other animals, uses a multistage search for food, where they initially explore a small area intensively (‘local search’) before switching to explore a much larger area (‘global search’). We demonstrate that these search strategies as well as the transition between them can be quantitatively explained by a maximally informative search strategy, where the searcher seeks to continuously maximize information about the target. Although performing maximally informative search is computationally demanding, we show that a driftdiffusion model can approximate it successfully with just three neurons. Our study reveals how the maximally informative search strategy can be implemented and adopted to different search conditions.
https://doi.org/10.7554/eLife.04220.001eLife digest
How an animal forages for food can make the difference between life and death, and there are several different searching strategies that may be adopted. Foraging could be more productive if animals could take into account any of the patterns with which food is distributed in their environment, but how much could they measure and memorize? Calhoun et al. show that a tiny worm called Caenorhabditis elegans can keep track of how its previous food finds were spread out, and uses this knowledge to optimize future searches for food.
When C. elegans forages, it begins by performing an intensive search of where it believes food is likely to be found. This strategy, called ‘local search’, is characterised by the worm making numerous sharp turns that keep it in its target search area. If the worm has not found food after 15 min, it abruptly switches its behavior to a socalled ‘global search’ strategy, which features fewer sharp turns and more forays into the surrounding area.
C. elegans is often thought to follow the smell of a food source in order to locate it. While reliable on small scale, this strategy can prove problematic when the distribution of food is patchy. Calhoun et al. show that in extreme conditions, such as when food is completely removed, the animals determine where and for how long to persist with their search based on their knowledge of what was typical of their environment. Such a strategy is called infotaxis, which literally means ‘guided by information’. While the neural circuits underpinning these behaviors remain to be found, Calhoun et al. propose a model that suggests that these circuits could be relatively simple, and made up of as few as three neurons.
https://doi.org/10.7554/eLife.04220.002Introduction
In considering animal behavior and decisionmaking, it is exciting to consider the proposal (Polani, 2009; Tishby and Polani, 2011) that animals may be guided by fundamental statistical quantities, such as the maximization of Shannon mutual information (Cover and Thomas, 1991). The advantage of mutual information as a measure is that its maximization encompasses optimization according to many other statistical measures, such as the peak height or the variance of the distribution. These other measures would give valid results only in certain contexts, such as for predominantly unimodal or Gaussian probability distributions underlying the decision variables. The fact that mutual information can be used with different types of probability distributions makes it possible to quantitatively compare the efficiency of behavioral decisions across species, sensory modalities, and tasks. Indeed, this idea has already yielded insights into diverse behaviors including human eye movements patterns (Najemnik and Geisler, 2005) and animal navigation in a turbulent environment (Vergassola et al., 2007; Masson et al., 2009). Both these patterns of behavior can be accounted for by adapting a maximally informative search strategy to the appropriate behavioral context. This model allocates some actions to improving the estimate of the goal's position rather than directly moving the animal towards the goal (Najemnik and Geisler, 2005; Vergassola et al., 2007). In these contexts, behavioral analyses have shown that strategies aimed at moving directly toward a goal are unable to explain key features of the animal's response. For example, humans sometimes make saccades to examine a region between, rather than directly at, the two likely locations for a target (Najemnik and Geisler, 2005). Similarly, birds and moths zigzag perpendicular to the wind direction to find the source of an odor plume (Vergassola et al., 2007). Informationmaximization (‘infotaxis’) is consistent with direct strategies such as chemotaxis in certain conditions. When the information content of the environment is very high, such as when chemical gradients can be tracked reliably, strategies based on information maximization converge to chemotaxis (Vergassola et al., 2007). Thus, an information maximization approach can be viewed as a generalization of following direct sensory gradients to a broader and more challenging set of behavioral tasks.
Among the multitude of decisions that animals make throughout the day, foraging for food is perhaps the most challenging and critical for survival. Interestingly, a number of species have been reported to spend more time in areas where they have recently observed food (Karieva and Odell, 1987; Benedix, 1993), suggesting that there might be an underlying logic to search that generalizes across species. Recent experimental studies have observed similar foraging patterns in the nematode Caenorhabditis elegans (Hills et al., 2004; Wakabayashi et al., 2004; Gray et al., 2005; Chalasani et al., 2007). After removal from food, the animal first performs an intense search around the area where it believes food is likely to be located (Figure 1A). This period is characterized by an increased number of abrupt turns allowing the animal to stay in the proximal area (Figure 1B) and is termed ‘local search’. After approximately 15 min, animals reduce their number of turns to a basal rate (Figure 1B). This produces more extended trajectories (Figure 1A) and allows the animal to leave the proximal zone and explore a much larger area (‘global search’). Although C. elegans is traditionally considered to be a chemotactic searcher (Ferree and Lockery, 1999; PierceShimomura et al., 1999; Iino and Yoshida, 2009), moving up or down chemical gradients to find the source of an odorant, in these conditions animals have no chemical gradient to follow. We set out to explore whether a single underlying strategy could explain the different aspects of food search behavior in this wellstudied model.
Results
Maximally informative search strategies describe both local and global search states
Since C. elegans performs a food search even in the absence of a gradient (Hills et al., 2004; Wakabayashi et al., 2004; Gray et al., 2005), they must have a prior belief about how food is distributed in the environment. For the sake of simplicity, we assume that the probability of finding food is initially distributed as a twodimensional Gaussian distribution, which imposes the minimal structural constraint beyond the variance of the spatial distribution (Jaynes, 2003). When searching through this space, an animal using the maximally informative trajectory should move in the direction that maximizes its information about the location of food. This can be calculated by taking into account the probability that the nearby environment would emit food odor and then estimating the change in information the animal expects will result from any detection or nondetection events (Vergassola et al., 2007). This means that even a nondetection of a food odorant is informative as it lessens the likelihood that food is nearby. Ultimately, the probability of detecting an odorant depends on the likelihood of food sources across the environment $\left(\overrightarrow{r}\right)$. Information maximization can be analyzed with respect to this quantity, see ‘Materials and methods’ for details. Analyzing these solutions, we find that the maximally informative trajectories first take the searcher towards the peak in the maximum likelihood of food distribution (Figure 2A) and then follow an outward motion (Figure 2B). This intensive search of a small area is qualitatively consistent with the local search performed by C. elegans.
Given an infinitely sized arena, this spirallike motion would continue indefinitely (Barbieri et al., 2011). However, searchers only have knowledge of a finite area (the full extent of the area shown in Figure 2A–B). Further, it is not necessarily true that there will always be food near where it has been seen before. Thus, we have to allow for the possibility that food will not be in the nearby area. In mathematical terms, we allow for the probability that a food source is in the nearby area to deviate from 1. Initially, this probability was set to be very close to one (within numerical accuracy, deviation from 1 was ∼10^{−100}). However as the search progressed, and no odorants were detected, this probability decreased according to the Bayesian rule:
where ${p}_{t+1}\left(A\right)={p}_{t+1}\left(An=0\right)$ is the updated probability given that n = 0 odor detections were observed. The update rule in Equation (1) reflects the fact that, while the searcher at each step expects to detect a certain number of odorants, none are detected because the source is absent. While initially ${p}_{0}\left(A\right)$ is set very close to 1, it gradually decreased to zero. We found that allowing the probability to decrease during the search causes the local search to consistently end abruptly at locations that were very far from the boundaries of modeled area A (Figure 2B). The abrupt transition occurred for any initial values of ${p}_{0}\left(A\right)$ as long as it was not identically equal to 1 at the start of the search. [If ${p}_{0}\left(A\right)=1$, then the probability to find food outside of the local area is zero and it will remain so even after the Bayesian update in Equation (1)]. After the transition, the search trajectory would then follow a straight path to the boundary of the modeled area (Figure 2A–B). These features of search trajectories are consistent with C. elegans transitioning between local and global search.
Maximally informative strategies quantitatively describe foraging trajectories
One interesting feature of this maximally informative search strategy is the abruptness of the transition. Movement around the peak initial belief is followed by a sudden switch to motion away from it. The transition from local to global search corresponds, at least in the model, with the searcher's estimate that the probability that food is located elsewhere equals 1 (Figure 2C). This indicates that qualitative changes in behavioral state arise due to beliefs that no longer reflect old information.
As described above, in the maximally informative model, the transition between the local and global states of the search occurs abruptly. In experiments, the reduction in the number of turns appears to occur gradually (Figure 1B). However this difference could be an artifact of the averaging of trajectories across multiple worms. In contrast, individual worm trajectories could still have sharply defined transitions, as can be seen in Figure 1A. To investigate the sharpness of this transition within individual worm trajectories, we applied a Hidden Markov model framework (Abeles et al., 1995; Seidemann et al., 1996; Bishop, 2004; Jones et al., 2007; Miller and Katz, 2010) to experimentally recorded trajectories. If segments of singleanimal trajectories represent mixtures of states corresponding to local and global parts of the search, then the probability of observing global search patterns will increase gradually. However, analysis of experimental traces revealed a sharp transition between local and global search states on the order of a few minutes (Figure 3A,B). Thus, the search trajectories both in experiment and theory exhibit a sharp transition between the local and global parts of the search.
Next, we examined whether the infotaxis framework could quantitatively account for the distribution of worm search trajectories. The infotaxis model contains three independent parameters: the width of the initial prior probability distribution, filter length representing physical parameters, and how close the initial values for ${p}_{0}\left(A\right)$ was set to 1 (See ‘Materials and methods’). Fitting these parameters of the infotaxis model, it is possible to quantitatively account for the experimental distribution of worm positions at the end of local search (Figure 4A). Importantly, the same set values of these parameters adjusted to match the spatial distribution (Figure 4A) also produced (without readjustment) the cumulative distribution of local search duration and matched experimental measurements (Figure 4B, twosample Kolmogorov–Smirnov test, p = 0.45). The conversion between the spatial axis in Figure 4A and the temporal scale in Figure 4B is set by the calculated value of the worm's speed (∼0.17 mm/s), and does not represent an adjustable parameter. Thus, the infotaxis model can quantitatively account for the properties of worm search behavior after removal from food.
Comparison with chemotaxis model
One may wonder whether other search strategies could also account for food search behavior in worms. Among these, chemotaxis represents the most widely used and parsimonious model of animal behavior (Brown and Berg, 1974; PierceShimomura et al., 1999, 2005; Iino and Yoshida, 2009). A searcher using this strategy would be expected to transiently increase its turning rate when removed from food due to a sudden, large change in food gradient. The subsequent decline in the number of turns would then be explained by adaptation to the low (zero) odorant concentration. Although this explanation seems plausible, it could not quantitatively account for three properties of foraging trajectories: (i) the long duration of the local search, (ii) the rapid exit from the local search state, and (iii) the inability of food concentration to influence local search. An explanation based on adaptation with a single time constant could be ruled out based on the juxtaposition between the relatively long duration of the local search with the fairly rapid transition between the local and global phases of the search. Adaptation with a slow time constant could explain the fairly substantial duration of the local search but not its sharp transition to a global search. On the other hand, adaptation with a short time constant could match the low number of turns during the global search but would underestimate the number of turns and the duration of the local phase of the search (Figure 5A). Quantitatively, adjusting the adaptation time constant to match the observed durations of the local search phase produces trajectories with much broader transitions between local and global phases of the search than is observed experimentally (Figure 3B, comparison between the solid line and histograms, see also Figure 4—figure supplement 1).
Perhaps a more striking illustration as to why chemotaxis does not fully describe the foraging trajectories comes from experiments where worms are transferred from patches of food of the same size but with different concentrations. The chemotaxis model makes predictions based on the change in odorant concentration. This change will be smaller for animals that are removed from patches with more diluted food. Therefore, the chemotaxis model in this case would predict that animals will make a smaller number of turns (Figure 5B). In contrast, the infotaxis model makes predictions based not on the last odorant concentration that the animal experienced prior to its removal from food, but on the relative distribution of food in the environment. The spatial variance of this distribution is not affected by the dilution. Therefore, the infotaxis model would predict that the animals will make the same number of turns regardless of the bacteria concentration within the lawn, provided the lawns have the same size. This prediction was supported by our measurements (Figure 5B). Overall, we have found that C. elegans behavior when removed from food cannot be explained as chemotaxis but is consistent with infotaxis.
Driftdiffusion approximation to the maximally informative search
The results we have presented so far argue that the quantitative characteristics of animals' behavior match what would be expected for an optimal, maximally informative (Vergassola et al., 2007) or (equivalently) Bayesian (Najemnik and Geisler, 2005) model of search. This model continuously updates the likelihood of a food source being present throughout the duration of search. At first glance, these calculations require the ability to maintain and update the corresponding “mental maps” of the environment. However, the animals could also approximate complex computations with empiricallytuned simple search heuristics that have only slightly smaller than maximal yields. Interestingly, as the search progresses, the log probability $1{p}_{t}\left(A\right)$ that the food is located elsewhere accumulates. Broader priors require more time to come to the conclusion that food is located elsewhere. When this probability reaches one, the local phase of the search ends and the global phase begins. The approximately linear increase in the log probability $1{p}_{t}\left(A\right)$ observed during most of the local search duration (after the initial period of supralinear increase, cf. Figure 2C) suggests that the timing of the transition from the local to global search could be accounted for by a simple driftdiffusion model (Bogacs et al., 2006; Insabato et al., 2006). In our setup the driftdiffusion model has effectively only one parameter—the drift rate. While in most applications driftdiffusion models are considered together with an adjustable threshold, here the lower threshold value is fixed to 0 because the dynamical variable represents probability. Similarly, the starting value of this probability (which we set to be just under 1) also has relatively weak influence on the duration of local search. This is because the initial decrease in $\text{ln}[1{p}_{t}\left(A\right)]$ occurs supralinearly before settling on the linear increase.
The key property of the maximally informative foraging strategies is that they depend on the width of the initial (‘prior’) distribution of food in the environment. We find that changing the width of the distribution σ changes the rate at which the evidence that food is elsewhere accumulates (Figure 6A). Adjusting the slope of the driftdiffusion model captures both the change in evidenceaccumulation as well as the observed distribution of transition times from local to global search (Figure 6B). Furthermore, the slope of the bestfitting driftdiffusion model scaled linearly with σ (Figure 6C). These observations suggest that animals could empirically learn the appropriate slope for different distributions of food, and in this way perform nearly optimal foraging strategies with minimum computational effort. Notably, the distribution of local search duration times produced by the chemotaxis model show the opposite dependence on the width of the prior distribution compared to the infotaxis model (Figure 6B).
The maximally informative foraging trajectories are affected not only by the width of the prior distribution but also by odorant characteristics. For example, the diffusivity of odorant molecules affects the calculation of the likelihood of food source. Perhaps fortuitously for animals with small neural circuits, we found that the changes in diffusivity primarily affected the rate of increase $\text{ln}[1{p}_{t}\left(A\right)]$, but the overall dynamics could still be described by the that driftdiffusion model (Figure 6D). The slope of the driftdiffusion model increased approximately linearly (Figure 6D) with the spatial extent L of the diffusion filter (Vergassola et al., 2007), see also Equation 3 in ‘Materials and methods’. Notably, the drift rate depends primarily on the ratio σ/L (Figure 6E–F). These results demonstrate that maximally informative foraging trajectories can be approximated by a simple driftdiffusion model across a range of behaviorally relevant conditions.
Discussion
In this work we have shown that the exploratory behavior of a small animal, the nematode C. elegans, meets the quantitative benchmarks expected for an optimal, maximally informative strategy. This analysis formally requires continuous updates to the likelihood of food sources based on incoming sensory inputs. At the same time, we find that the resulting maximally informative foraging strategies can also be implemented with a combination of a random confined walk with an internal driftdiffusion variable to encode the transition to a new search strategy. The presence of such transitions illustrates how emergent discrete decisions occur as a result of continuous exploration of the environment. Previous studies have shown that driftdiffusion models can provide a substrate for optimal calculations in two alternative forcedchoice tasks (Bogacs et al., 2006). Here, we find that these models can also help approximate optimal calculations in cases where alternative choices are not imposed externally but emerge as an intrinsic part of behaviors that optimize gain over long time scales.
A tentative circuit
The C. elegans neuroanatomy (Gray et al., 2005) suggests that multiphase foraging strategies can be implemented at the neuronal level, even in simple nervous systems. Taking the twophase foraging circuit (Gray et al., 2005) that we analyzed here as an example, one may hypothesize that the initial trigger for the start of the search is provided by sensory input, likely through the AWC sensory neuron (Figure 7). AWC neurons respond to a decrease in odorant concentration (Chalasani et al., 2007). However, these responses are transient (Chalasani et al., 2007) and do not last long enough to account for the long duration of local search (∼15 min). Instead, we hypothesize that local search is maintained based on the responses of one of the interneurons. The gradual change in state of these neurons that receive sensory input, for example the AIB and AIZ interneurons, may encode the passage of time since the start of local search. Modulating the rate at which the internal state of a neuron changes allows the animal to adjust the duration of local search after sensing aspects of the environment such as how food is spatially distributed and how far the odorant molecules diffuse from this particular food source. We hypothesize that this modulation occurs through neuromodulatory signaling, which is known to be involved in local search behavior (Hills et al., 2004). In summary, a neural circuit based on just a few neurons suffices to implement foraging strategies that approximate the maximally informative, but computationally intensive, decisions.
Infotaxis vs chemotaxis
There are a wide range of possible foraging strategies that animals might follow. These include chemotaxis based on local sensory cues, different types of random walks (Bartumeus et al., 2002; Humphries et al., 2010; Viswanathan et al., 2011; Humphries et al., 2012), and computationally intensive models based on detailed memories of past experiences. This raises the question of whether the optimal foraging strategy is constrained not only by the physical environment but also by the computational complexity of its implementation (Tishby and Polani, 2011). One approach to this solution is provided by the conventional chemotaxis model (Ferree and Lockery, 1999; PierceShimomura et al., 1999). A chemotactic behavior can be implemented based on responses of just one sensory neuron. However, the resulting search strategies are driven directly by changes in the gradient and do not necessarily reflect the typical size of food patches. While infotaxis and chemotaxis strategies converge under conditions of smoothly varying gradients, this is not so in cases where transitions between patches are common. In fact, we found that chemotactic trajectories exhibited not only a much weaker dependence on the patch size compared to infotaxis trajectories but also predicted the opposite relationship between patch size and search duration (Figure 6B). In addition, we found that worm and foraging trajectories were unaffected by the overall food concentration within the patch (Figure 5B), in agreement with infotaxis but in contrast to chemotaxis predictions. The addition of interneurons to the circuit, as schematized in Figure 7, makes it possible to dissociate the change in the gradient from the duration of local search. Thus, the modest increase in computational cost associated with the addition of interneurons allows for more flexible behavior than would be seen in a simple chemotaxis strategy.
It has been noted that foraging strategies that maximize the mutual information about target locations do not always produce the maximal yield. Such situations have been observed in cases where the targets are mobile (Agarwala et al., 2012). In this case, although the searcher knows precisely where the food is located at a given time, it might not be able to get to the food source before it moves again. Animals might counteract this problem with predictive coding, using foraging strategies that maximize information about the food source location at a sufficient time in the future (Polani, 2009; Tishby and Polani, 2011; Bialek, 2013). So long as the parameters of simpler models can be easily learned through experience, there are no barriers to implementing such strategies with a few neurons. Indeed, including predictive information may take the form of an increased rate of evidence accumulation in an infotaxislike model.
The fact that both infotaxis and driftdiffusion models can account for the properties of foraging trajectories does not take away from the stated goal that animal behavior is guided by information maximization. After all, the driftdiffusion models are fitted to parameters of infotaxis trajectories. These fits dictate how the animals should adjust search times depending on the typical widths of food patches in the environment. While the infotaxis model predicts that local search should last longer for wider food patches, the chemotaxis model makes the opposite prediction (Figure 6). At the same time, even to set parameters of driftdiffusion models, one would need to estimate the variance of the food distribution across space. This is quite a feat for such a small animal as C. elegans. Further, it might be possible that C. elegans are capable of adjusting their behavior based on higherthansecond moments of the probability distribution. Demonstrating this would require more finescale experiments to control differences in both size and shape of food patches from which the animals are removed. If such sensitivities are observed, they would implicate the involvement of more complicated circuits that could be mapped onto a single driftdiffusion model. Finally, it is worth noting that the local search of C. elegans exhibits striking similarities to other invertebrates, such as crabs (Zeil, 1998), bees (Gould, 1986; Dyer, 1991), and ants (Wehner et al., 2002). In particular, the search patterns of desert ants that have been displaced on their return to the nest (Wehner et al., 2002). When arriving near the presumed location of the nest, animals follow a spiral search pattern that is consistent with infotaxis trajectories (Barbieri et al., 2011). Following large displacements, ants have great difficulties finding the nest with local search patterns and transition to a strategy that is reminiscent of the global search executed by C. elegans (Wehner, 2003; Wehner et al., 2006). The largescale foraging patterns in ants are difficult to study quantitatively because of the large areas involved and a few published foraging trajectories (Wehner et al., 2002). Our results add to these by showing that invertebrates can integrate more abstract quantities than spatial position and operate directly on the probability that the food (or nest) is located elsewhere. Importantly, the animals do not need to perform informationtheoretic calculations all of the time; instead they can set parameters of the approximating models through learning and experience.
In summary, animals appear to guide their foraging behavior by searching for information. This simple behavioral rule is able to account for multiple search strategies, as well as the emergent transitions between them. While seemingly complex, this strategy can be easily implemented in a reduced neural system. We anticipate that this principle will prove useful as a general theory of search and decisionmaking in a wide range of contexts.
Materials and methods
Quantification of animal behavior
Request a detailed protocolC. elegans in the L4 larval stage were allowed to grow overnight on an agar plate containing a 100 μl circular patch of the E. coli OP50 strain (OD_{600} = 0.4). For testing, animals were moved to an agar observation plate without any food where they were corralled into a 1″ square by a filter paper soaked in 200 mM CuSO4, which animals generally avoid. Moving an animal requires them to picked up using a metal object. These animals spend roughly 2 min moving forward before initiating their search. Worm movement was recorded for 30 min at three frames per second, and the first 2 min are ignored.
Computation of infotaxis trajectories
Request a detailed protocolInfotaxis trajectories were modeled using a 128 × 128 grid representing position and probability distribution of the food source in the environment. At each step, the anticipated change in entropy was computed taking into account two possibilities: observing or not observing odorant hits. Although the initial descriptions of the model separated odorant detection events according to the number of odorant hits, in our setup (absent food source) the computation of those probabilities was numerically unstable. This is the reason why we reduce the coding to binary, either ‘no hits’ or ‘a nonzero number of hits’. As such, each change in entropy is calculated using the probability to receive a hit or the probability to receive zero hits. Computations are halted upon being within one space of the border or after no movement for 15 time steps. Trajectories are computed as in (Vergassola et al., 2007). The probability of an odor source being located at location r_{0} after observing a trace of odor encounters is given by
or, alternately,
where H is the number of hits observed during the trajectory at time t_{i}. Here, ${L}_{{\mathbf{r}}_{0}}\left({T}_{t}\right)$ is the likelihood of observing the trace T_{t} odor encounters from a source located at r_{0}. When a region is visited and the source is not found, that region has its probability set to 0. R is the function representing the mean hit rate observed at location r if the source is at r_{0}. It has the following form:
where a is the size of the searcher, $\rho $ is the particle emission rate, D is the diffusivity of the particles, the particles have a finite lifetime $\tau $, and K_{0} is the modified Bessel function of order 0. The filter width L is defined as $\text{L}=\sqrt{D\tau}$.
During movement, the expected change in entropy when moving is
where ${\rho}_{0}$ represents the probability that 0 detections are made at r_{j} during a timestep and ${\rho}_{1}$ represents the probability that any detections are made. In this cases, the expected number of hits $h\left({\mathbf{r}}_{j}\right)={\displaystyle \int}{P}_{t}\left({\mathbf{r}}_{j}\right)R\left({\mathbf{r}}_{j}\right{\mathbf{r}}_{0})d{\mathbf{r}}_{0}$ with the probability of hits following a Poisson law. In other words, ${\rho}_{1}=\left(1{p}_{t}\left(A\right)\right)(1{e}^{h})$.
During search, while a hit results in a change in the probability landscape of $R\left(\mathbf{r}\text{}{\mathbf{r}}_{0}\right)$, no hits will update the prior by convolving it with $\mathrm{exp}(R\left(\mathbf{r}\text{}{\mathbf{r}}_{0}\right))$. The length scale of this filter is calculated by fitting it with an exponential function $\mathrm{exp}(x/L)$ with an adjustable length scale L.
Driftdiffusion model
Request a detailed protocolThe decision variable was modeled as an accumulating value with initial value set at −100 to represent the loglikelihood that food is elsewhere. Drift and diffusion parameters were extracted from the time series of infotaxis trajectories and decisions were simulated using the following equation:
Here, x is the current evidence in favor of a decision. It grows with mean drift rate A and Gaussian noise dW is drawn with standard deviation s. Simulations were ended once the decision variable reached 0.
Data availability

Data from: Maximally informative foraging by Caenorhabditis elegansAvailable at Dryad Digital Repository under a CC0 Public Domain Dedication.
References

Cortical activity flips among quasistationary statesProceedings of the National Academy of Sciences of USA 92:8616–8620.https://doi.org/10.1073/pnas.92.19.8616

Pursuit of food versus pursuit of information in a Markovian perceptionaction loop model of foragingJournal of Theoretical Biology 304:235–272.https://doi.org/10.1016/j.jtbi.2012.02.016

Optimizing the encounter rate in biological interactions: Levy versus Brownian strategiesPhysical Review Letters 88:097901.https://doi.org/10.1103/PhysRevLett.88.097901

Temporal Stimulation of chemotaxis in Escherichia coliProceedings of the National Academy of Sciences of USA 71:1388–1392.https://doi.org/10.1073/pnas.71.4.1388

Computational rules for chemotaxis in the nematode C. elegansJournal of Computational Neuroscience 6:263–277.https://doi.org/10.1023/A:1008857906763

A circuit for navigation in Caenorhabditis elegansProceedings of the National Academy of Sciences of USA 102:3184–3191.https://doi.org/10.1073/pnas.0409009101

Dopamine and glutamate control arearestricted search behavior in Caenorhabditis elegansThe Journal of Neuroscience 24:1217–1225.https://doi.org/10.1523/JNEUROSCI.156903.2004

Foraging success of biological Levy flights recorded in situProceedings of the National Academy of Sciences of USA 109:7169–7174.https://doi.org/10.1073/pnas.1121201109

Parallel use of two behavioral mechanisms for chemotaxis in Caenorhabditis elegansThe Journal of Neuroscience 29:5370–5380.https://doi.org/10.1523/JNEUROSCI.363308.2009

The influence of Spatiotemporal Structure of Noisy stimuli in decision makingPLOS computational biology 10:e1003492.https://doi.org/10.1371/journal.pcbi.1003492

BookProbability Theory: The Logic of ScienceCambridge, England: Cambridge University Press.

Natural stimuli evoke dynamic sequences of states in sensory cortical ensemblesProceedings of the National Academy of Sciences of USA 104:18772–18777.https://doi.org/10.1073/pnas.0705546104

Swarms of predators exhibit “preytaxis” if individual predators use arearestricted searchThe Americal Naturalist 130:233–270.https://doi.org/10.1086/284707

Chasing information to search in random environmentsJournal Of Physics A: Mathematical And Theoretical 42:434009.https://doi.org/10.1088/17518113/42/43/434009

Stochastic transitions between neural states in taste processing and decisionmakingThe Journal of Neuroscience 30:2559–2570.https://doi.org/10.1523/JNEUROSCI.304709.2010

The fundamental role of pirouettes in Caenorhabditis elegans chemotaxisThe Journal of Neuroscience 19:9557–9569.

Analysis of the effects of turning bias on chemotaxis in C. elegansThe Journal of Experimental Biology 208:4727–4733.https://doi.org/10.1242/jeb.01933

Simultaneously recorded single units in the frontal cortex go through sequences of discrete and stable states in monkeys performing a delayed localization taskThe Journal of Neuroscience 16:752–768.

BookInformation theory of decisions and actionsIn: Tishby N, Polani D, Cutsuridis V, Hussain A, Taylor JG, editors. Perceptionaction cycle: Models, architectures, and hardware. New York: Springer. pp. 601–636.

BookThe physics of foraging : an introduction to random searches and biological encountersCambridge, New York: Cambridge University Press.

Neurons regulating the duration of forward locomotion in Caenorhabditis elegansNeuroscience Research 50:103–111.https://doi.org/10.1016/j.neures.2004.06.005

Calibration processes in desert ant navigation: vector courses and systematic searchJournal of Comparative physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 188:683–693.https://doi.org/10.1007/s0035900203408

Ant navigation: oneway routes rather than mapsCurrent Biology 16:75–79.https://doi.org/10.1016/j.cub.2005.11.035

Desert ant navigation: how miniature brains solve complex tasksJournal of Comparative physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 189:579–588.https://doi.org/10.1007/s0035900304311

Homing in fiddler crabs (Uca lacteal amnulipes and Uca vomeris: Ocypodidae)Journal of Comparative Physiology 183:367–377.https://doi.org/10.1007/s003590050263
Decision letter

Ranulfo RomoReviewing Editor; Universidad Nacional Autonoma de Mexico, Mexico
eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.
Thank you for sending your work entitled “Maximally informative foraging by Caenorhabditis elegans” for consideration at eLife. Your article has been favorably evaluated by a Senior editor, a Reviewing editor, and 2 reviewers.
The Reviewing editor and the other reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.
The authors studied the foraging behavior of the C. elegans worm and found that the transitions from a local (search intensive) to a global (less turns, larger area) search strategy can be explained by a drift diffusion model that effectively maximizes information about the location of the food source. This infotaxis model has greater explanative value as compared to a chemotaxis mechanism that only follows the gradient of odorant concentration and the authors demonstrate that is could be implemented by a threeneuron circuit.
1) However, there are some few issues that the reviewers did not fully understand, and I agree: is the driftdiffusion model described by Equation (3) connected to the worm's sensory data in some way? In particular, decisions are reached when x reaches a value of zero, but is this initial value of x at the start of the local search? One of the reviewers thinks this is not clearly stated, and that might not be true, and I agree.
2) One of the reviewers thinks that the qualitative features (e.g., the abrupt change from local to global search; food concentration independence of optimal infotaxic search) show that this model is distinct from chemotaxis, and that it would be helpful to make direct comparisons with best fits from situations of chemotaxis in the figures (Figures 4a,b, 5B and 6b).
Minor comments/questions:
1) How similar are the foraging behaviors in other invertebrate species mentioned, and are there any other instances of infotaxis/mutual information searching in animal behaviors other than visual saccades and bacteria?
2) It would be interesting if the authors could describe an intuitively appealing rationale for why infotaxis as a method should be predicted to work. For example, when considering foraging behavior and various possible mathematical descriptions, is there a good a priori reason to expect that mutual information search would be favored?
3) The model contains three independent parameters that are fitted to reproduce the experimental distribution of the animal's positions at the end of the local search. Then the authors write, “Importantly, the same set of parameters can also account for the cumulative distribution of the durations of local search (Figure 4b).” Do they mean that the same set of parameters can be used to do this or that the same numerical values of the parameters account for the optimal durations?
4) “Predictions of the infotaxis model are based on the relative distribution of food across space, rather than on its absolute concentration.” Sounds a bit confusing; chemotaxis is based on a gradient, not an absolute concentration.
5) “At first glance, these calculations require the ability to maintain and update such “mental maps”, which may seem too computationally demanding to be implemented in a small neural circuit, such as the one found in C. elegans. Therefore, one may wonder whether the animals approximate the maximally informative foraging strategies.” This reference to a “mental map” in C. elegans is striking: does this model predict that as a result of this search strategy, the neuronal network acquires an internal representation of the environment?
6) “The temporal dynamics of this probability, and the fact that it represents the accumulating evidence, suggests that a driftdiffusion model may be able to approximate this logprobability.” An immediate jump to a driftdiffusion model might not occur to me. What are the arguments for using this model (other than that it seems to work post factum, e.g., reproduce the linear slope, etc.)? This model serves as a basis of approximation, so what exactly is being “approximated” in this approach, i.e., what is disregarded and what is kept? The authors say “Ultimately, maximally informative trajectories could potentially exhibit a much greater degree of diversity than can be accounted for by the simpler driftdiffusion model.” So what do we sacrifice to get the approximation?
7) “Invariance properties that characterize the maximally informative foraging trajectories” – what is meant by “invariance” here?
8) “Although initial descriptions of the model suggested that the reduction in entropy associated with each discrete number of odorant hits should be computed, in our experience any approximation of this results in unwanted behavior. Therefore, we limited the maximal number of odorants that can be simultaneously detected to 1”. This paragraph is hard to parse, please explain more clearly.
9) Am I correct in understanding that optimal infotaxis results are shown in both Figures 5a and 5b?
10) I assume the red and black curves in Figure 6b follow the same convention as Figure 5a (but not of later panels in this figure) but it would be helpful to explicitly state what the color convention is in the caption for Figure 6b.
https://doi.org/10.7554/eLife.04220.011Author response
1) However, there are some few issues that the reviewers did not fully understand, and I agree: is the driftdiffusion model described by Equation (3) connected to the worm's sensory data in some way?
Authors: In the manuscript, we fit the driftdiffusion models to parameters of the infotaxis trajectories. Because these parameters also match experimental data, they could also be thought as a fit to experimental trajectories. We have added a discussion of this point.
The more general implication is that the sensory data sets the parameters of the driftdiffusion model. First, a sensory signal of odorant withdrawal starts the timer for the evidence accumulation in the driftdiffusion model. The drift rate depends on the parameters of food distribution that animals are acclimated to (e.g. small or wide patch of food). The dependence of the drift rate on the width of food patches matches what is needed to maximize mutual information. This drift rate is likely to be encoded using neuromodulatory signals (likely dopamine) that would integrate sensory data over much longer time scales to determine the environmental context for foraging conditions.
In particular, decisions are reached when x reaches a value of zero, but is this initial value of x at the start of the local search? One of the reviewers thinks this is not clearly stated, and that might not be true, and I agree.
The starting value for variable x, which represents the probability that the food is located within the local search area, was initially set to be slightly below 1. If the probability is set to exactly 1, meaning that the probability that the food source is outside the local search area, then it will never change. However, even when the deviations of this probability from 1 are as small as 10^{100} the transition from the local to global search takes place (as in Figure 2C where log(1x) is plotted). We have now clarified this point in the manuscript.
2) One of the reviewers thinks that the qualitative features (e.g., the abrupt change from local to global search; food concentration independence of optimal infotaxic search) show that this model is distinct from chemotaxis, and that it would be helpful to make direct comparisons with best fits from situations of chemotaxis in the figures (Figures 4a,b, 5B and 6b).
We now provide the corresponding comparisons between measurements and best predictions of the chemotaxis model in Figures 4, 5b, 6b. The chemotaxis model makes qualitatively different (and in some cases opposite) predictions compared to the infotaxis model. For example, in Figure 5b, infotaxis predicts no change in animals’ behavior following the dilution of bacteria concentration, whereas the chemotaxis model predicts a decrease in the number of turns. In Figure 6b that compares the distribution of local search durations as a function of the width of the food patch, the infotaxis model predicts longer durations for wider patches whereas the chemotaxis model predicts shorter durations. The infotaxis predictions match experiments. Because Figure 4 appears in the manuscript much before we discuss the chemotaxis model, we first show this figure without the chemotaxis predictions, and then show the same data with chemotaxis predictions as Figure 4–figure supplement 1.
Minor comments/questions:
1) How similar are the foraging behaviors in other invertebrate species mentioned, and are there any other instances of infotaxis/mutual information searching in animal behaviors other than visual saccades and bacteria?
We have added references on foraging behaviors of ants, crabs, and bees that demonstrate qualitatively similar features of foraging that we describe here for C. elegans. For example, when ants that have been displaced on their way to the nest search the presumed nest location for a while but then transition to what could be equivalent to the global search. We have expanded this discussion point in the manuscript. There are only a handful published examples of trajectories after sufficiently large displacements that ants do not find their nest during the local search phase. We also hope to study foraging behaviors of the zebrafish larvae soon, which would provide an example of foraging in three dimensions.
2) It would be interesting if the authors could describe an intuitively appealing rationale for why infotaxis as a method should be predicted to work. For example, when considering foraging behavior and various possible mathematical descriptions, is there a good a priori reason to expect that mutual information search would be favored?
We now include the motivation for the use of mutual information in the Introduction section. To briefly restate the argument here, a number of measures other than the mutual information can be used to quantify a decrease in the uncertainty of the target position during foraging. These include the variance of its probability distribution or the height of the peak of this distribution. However, mutual information is unique in that it does not assume a certain set up of the foraging problem or properties of the distribution describing the distribution of food in the environment. For example, variance as a measure is only technically valid for Gaussian probability distributions of food locations. Once the probability distribution becomes nonGaussian (this happens often after the exploration near the initial peak of the distribution), using variance to characterize the distribution leads to misleading results. Thus, the mutual information provides a more general measure to quantify changes in the probability distribution under a variety of circumstances, although one can use other measures in specific cases.
3) The model contains three independent parameters that are fitted to reproduce the experimental distribution of the animal's positions at the end of the local search. Then the authors write, “Importantly, the same set of parameters can also account for the cumulative distribution of the durations of local search (Figure 4b).” Do they mean that the same set of parameters can be used to do this or that the same numerical values of the parameters account for the optimal durations?
This sentence has been rewritten to clarify that the same numerical values accounted for both probability distributions in Figure 4a and 4b.
“Importantly, the same set values of these parameters adjusted to match the spatial distribution (Figure 4a) also produced (without readjustment) cumulative distribution of local search duration that matched experimental measurements (Figure 4b, twosample KolmogorovSmirnov test, p=0.45).”
4) “Predictions of the infotaxis model are based on the relative distribution of food across space, rather than on its absolute concentration.” Sounds a bit confusing; chemotaxis is based on a gradient, not an absolute concentration.
The sentence and paragraph that contained it has now been rewritten to address this question as follows:
“The chemotaxis model makes predictions based on the change in odorant concentration. This change will be smaller for animals that are removed from patches with more diluted food. Therefore, the chemotaxis model in this case would predict that animals will make smaller number of turns (Figure 5b). In contrast, the infotaxis model makes predictions based not on the last odorant concentration value that the animal has experienced prior to its removal from food, but on the relative distribution of food in the environment. The spatial variance of this distribution is not affected by the dilution. Therefore, the infotaxis model would predict that the animals from regular and diluted bacteria lawns will make the same number of turns. This prediction was supported by our measurements (Figure 5b).”
5) “At first glance, these calculations require the ability to maintain and update such “mental maps”, which may seem too computationally demanding to be implemented in a small neural circuit, such as the one found in C. elegans. Therefore, one may wonder whether the animals approximate the maximally informative foraging strategies.” This reference to a “mental map” in C. elegans is striking: does this model predict that as a result of this search strategy, the neuronal network acquires an internal representation of the environment?
This was indeed our initial supposition. However, given that both the infotaxis and driftdiffusion models account for the data, at present one can only state that animals estimate the variance of food distribution in space and adjust search times accordingly. The fact that search times are adjusted in ways that maximizes mutual information supports the argument for information maximization as the behavioral goal. Future experiments with more detailed monitoring of animals’ movements could provide an indication that animals estimate highorder parameters of the food distribution. We have added this discussion point in the manuscript.
6) “The temporal dynamics of this probability, and the fact that it represents the accumulating evidence, suggests that a driftdiffusion model may be able to approximate this logprobability.” An immediate jump to a driftdiffusion model might not occur to me. What are the arguments for using this model (other than that it seems to work post factum, e.g., reproduce the linear slope, etc.)? This model serves as a basis of approximation, so what exactly is being “approximated” in this approach, i.e., what is disregarded and what is kept? The authors say “Ultimately, maximally informative trajectories could potentially exhibit a much greater degree of diversity than can be accounted for by the simpler driftdiffusion model.” So what do we sacrifice to get the approximation?
Our main goal was to account for the timing of the transition between local and global search. We tried many different criteria, including the entropy of the likelihood distribution of food source location at the decision time, local variance of this distribution near the decision point, direction of the gradient, etc. However, none of these measures could reliably predict the switching point between local and global phases of the search, until we looked at the normalization constant representing that the likelihood of finding food elsewhere. When this probability reached 1, the switch from local to global phase of the search occurred. The driftdiffusion model captures both the approximately linear increase and stochastic properties of foraging.
The simplified driftdiffusion model has only one parameter; the drift rate. Thus, for a given odorant, the drifdiffusion model can only encode one parameter of typical food patches, i.e. their size. To modify their behavior according to additional highorder parameters of the food patch distribution, the animals will have to use more complex models. We have expanded the discussion of these two points in the manuscript.
7) “Invariance properties that characterize the maximally informative foraging trajectories” – what is meant by ”invariance” here?
We have now clarified at this point in the text that invariance refers to the independence of the search strategies to the concentration of food within the patch. We also no longer use the term “invariance” to avoid confusion.
8) “Although initial descriptions of the model suggested that the reduction in entropy associated with each discrete number of odorant hits should be computed, in our experience any approximation of this results in unwanted behavior. Therefore, we limited the maximal number of odorants that can be simultaneously detected to 1”. This paragraph is hard to parse, please explain more clearly.
We have rewritten this paragraph to clarify that we use binary representation separating odorant detection events into zero hit event and nonzero hit events. Keeping track of separate probability of observing 1,2,3, etc. hits at a time led to numerically unstable results.
9) Am I correct in understanding that optimal infotaxis results are shown in both Figures 5a and 5b?
We have added predictions based on infotaxis and chemotaxis models to Figure 5b. Panel A shows that measured trajectories (red line) are not consistent with different possible chemotaxis models.
10) I assume the red and black curves in Figure 6b follow the same convention as Figure 5a (but not of later panels in this figure) but it would be helpful to explicitly state what the color convention is in the caption for Figure 6b.
That is correct. We have now clarified that the panels A and B of Figure 6 follow the same color convention that is different from the color convention used in panels CF.
https://doi.org/10.7554/eLife.04220.012Article and author information
Author details
Funding
National Science Foundation (Career award 1254123)
 Tatyana O Sharpee
National Institutes of Health (R01EY019493)
 Tatyana O Sharpee
National Science Foundation (Graduate Research Fellowship)
 Adam J Calhoun
Rita Allen Foundation
 Sreekanth H Chalasani
McKnight Endowment Fund for Neuroscience (McKnight Scholarship)
 Tatyana O Sharpee
Ray Thomas Edwards Foundation
 Tatyana O Sharpee
University of California, San Diego (UCSD) (Institute for Neural Computationa graduate fellowship)
 Adam J Calhoun
National Institutes of Health (P30EY019005)
 Tatyana O Sharpee
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
We thank Z Cecere and members of the Sharpee lab for key help, advice, and insights. This research was supported by the National Science Foundation (NSF) CAREER award number 1254123, the National Eye Institute of the National Institutes of Health under Award Number R01EY019493, P30EY019005, McKnight Scholarship and Ray Thomas Edwards Career Award (TOS), a graduate research fellowship from the NSF , the National Institute of Mental Health (NIMH) and the University of California, San Diego Institute for Neural Computation graduate Fellowship (AJC) and by the Rita Allen Foundation (SHC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health and the National Science Foundation.
Reviewing Editor
 Ranulfo Romo, Universidad Nacional Autonoma de Mexico, Mexico
Publication history
 Received: August 1, 2014
 Accepted: November 3, 2014
 Version of Record published: December 9, 2014 (version 1)
Copyright
© 2014, Calhoun et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 4,229
 Page views

 509
 Downloads

 65
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading

 Neuroscience
Artificial neural networks could pave the way for efficiently simulating largescale models of neuronal networks in the nervous system.

 Neuroscience
A key question in decision making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trialanderror learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a wellestablished manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.