Abstract
Many organisms—from bacteria to nematodes to insect larvae—navigate their environments by biasing random movements. In these organisms, navigation in isotropic environments can be characterized as an essentially diffusive and undirected process. In stimulus gradients, movement decisions are biased to drive directed navigation toward favorable environments. How does directed navigation in a gradient modulate random exploration either parallel or orthogonal to the gradient? Here, we introduce methods originally used for analyzing protein folding trajectories to study the trajectories of the nematode Caenorhabditis elegans and the Drosophila larva in isotropic environments, as well as in thermal and chemical gradients. We find that the statistics of random exploration in any direction are little affected by directed movement along a stimulus gradient. A key constraint on the behavioral strategies of these organisms appears to be the preservation of their capacity to continuously explore their environments in all directions even while moving toward favorable conditions.
https://doi.org/10.7554/eLife.30503.001Introduction
The trajectories of small organisms often involve stochastic transitions between distinct motor states. A classic example is the swimming behavior of Escherichia coli (; Berg, 1993), which is characterized by an alternating sequence of runs and tumbles. During runs, the bacteria swim in roughly straight lines, while during tumbles, the bacteria move erratically in place, ultimately picking the direction of a new run at random. The trajectories of larger animals like nematodes and insect larvae are qualitatively similar (PierceShimomura et al., 1999; Luo et al., 2010). Caenorhabditis elegans alternate periods of forward movement with either large angle reorientation maneuvers called pirouettes or small angle turns. Crawling Drosophila larvae alternate periods of forward movement with turns (Lahiri et al., 2011) where they pause forward motion and use the angle of head swings to pick a new forward orientation.
C. elegans also modulates its random exploration in isotropic environments over time (Wakabayashi et al., 2004; Chalasani et al., 2007). When a worm is placed in a new environment, it first executes a local search, where runs are short. Over time, worms transition to a global search with longer runs. It has been suggested that the transition between local and global searching is discontinuous (Calhoun et al., 2014), and that local search and global search represent two distinct behavioral states.
In stimulus gradients, bacteria, nematodes, and insect larvae bias their random walks toward favorable environments by modulating the statistics of transitions between forwardmoving runs and reorientation events. For example, all three organisms exhibit longer runs when pointed toward favorable conditions. Worms and larvae further augment the time spent pointed toward favorable directions by increasing the probability of ending reorientation events with a run pointed in a favorable direction or by gradually steering runs toward favorable directions.
The navigational dynamics of worms and larvae have some parallels with the complex dynamics of a polypeptide chain navigating to the native structure of the protein to which it corresponds. Both are examples of stochastic search processes: the protein needs to fold to the correct native structure, while an organism needs to find food and favorable temperatures, for example. Neither search can be purely random, because it would not be effective. For the protein folding case, it would lead to the Levinthal paradox; that is, it would take an essentially infinite amount of time to fold, while in fact it takes on the order of seconds to minutes (Zwanzig et al., 1992; Karplus, 1997). The stochastic search of a protein is biased toward the native structure by the potential energy, which is encoded in the sequence as the result of evolutionary selection. The stochastic component of the biased search is necessary to avoid being trapped in local minima on the potential energy surface. Trapping in such metastable states has been observed in protein folding trajectories, where an escape is made possible only by the stochastic nature of the dynamics.
Analogous considerations apply to the navigation dynamics of worms and fly larvae. A purely random search would by very inefficient due to the large size of the space accessible in their normal environment. Thus, living organisms use cues to bias their search. An example is a temperature gradient which plays the role of the potential energy. A purely deterministic search would not be effective here either, because there can be traps (local minima) in the accessible space. These minima could have a physical origin or be due to a complex nonmonotonic nature of the cues. A stochastic component in the biased search allows the organisms to overcome the trapping problem. The actual details of the navigational dynamics are specified by the neural circuitry that enervates the muscles. This is optimized by evolution, in analogy to the amino acid sequence in proteins.
The correspondence outlined above suggests that it would be of interest to see whether approaches developed for understanding protein folding dynamics can be used to study the navigational dynamics of worms and larvae. The folding dynamics can be quantitatively described as diffusion (random walk) on a freeenergy landscape. In particular, the free energy, $F$, defines the equilibrium probability of the system to be found at a particular position $P\sim exp(F/kT)$, that is, the system prefers regions with low free energy. The freeenergy barrier between unfolded and folded states defines the bottleneck of the folding reaction. The diffusion coefficient describes how quickly the system—whether a worm, larvae, or peptide sequence—explores the configuration space. Together, the freeenergy barrier and the diffusion coefficient determine the rate of the process. Such a picture provides a simplified and intuitive, while quantitatively accurate, description of the dynamics. We note also that the freeenergy landscape framework is generic and has been successfully applied to many different types of complex dynamics, for example, the dynamics of the game of chess (Krivov, 2011b) or patient recovery dynamics after kidney transplant (Krivov et al., 2014), as well as to protein folding.
Detailed descriptions of worm or larva dynamics (i.e. how run — turn — run — $\mathrm{\dots}$ sequences are chained together) are important to show how complex navigational dynamics are realized in a particular case. However, there are many variants of detailed motions, which are likely to result in very similar larger scale navigational dynamics. Thus, it is of interest to understand and accurately characterize the invariants of largescale dynamics. It is precisely these invariants that are expected to be optimized by evolution. Moreover, a description making use of the freeenergy landscape framework can provide an intuitive picture of the complex navigational dynamics as a whole, versus the localized description of dynamics. It could be used, in particular, to locate equilibrium populations, biases, and bottlenecks during the navigation toward the target in complex environments.
The freeenergy landscape of a protein can be determined from long equilibrium trajectories (Krivov and Karplus, 2004; Krivov, 2011a; Banushkina and Krivov, 2016). However, the experimental trajectories of the crawling animals treated here are too short to be considered to be at equilibrium in comparison with those in the reversible folding/unfolding of proteins at equilibrium (Shaw et al., 2010). To determine the equilibrium properties, which are required for the construction of the freeenergy landscape, we introduce another general approach to random dynamics, the Markov state model (MSM). This exploits the information contained in a large number of short trajectories measured under identical conditions (Lane et al., 2011; Rao and Caflisch, 2004; Krivov et al., 2002). One refers to the dynamics as Markovian if the next crawling step of an animal depends only on its current spatial position; that is, it quickly forgets the history of its motion. A collection of short trajectories can then be used to determine a probability distribution of future positions of the animal starting from a current position. In particular, the steady state probability distribution can be determined in this way. The analysis is based on the construction of the transition probability matrix, as described in Materials and methods, where it is shown that the steady state distribution is, in fact, the equilibrium distribution for the worms. This matrix provides a complete description of the stochastic dynamics and can be used to determine long time scale behavior.
We first observed the power of this approach when investigating C. elegans trajectories in a thermal gradient, with worms placed at their cultivation temperature. It had been thought that the worms would equally avoid both lower and higher temperatures. The analysis of a large number of trajectories with the MSM showed that worms do not strictly avoid warmer temperatures, potentially uncovering a different interpretation of isothermal tracking behavior (see Results section). Encouraged by this result, we extend the protein folding approach for combining trajectories to a more general study of C. elegans and Drosophila larvae. Specifically, we employ the diffusion coefficient, $D(t)$, which represents the rate of change of the mean square displacement as a function of time for the data set. For both species, in the presence of environmental gradients (e.g. thermal or chemical), it is found that $D(t)$ increases linearly with time for short times (ballistic dynamics), while it approaches a constant value at longer times (stochastic dynamics).
In what follows we investigate the behavior of C. elegans and Drosophila larvae in the presence of different environmental gradients within this framework. Given the recent interest in search strategies in the absence of information (Polani, 2009; Calhoun et al., 2014), we also study the motion of both species in a uniform environment (i.e. in the absence of applied gradients).
Results
Diffusion and search patterns under isotropic conditions
Navigation in C. elegans, Drosophila, and other organisms has been treated as a biased random walk (Berg, 1993; PierceShimomura et al., 1999; Ryu and Samuel, 2002), where animals repeatedly transition between bouts of relatively straight forward crawling (‘runs’) and distinct, often large changes in heading (‘turns’). To investigate the relationship between trajectories built in this fashion and more general phenomena of diffusion and Markovian processes, we first studied 2D free crawling behavior in both worm and larva systems in isotropic environments with no applied stimulus. These trajectories (Figure 1A,A’) do exhibit diffusive behavior, but do not demonstrate active movement in a particular direction, as demonstrated by their very small values for the dimensionless drift velocity (see Materials and methods).

Figure 1—source data 1
 https://doi.org/10.7554/eLife.30503.004
We observed that worm and larvae dynamics at small time scales are close to deterministic—that is, the animals maintain direction and their trajectories are smooth. At longer time scales movement becomes stochastic or diffusive; in other words, the dynamics in configuration space can be approximately described as Markovian. To estimate the time scale of the transition between these regimes, we inspect the time dependence of the diffusion coefficient, $D(t)$, defined in Materials and methods (Figure 1B,B’). For deterministic ballistic dynamics in the $x$ direction, $\mathrm{\Delta}x\sim {v}_{x}t$ and ${D}_{x}(t)$ increases linearly with time, while for diffusive dynamics, ${D}_{x}(t)$ is constant. Figure 1B,B’ suggest that the transition from deterministic to diffusive regimes happens at $t\sim 1000$ s.
C. elegans crawling under isotropic conditions drastically reduce their turning rate (i.e. make longer runs) throughout an experiment (Figure 1B), as also noted in previous work (Calhoun et al., 2014); here, it is studied over a substantially longer time of ~1 hr. We define the turning rate of a population as the total number of turns made, divided by the total time all animals put together spend in forwardcrawling runs (i.e. the total time where animals could have turned, but did not—see Materials and methods for details). In particular, the turning rate decreases exponentially with a time constant of approximately 800 s (Figure 1B). Inspecting the run durations for individual worm trajectories (Figure 1—figure supplement 1), we do not see strong evidence of the turning rate undergoing an abrupt transition from local to global searching, but rather a regular decline, which in the population averages to steady exponential decay.
Turning rate has a clear connection to the dynamics, as frequent turns within a random walk will reduce the diffusion rate. Noting the dramatic reduction in turning rate in Figure 1B, we determined the diffusion coefficients in $x$ and $y$ from the first part of the experiment, $0$ to 900 s, and then separately for the next 900 s. As expected, the diffusion coefficients converge to different limiting values, with diffusion during the second part of the experiments nearly double that during the first part. This dependence agrees with the simple mean free path estimate of the diffusion coefficient $D\sim {v}^{2}/R$, where $v$ is the crawling speed, and the mean turn rate $R$ in the first half of the trajectory is approximately double of that in the second part. We note that ${D}_{x}$ and ${D}_{y}$ are essentially identical throughout the experiments, only diverging slightly at longer times where the uncertainty has increased (fewer individual tracks last up to 1000 s).
Drosophila larvae under isotropic conditions (Figure 1A’–C’) exhibit similar behavior in terms of trajectory structure (Figure 1A’) and the transition to a diffusive regime, but they do not exhibit a marked decline in turning rate (Figure 1B’), which stabilizes after only a few minutes and remains constant throughout their searching behavior. Thus, the transition from local to global searching does not appear on the ~15min time scales we measured for larva behavior. Given the relatively constant turning rate, it follows that diffusion coefficients calculated for the first ($0$ to 450 s) and second (450 to 900 s) halves of the experiments converge to similar values. Further, in this larger data set ${D}_{x}$ and ${D}_{y}$ are essentially identical throughout (Figure 1C’). We also note that while $D$ and the turning rate both increase in the first 100 s, the system has not entered a diffusive regime, and the increase in $D$ can be attributed to the ballistic character of the trajectories at this stage.
Taken together, these data show similarity between C. elegans and Drosophila in the makeup of run — turn — run — $\mathrm{\dots}$ sequences in crawling behavior, and both conform to a model of diffusion at longer times, but the two animals differ in their long time scale search strategies.
Diffusion persists alongside thermotaxis and chemotaxis
We next sought to determine what happens to the behavior as animals navigate while exposed to a stimulus along one axis of the crawling surface. Is the diffusive behavior maintained along the axis perpendicular to the stimulus gradient while motion along the parallel axis transforms into a new mode? To investigate this, we observed both C. elegans and Drosophila navigating along a 1D spatial temperature gradient. The apparatus (Figure 2A), as previously described (Klein et al., 2015), maintains a stable linear gradient in $x$, and constant temperature in the $y$direction for fixed $x$values. Worms cultivated at 15°C and placed in a gradient centered at 20°C exhibited negative thermotaxis (also called ‘cryophilic’ behavior), while larvae placed at 17.5°C in the same gradient crawled away from cold conditions, exhibiting positive thermotaxis to a preferred range that is independent of cultivation conditions.

Figure 2—source data 1
 https://doi.org/10.7554/eLife.30503.007
Figure 2B,B’ shows significant diffusion in both $x$ and $y$ directions, even though both types of animals are migrating along the $x$axis. This suggests that navigation does not eliminate stochasticity along the axis of purposeful navigation. That is, the animals conduct a random search in all directions irrespective of whether they have adopted a target direction. At the same time, the limiting values of ${D}_{x}$ and ${D}_{y}$ are not equal, with diffusion in the $y$direction greater than diffusion in the $x$direction. This suggests that there is a tradeoff between searching along an axis and purposeful travel in that direction.
Both animals move along the $x$direction toward more favorable environmental conditions by biasing their turning rates. For example, worms undergoing negative thermotaxis reorient their crawling direction more frequently when heading up the temperature gradient, and maintain longer runs when heading down the gradient (Figure 2B, lower right). Thermotaxing larvae, similarly, have a higher turning rate when crawling toward aversive colder conditions, and maintain longer runs crawling up the gradient (Figure 2B’, lower right). As was true for isotropic conditions, worms decrease their turning rate over time (and larvae maintain a stable level). However, both animals maintain an approximately constant ratio between towardwarm turning rates and towardcold turning rates. That is, the primary navigational bias that produces thermotaxis is not altered, even at long time scales. Importantly, this supports the method of using early behavior to model long term behavior.
When the dynamics are Markovian, as described in the Introduction, one can use short experimental trajectories to determine the long term equilibrium probability distribution of worms and larvae. Figure 2C,C’ shows the distribution of both types of animals along the $x$ and $y$axes, as determined using the Markov state model (see Materials and methods). Distributions are computed using lag times near the transition to diffusive dynamics. As lag time increases, the dynamics become more Markovian and the distributions converge to the limiting distribution( Figure 2—figure supplement 1B). The remaining fluctuations are due to relatively small statistics at long times. The limiting distribution along the $y$ axis, ${P}_{eq}(y)$, is constant (up to fluctuations around the boundaries), in agreement with absence of any stimulus along $y$. For worms, the limiting distribution along the $x$ axis, ${P}_{eq}(x)$, is approximately constant for $T<{19.5}^{\circ}$C and then decreases exponentially for $T>{19.5}^{\circ}$C. For larvae, the distribution is approximately constant for $T>{17.5}^{\circ}$C and then decreases exponentially for $T<{17.5}^{\circ}$C. This demonstrates that the worms and larvae diffuse towards $x$ values of their preference, as well as remaining in the regions of their preference, if they are already there. We indicate a more direct connection to the protein folding methods by showing the $x$distributions in terms of the free energy $F/kT$ (Figure 2C,C’, right).
To confirm that these crawling dynamics are not unique to a temperature response, we examined navigation of C. elegans exposed to a chemical stimulus corresponding to a 1D linear salt concentration gradient, previously described in Luo et al. (2014). Worms chemotax either up or down salt gradients, depending on the baseline salt level (Figure 3A). At low baseline salt concentrations (25 mM), worms move toward higher salt levels, and at high concentrations (75 mM) they crawl down the gradient toward lower salt levels. As with thermal navigation, worm behavior converges to diffusive behavior at longer times (Figure 3B), and local searching transitions gradually to global searching via a reduced turning rate (Figure 3C). The Markov state model predicts equilibrium population distributions consistent with the net motion of the population (Figure 3D).

Figure 3—source data 1
 https://doi.org/10.7554/eLife.30503.009
Despite relatively deterministic movement along one axis, the equilibrium distributions show that the worms and larvae are dispersed over a significant range. This enables them to avoid local 'traps’ arising from chemical or thermal cues.
C. elegans diffuse towards warmer temperatures during isothermal tracking
As noted in the Introduction, we applied our diffusion analysis to the distinctive C. elegans behavior of isothermal tracking (Hedgecock and Russell, 1975; Luo et al., 2006). In this behavior, worms placed near their original cultivation temperature (${T}_{cult}$) will follow isotherms with extreme precision, indicating a high degree of sensitivity in their thermal response. Since the temperature in our 1D thermal gradient is approximately constant in the $y$direction, we expect to observe qualitatively different trajectories, with more prominent movement in that direction, and very limited navigation in the perpendicular $x$direction.
Although we do observe greater diffusion and navigation in the $y$direction (Figure 4B), an examination of $x$direction navigation revealed a significant asymmetry. The longterm equilibrium position probability distributions (${P}_{x}$ and ${P}_{y}$) are approximately constant in $y$, but not in $x$ (Figure 4C). In particular, there is an extremely low probability for the worms to be in the $T<{T}_{cult}$ region, and a substantially higher probability for them to occupy warmer regions. This suggests either a mild preference for $T>{T}_{cult}$, specific aversion to $T<{T}_{cult}$, or some other disruption of the traditional interpretation of isothermal tracking behavior.

Figure 4—source data 1
 https://doi.org/10.7554/eLife.30503.011
Discussion
We studied the navigation of C. elegans and Drosophila larvae in both isotropic environments and stimulus gradients to assess the relationship between directed movement toward target conditions and the diffusive properties of the overall search patterns. We also studied the negative thermotaxis of C. elegans moving toward colder temperatures and the positive thermotaxis of Drosophila larvae moving towards warmer temperatures. These behavioral modes represent the better studied forms of thermotaxis in these animals. We then examined the ascent and descent of C. elegans moving toward preferred salt concentrations.
Treating the motion of small animals in isotropic environments as diffusive random walks is an established method (Berg and Brown, 1972; Berg, 1993), even yielding analytic solutions under certain conditions (Lovely and Dahlquist, 1975). Here, we have focused on diffusion along perpendicular axes, and used Markov analysis techniques to investigate the combination of exploratory diffusion and targeted navigation. We found that the general framework of diffusion and Markov processes can be used to combine a large number of short trajectories obtained under identical conditions in both isotropic environments and in the presence of stimulus gradients. This approach made it possible for the first time to quantify the statistics of a random search that is concurrent with steady progression towards favorable environments. In both animals and across stimulus types, we found importantly that random exploration in all directions and across all time scales is remarkably robust to progression in a selected direction in a graded environment. That is, the animals undergo diffusive motion (as opposed to ballistic) in both the $x$ and $y$directions, even during persistent navigation along the $x$axis. The diffusion coefficients ${D}_{x}$ and ${D}_{y}$ are not equal during thermotaxis and chemotaxis, but $D$ vs. $t$ plots become approximately constant, indicating a diffusive regime. When nematodes and insect larvae encounter stimuli that bias their random walks in specific directions, the effectiveness of random searching is largely unaffected either parallel or orthogonal to the direction of motion. In these animals, a constraint on the mechanisms that generate navigation in a preferred direction appears to be the preservation of the statistics of random exploration in all directions across time scales. Analysis of the entropy of trajectory configurations, which avoids settling into traps, provides information that is not readily apparent in conventional metrics of drift rates and stimulusevoked turn rates. Moreover, the approach makes possible largescale and long time descriptions of the navigational dynamics beyond those available from the standard localized run — turn — run measurements.
We found, in both animal model systems investigated here, that the transition from ballistic to diffusive motion during navigation occurs over a ~1000 s time scale, which is longer than most behavior experiments in studies of these animals. Experiments are typically limited by animals leaving the arena, especially for faster moving late instar Drosophila larvae. Combined with the observation, in broad agreement with recent results from other experimenters (Calhoun et al., 2014), that the rate of behavioral transitions changes over time (especially in worms), it is possible that further behavioral transitions at longer time scales have yet to observed. Experimental techniques that enable longtimescale measurements may be essential for uncovering a more complete picture of the behavior in these animal systems. This would also enable further testing of the probability distributions predicted by the Markov state model.
Additionally, we note that the transition to more global searching (lower turning rate) occurs at very different times in the two model systems under consideration here. We speculate that the much greater mass of the second instar Drosophila larva would allow it to delay the transition, as it can afford more time without food. While the global search transition in larvae was not observed on the time scales used here, further experiments could illuminate the issue, such as comparisons in turning rates between fed and starved larvae of the same age—starved animals effectively perform searches, even if not placed in a behavioral arena.
By drawing distinctions between the behavioral transition rates in different crawling directions, we note that the overall changes in the average turning rate (Figure 2) are not accompanied by changes in the ratios of the turning rates. This means the navigational bias is preserved, while other aspects of search strategy are modulated. However, the navigational dynamics studied in the cases presented here are rather simple. Consequently, this work may be considered a proof of principle of the utility of employing methods developed for protein folding to understand the behavior of worms and larvae. It will be of interest to study their navigational dynamics in complex conditions with various obstacles, which in the language of protein folding give rise to both enthalpic and entropic barriers. It is important to know where the bottlenecks are in the navigational dynamics towards the target. How the dynamics changes with time (i.e. learning or habituation) in response to different stimuli and different cultivation conditions should also be examined. We expect that to study such questions the description of the dynamics as diffusion on a freeenergy landscape will be useful for obtaining a global understanding of the processes involved.
Materials and methods
Worm and larva handling
Request a detailed protocolAdult N2 wildtype worms were raised on agar plates ($2\%$ wt./vol) with NGM food. For each experiment, around 20 worms (each approximately 1 mm long) were selected under a dissection microscope, rinsed, and placed with a pipette onto the behavior arena in small water droplets. Upon evaporation of the water droplets, the worms began crawling and their movement was recorded.
Wildtype (CantonS) adult flies were kept in cages (Genesee Scientific) with 6 cm Petri dishes with grape juice and yeast food, with new plates exchanged every 24 hr. Larvae were collected from the plates, with second instar larvae selected by age (2472 hr AEL) and spiracle development of each individual. The typical larva size at this instar is 12 mm in length. For each experiment, between 20 and 30 larvae were rinsed in distilled water, allowed to crawl on agar gel (3% wt./vol) for 5 min, then placed in the behavior arena for video tracking of navigation.
For both worms and larvae, all animals for the experiment are placed on the agar surface together, near the center, with approximately 1 cm separating each animal. Given the small fraction of the available space taken up by the animals, collisions are infrequent. Importantly, when a collision does occur, the event is not flagged as a turn for the purposes of turning rate computation (see below), so if the collision rate decreases over time as animals spread out, the extracted turning rate is not affected.
Video acquisition and behavioral analysis
Request a detailed protocolA 5 MP CCD camera placed above the arena recorded crawling, with images acquired at 5 Hz. Movies were processed using the MAGAT Analyzer software (Gershow et al., 2012), which extracts the position and shape of each animal. Subsequent analysis using custom MATLAB scripts (source code download available, Source code 1) segmented the path of each crawling animal into tracks comprised of a sequence of runs (periods of straight crawling) and turns (cessation of forward movement and orientation to a new direction). The runturnrun… sequences were used for navigation analysis, and the raw trajectories used for diffusion and Markov state model distributions.
The turning rate describes how often animals alter their crawling direction, and changing turning rate as a function of crawling direction is the primary behavioral modulation that leads to navigation. We compute the turning rate in the following way. In a given time window, animal $i$ makes ${n}_{i}$ turns, with periods of forward crawling (‘runs’) in between, each run $j$ of duration $\mathrm{\Delta}{t}_{j}$ (see Figure 1A,A’). The total time spent during runs for this animal is ${T}_{i}=\sum \mathrm{\Delta}{t}_{j}$. The turning rate for the individual animal $i$ is ${r}_{i}={n}_{i}/{T}_{i}$, and the total turning rate for the population during this window is $R=N/T$, where $N=\sum {n}_{i}$ and $T=\sum {T}_{i}$. In particular, $T$ is the total time where an animal could have turned but failed to do so. In Figures 2 and 3 turning rates ${R}_{\theta}$ are computed for different crawling directions, where only turns and runs that occur within the specified cone of crawling direction are counted.
For a navigation strength metric, we used the dimensionless drift velocity, $<{v}_{x}>/<v>$, the average velocity of the population in the $x$direction, normalized by the overall average speed during runs. This serves as a dimensionless measure of navigation strength. A value of +1 would correspond to every animal crawling directly along the $+x$ direction for the entire experiment; a value of $1$ would correspond to $x$ direction crawling; and a value of $0$ would indicate no movement at all, or no bias in crawling direction. For both worms and fly larvae, isotropic conditions result in a very small (order 0.01) navigation strength, while in thermal or chemical gradient environments the navigation strength is of order $0.1$. This metric is also employed in (Luo et al., 2010; Gershow et al., 2012; Klein et al., 2015). Green arrows in Figure 1A,A’, Figure 2A, and Figure 3A indicate the navigation strengths for the full population measured.
Stimulus delivery
Request a detailed protocolFor both Drosophila larvae and adult C. elegans, a temperaturecontrolled 2D platform established a 1D linear spatial gradient. A large aluminum metal block one each side was maintained at a constant temperature. The cold side was maintained with two thermoelectric coolers (TECs) under PID control, with chilled circulating liquid (a water and antifreeze mixture) acting as a dissipation reservoir. The hot side was maintained using resistive heaters under PID control. A thin aluminum slab connected the two blocks, which established a smooth linear gradient in the $x$direction and constant temperature for fixed $x$ in the $y$direction. An agar gel (3% wt./vol. for larvae, 2% wt./vol. for worms) was placed on the slab. For larva experiments, the temperature across the gel ranged from 13°C to 21°C (17°C in the center, 0.36°C/cm gradient); for worm experiments, the temperature range was 18°C to 22°C (20°C in the center, 0.19°C/cm gradient).
For C. elegans experiments using salt concentration gradients, agar gels were poured in two stages to establish a stable, linear salt concentration gradient. We followed the procedure outlined in Luo et al. (2014).
Determination of steady state and equilibrium probabilities
Request a detailed protocolThe equilibrium probabilities (${P}_{eq}(x)$ and ${P}_{eq}(y)$) were computed using the Markov state model (MSM) formalism. To this end, the coordinate (either $x$ or $y$) was partitioned into bins with size $\mathrm{\Delta}=1$ and the numbers of transitions from bin $i$ to bin $j$ after time interval $\mathrm{\Delta}t$ (${n}_{ji}(\mathrm{\Delta}t)$) were computed. The transition probability matrix ${P}_{ji}(\mathrm{\Delta}t)$, the probability to move to bin $j$ from bin $i$ after time interval $\mathrm{\Delta}t$ was estimated as ${P}_{ji}(\mathrm{\Delta}t)={n}_{ji}(\mathrm{\Delta}t)/{\sum}_{j}{n}_{ji}(\mathrm{\Delta}t)$. This matrix describes the time evolution of the probability vector as ${P}_{i}(t+\mathrm{\Delta}t)={\sum}_{j}{P}_{ij}(\mathrm{\Delta}t){P}_{j}(t)$. The stationary, steady state probability distribution ${P}^{st}$ is computed as the solution of equation ${P}_{i}^{st}={\sum}_{j}{P}_{ij}(\mathrm{\Delta}t){P}_{j}^{st}$.
We have also checked whether reversibility and detailed balance are satisfied. First, we computed the steady state fluxes in positive ${J}^{+}(x)={\sum}_{j<x<i}{J}_{ij}$ and negative ${J}^{}(x)={\sum}_{i<x<j}{J}_{ij}$ directions, where ${J}_{ij}={P}_{ij}(\mathrm{\Delta}t){P}_{j}^{st}$ is the steady state flux from bin $j$ to bin $i$. The fluxes agree with high accuracy (Figure 2—figure supplement 1A), meaning that the net flux is zero and we can consider the steady state probability as the equilibrium probability.
The detailed balance, is a more stringent condition, where the fluxes between any two bins must be equal ${J}_{ij}={J}_{ji}$. Due to the limited statistics, and thus higher noise, direct comparison of the fluxes between bins is not informative. We compared a related quantity —the steady state fluxes in positive ${J}_{a}^{+}(x)={\sum}_{i,a<x<i}{J}_{ia}+{\sum}_{i,i<x<a}{J}_{ai}$ and negative ${J}_{a}^{}(x)={\sum}_{i,a<x<i}{J}_{ia}+{\sum}_{i,i<x<a}{J}_{ai}$ directions, restricted to transitions to or from a particular node ($a$). The fluxes between bin $x$ and bin $a$ are proportional to the derivatives $d/dx{J}_{a}^{+}(x)$ and $d/dx{J}_{a}^{}(x)$ and hence from ${J}_{a}^{+}(x)\sim {J}_{a}^{}(x)$ it follows that ${J}_{xa}\sim {J}_{ax}$. Figure 2 supplemental A compares ${J}_{a}^{+}(x)$ and ${J}_{a}^{}(x)$ for e.g., $a=2$. Increasing statistics by considering all the bins in $1.5<a<2.5$ improves the agreement.
The sampling interval (the lag time) $\mathrm{\Delta}t$ should be chosen sufficiently large so that the dynamics become Markovian. Figure 2 supplemental B shows how with increasing lag time the determined equilibrium probabilities converge to the limiting one.
Inclusion of other parameters such as the body angle and whether the animals were stationary or moving did not significantly change the results.
Determination of diffusion coefficients
Request a detailed protocolFor flat freeenergy profiles, with no drift term, $F(x)\sim \mathrm{const}$, the diffusion coefficient can be estimated as ${D}_{x}(t)=\frac{1}{2}\u27e8\mathrm{\Delta}{x}^{2}(t)\u27e9/t$. For freeenergy profiles with constant drift term, $F(x)\sim ax$, ${D}_{x}(t)=\frac{1}{2}\u27e8{(\mathrm{\Delta}x\mathrm{\Delta}{x}_{avg})}^{2}\u27e9/t$, where $\mathrm{\Delta}{x}_{avg}=\u27e8\mathrm{\Delta}x\u27e9$ is the averages of the corresponding displacements after the time interval $t$. The statistical uncertainties were estimated by bootstrapping.
Changepoint detection in C. elegans trajectories
Request a detailed protocolThe change points in Figure 1 (supplemental) were computed using the ‘findchangepts’ function in MATLAB, which detects the point in a sequence with the maximum difference between the means of values below the point and the mean of the values above the point.
References

Optimal reaction coordinatesWiley Interdisciplinary Reviews: Computational Molecular Science 6:748–763.https://doi.org/10.1002/wcms.1276

Controlling airborne cues to study small animal navigationNature Methods 9:290–296.https://doi.org/10.1038/nmeth.1853

The Levinthal paradox: yesterday and todayFolding and Design 2:S69–S75.https://doi.org/10.1016/S13590278(97)000679

The free energy landscape analysis of protein (FIP35) folding dynamicsThe Journal of Physical Chemistry B 115:12315–12324.https://doi.org/10.1021/jp208585r

Optimal reaction coordinate as a biomarker for the dynamics of recovery from kidney transplantPLoS Computational Biology 10:e1003685.https://doi.org/10.1371/journal.pcbi.1003685

Markov state model reveals folding and functional dynamics in ultralong MD trajectoriesJournal of the American Chemical Society 133:18413–18419.https://doi.org/10.1021/ja207470h

Statistical measures of bacterial motility and chemotaxisJournal of Theoretical Biology 50:477–496.https://doi.org/10.1016/00225193(75)900946

Sensorimotor control during isothermal tracking in Caenorhabditis elegansJournal of Experimental Biology 209:4652–4662.https://doi.org/10.1242/jeb.02590

Navigational decision making in Drosophila thermotaxisJournal of Neuroscience 30:4261–4272.https://doi.org/10.1523/JNEUROSCI.409009.2010

The fundamental role of pirouettes in Caenorhabditis elegans chemotaxisJournal of Neuroscience 19:9557–9569.

Analysis of the effects of turning bias on chemotaxis in C. elegansJournal of Experimental Biology 208:4727–4733.https://doi.org/10.1242/jeb.01933

The protein folding networkJournal of Molecular Biology 342:299.https://doi.org/10.1016/j.jmb.2004.06.063

Thermotaxis in Caenorhabditis elegans analyzed by measuring responses to defined Thermal stimuliJournal of Neuroscience 22:5727–5733.

Neurons regulating the duration of forward locomotion in Caenorhabditis elegansNeuroscience Research 50:103–111.https://doi.org/10.1016/j.neures.2004.06.005
Decision letter

Yibing ShanReviewing Editor; DE Shaw Research, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
Thank you for submitting your article "Exploratory search during directed navigation in C. elegans and Drosophila larva" for consideration by eLife. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and K VijayRaghavan as the Senior Editor. The reviewers have opted to remain anonymous.
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
Summary:
In this manuscript, the authors attempted to apply the framework of diffusion processes and Markov State Models (MSMs) to analyze the navigational dynamics of C. elegans and Drosophila larva in an isotropic environment and environments with temperature or salt concentration gradients. They found that in the absence of external stimuli, both organisms follow deterministic dynamics at small time scales, while switch to stochastic (or diffusive) dynamics at long time scales. It was further shown that the introduction of temperature or chemical stimuli has little impact on the diffusive random motion, even though worms eventually navigate towards the favorable environments. Overall, the results provide new insights in understanding navigational dynamics of different organisms and point at possibilities of applying methodologies of analyzing protein folding simulations to these navigational trajectories.
Essential revisions:
The analogy made between the navigation organisms and protein folding should be better explained in the text, especially in the introduction part. The manuscript will be stronger if the authors can elaborate further the link between the search process approaching to nonequilibrium steady state, and the free energy landscape of protein folding, beyond stating that "the navigational dynamics of worms and larvae have some parallels with the complex dynamics of a polypeptide chain navigating to the native structure of the protein to which it corresponds. Both dynamics are stochastic, both need to avoid traps due to local minima, and both were developed by evolution. Hence it is of interest to see whether approaches developed for understanding protein folding dynamics can be used to study the navigational dynamics of worms and larvae.…" Protein folding free energy landscape is rugged and contains numerous metastable states leading to the separation of timescales. For the navigational dynamics of worms, are there also metastable states along the direction of the gradient? Judging from Figure 2C and C' (right panel), it seems that a number of metastable regions do exist. If so, what are the features of these states?
The manuscript needs to further demonstrate the Markovian nature of the studied diffusive and thus the applicability of MSM analysis? When MSMs are applied to protein folding, it is implied that the detailed balance is satisfied due to the reversibility of molecular dynamics simulations. Is this the case also for worms' navigational trajectories? When the authors calculate the transition counts, do they observe a symmetric pattern? In particular, do they observe substantially more counts moving forward than backward along the stimulus gradient (e.g. the first panel in Figure 2C)? If their data is largely deviated from the detailed balance, the authors may not obtain faithful estimation of the equilibrium populations.
It would be great if the authors can provide evidence to show that, as stated in the manuscript, there are no active movements. For example, histograms of displacements showing in average there is no net movement will serve this purpose.
The organisms or active particles do reveal biased movements in the short time scale (ballistic motions). The authors should comment on why no biased movements appear in the long time scale (no stimulus), even though there is basically no energetic constraint for the active particles.
C. elegans and Drosophila larva display different behaviors in their navigational dynamics. For example, the transition from local to global search for C. elegans occurs at ~900s (with turn rate reduced by half), while this transition occurs at a longer timescale (relatively constant turn rate). From the biological point of view, could the authors provide some explanation?
https://doi.org/10.7554/eLife.30503.014Author response
Essential revisions:
1) Improve the explanation of the analogy between navigating animals and protein folding.
We have expanded our Introduction section with an additional two paragraphs that make more explicit the connection between protein folding and the navigational dynamics of the two invertebrate systems. We have added more specific examples of traps and free energy landscapes, and noted that this framework has been applied to other systems as well (Introduction, fourth, fifth and sixth paragraphs).
2) Demonstrate in more detail that Markovian description is applicable
We have expanded the Materials and methods section to make the Markovian formulation clearer and shown how to obtain steady state probabilities. In addition (and this is not required for Markovian behavior), we demonstrate that detailed balance is satisfied, which means that equilibrium probabilities are obtained from the analysis. New text in the Materials and methods section explains how this was done by looking at fluxes between bins, and a new figure panel (Figure 2—figure supplemental 1A) visualizes it (subsection “Determination of steady state and equilibrium probabilities”, second paragraph; Figure 2—figure supplement 1).
3) Evidence of “no active movement” under isotropic conditions
4) Comment on why no biased movement appears at long time scales
For all figure panels that show representative trajectories we have added a green arrow that indicates the drift velocity of the population in the xdirection. This denotes a dimensionless drift velocity, normalized to the average speed of animals, and gives the strength/efficiency of navigation. Details of how this is computed (this metric has been used in other work, which we have cited) is included in Materials and methods. We note here that under isotropic conditions the navigation strength is approximately 10x weaker than during thermotaxis or chemotaxis. Individual animals may show biased movement, but the population on average does not (subsection “Diffusion and search patterns under isotropic conditions”, first paragraph; subsection “Video acquisition and behavioral analysis”, last paragraph and Figures 1, 2, 3 and legends).
5) Local → Global search transitions in worms vs. larvae
While we claim that this transition is not as abrupt as previously thought, it is true that it is not observed during the ~1000 s time frame of our larva experiments. We suspect this could be mainly due to much greater mass of larvae compared to worms, and have added this speculation as a paragraph in the Discussion section, along with an idea of how this could be measured in Drosophila with systematically starved animals (Discussion, fourth paragraph).
https://doi.org/10.7554/eLife.30503.015Article and author information
Author details
Funding
National Science Foundation (BRAIN Initiative EAGER Award)
 Aravinthan DT Samuel
National Institutes of Health (1P01GM103770)
 Aravinthan DT Samuel
CHARMM Development Project
 Martin Karplus
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
The authors thank Kevin Collins and Sheyum Syed for comments on the manuscript. ADTS is supported by grants from the NSF and NIH. MKa is partially supported by the CHARMM Development Project.
Reviewing Editor
 Yibing Shan, DE Shaw Research, United States
Publication history
 Received: July 20, 2017
 Accepted: October 11, 2017
 Version of Record published: October 30, 2017 (version 1)
Copyright
© 2017, Klein et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 1,897
 Page views

 304
 Downloads

 16
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.