We model the locomotion of wild-type flies exploring a circular arena (Jung et al., 2015) whose center (odor-zone) consists of a fixed concentration of odor (Figure 1A). The arena and the experimental procedure was previously described (Jung et al., 2015). Briefly, locomotion of each of the 34 flies in our dataset was measured 3 min before an odor (apple cider vinegar or ACV) was turned on, and 3 min during the presence of ACV. Sample trajectories are shown in Figure 1B.

Figure 1 with 1 supplement see all

Download asset Open asset

We first attempted to model the fly’s locomotion using Hidden Markov Model (HMM) (Gallagher et al., 2013; Isakov, 2016). HMMs create discrete states based on a time series of observables such as position, speed or acceleration. The advantage of using HMMs in modeling locomotion is well described in earlier studies (Gallagher et al., 2013) (see Materials and methods).

In this study, we use observables that describe the change of position as a function of time, and hence our analysis will focus on behavioral states in the velocity space. Speed and angular speed are commonly used measures of velocity. But, because it is difficult to measure angular speed accurately at low speeds (Gallagher et al., 2013) (see Materials and methods section 2 for details), we fit the model to the component of speed parallel (${\displaystyle {\hat{v}}_{||}}$) and perpendicular (${\displaystyle {\hat{v}}_{\perp }}$) to its movement during the previous time point (Figure 1C, Materials and methods section 2). If the fly walks straight ahead, ${\displaystyle {\hat{v}}_{\perp }}$ would be zero, therefore, ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$are closely related to speed and angular speed. We fit a time series of ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ to an HMM. The HMM architecture is shown in Figure 1D. We employed models with 24 to 50 states. HMM was only modestly successful because it was never able to classify >70% of the tracks into one of the states with >85% confidence.

The transition probability matrix for the HMM was sparse (Figure 1—figure supplement 1) suggesting that from each state there are transitions to only a handful of other states. One method to improve upon HMM performance is to cluster the states obtained by HMM according to the transition probability matrix. A common approach to clustering is to block-diagonalize the transition probability matrix (Berman et al., 2016). Tracks corresponding to the 10 states obtained by clustering are shown in Figure 1—figure supplement 1. Some of these states appear to describe recognizable features in the data such as left (state 10) or right turn (state 9). But efforts to block-diagonalize the transition probability matrix were only partially successful. The most obvious failure corresponds to the states with little movement. These states – describing the absence of movement – can occur in many different contexts, such as the fully stopped state or intermittent runs. When the same state is used in different contexts, an approximate block diagonalization of the transition probability matrix fails because the same state belongs to two blocks. In these cases, the different states that correspond to the absence of movement did not cluster, and appeared alone even after block-diagonalization (Figure 1—figure supplement 1). Thus, the existence of the same state in different contexts is one important reason for the modest success of HMM (Marco et al., 2017; Wonjoon et al., 2010; Michele Weiland and Nelson, 2005; Nguyen et al., 2005; Murphy and Paskin, 2001; Chou, 2006).

We employed a two-layered Hierarchical Hidden Markov model (HHMM) to model the data (Figure 2A). We reasoned that the low-level states (LL states) would be represented by Gaussian distributions on the observables, and the high-level states (HL states) would therefore be a mixture of Gaussians and would be able to model the experimental data better. Moreover, these HL states would have longer duration than the states discovered by HMM allowing it to more naturally model composite states.

Figure 2 with 3 supplements see all

Download asset Open asset

Indeed, in a Bayesian model comparison (see Materials and methods section 4), HHMMs outperformed HMMs with the same number of states. Since HMMs rarely used more than 35 states (even when models with higher number of states were fit), to perform model comparisons, we used models with less number of states than the particular HHMM we eventually employed. We compared a two-level HHMM with 6 HL states and 4 LL states to a single-level HMM with the same number of states (4 × 6 = 24 states). We labeled the non-hierarchical model as the null model, and were able to reject the null model using Bayesian model comparison at p < 0.0001, implying that a hierarchical model is necessary. Another model comparison – a two-level model with 8 HL and 4 LL states compared to a single-level model with 32 states – also yielded similar results. The objectively better performance of an HHMM compared to an HMM suggests that a model that includes a hierarchical structure is more consistent with a fly’s locomotion. It is important to note that, HHMMs are actually simpler than HMMs with the same number of states. This simplicity comes from the fact that any HHMM – which puts very specific constraints on the transition probability matrix – can be represented by an HMM but not vice-versa. HHMMs with the same number of states has far fewer parameters. Thus, for the two comparisons above, the HHMM has 6² + 6*4² + 6*4*5 = 252, and 8² + 8*4² + 8*4*5 = 352 parameters, and the HMM has 24² + 24*5 = 696 and 32² + 32*5 = 1184 parameters respectively; therefore, HHMM has fewer parameters. When a simpler model better characterizes the data, we can conclude that the additional structure contained in that model provides a more accurate characterization of the structure within the data.

The model we chose has 10 HL states (Figure 2A) and 5 LL states for each HL state. The model was fit to the entire dataset – both before and during the presence of the odors. The fitting process initializes by fitting each fly’s tracks to its own HHMM and then clusters these 34 HHMMs – one for each fly – using a Gaussian mixture model, resulting in a smaller number of models. Remarkably, a single HHMM is an excellent fit for all the data suggesting that the behavior of wild-type flies is composed of similar components. The model was able to successfully assign an HL state (defined as >85% confidence) for >80% of the data points (Figure 2B, median 81%). This percentage was consistently high for all flies in our dataset (Figure 2B). In comparison, an HMM with 50 states can only classify 68% of the data with the same level of confidence. Tracks of a fly with each HL state labeled with a different color are shown in Figure 2C.

To make the difference between HHMM and HMM clearer, we compare the HHMM above to a HMM. As expected the time a fly spends in a HMM state is shorter than that in a HHMM state (Figure 2—figure supplement 1A). The longer time a fly spends in a HHMM state results from its hierarchical structure, and allows a HHMM to more accurately assign states, and is illustrated with two examples. First, consider a track that is assigned as a left turn by the HHMM, the HMM only classifies parts of the track as a left turn because of the inability of HMMs to consider longer duration trends in the observables (Figure 2—figure supplement 1B1). Short-term inhomogeneity in the data throws the HMM off; as soon as the ${\displaystyle {\hat{v}}_{\perp }}$ decreases, the HMM exits its left turn state. Another example (Figure 2—figure supplement 1B2) shows that the HMM exits the stopped state as soon as there is a small movement. The net result is that the HHMM can classify all long stops into a single state while HMM needs four different stop states. HHMM also assigns more of the left turn as such (6% compared to only 2% by HMM).

The duration of the LL states of a HHMM is shorter than the duration of the HL state state (Figure 2—figure supplement 2A). Moreover, as the duration of HL states becomes longer, the mean number of transitions increase. The shorter duration of LL states compared to the HL states, and increased number of transitions between LL states within each HL state transition support the idea that there is structure at multiple timescales, and some of this structure is captured by the HHMM.

The limitations of HMM in describing phenomenon which have hierarchical and shared structure because of the short duration of its states is well documented (Marco et al., 2017; Wonjoon et al., 2010; Michele Weiland and Nelson, 2005; Nguyen et al., 2005; Murphy and Paskin, 2001; Chou, 2006). Therefore, an HHMM is an objectively a better model of fly walking data than an HMM.

Both the organized transitions between HL states, and the narrow range of observables associated with each state shows that the fly’s locomotion is structured: The transition probability matrix is sparse – a vast majority of transitions from each HL state were to 2-3 other HL states. When we reordered the states (see Materials and methods section 5) from low-speed-high-turn-states to high-speed-low-turn states, we found that from any state the flies transitioned to the neighboring states with a high probability (Figure 2—figure supplement 3) suggesting a gradual transition from low-speed-high-turn states to high-speed-low-turn states. This gradual transition is not because flies cannot make large transitions due to biomechanical limitations because 47/81 1 possible transitions between HL states have a non-zero probability. Rather, transitions to states with similar kinematics show that under our experimental conditions – locomotion in a dark, small circular arena - flies locomote at similar ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ for extended periods of time, and represent one way in which locomotion is organized.

More important to the organization is the narrow distribution of observables - ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$- associated with each HL-state. The distribution of observables for a HL state is a composite of the distributions of its LL states (Figure 3A). Both the model (solid line in Figure 3A1) and a random sample of observables drawn from the time points assigned to a given LL state (gray markers) show that during each LL state within HL state 10, the observables are limited to a narrow range of values. In each LL state, ${\displaystyle {\hat{v}}_{||}}$ is large and ${\displaystyle {\hat{v}}_{\perp }}$ is negative implying that in HL state 10, flies turn counter-clockwise at high speeds as observed in sample tracks corresponding to a single transition to HL state 10 (Figure 3B). The sample tracks also show that within each transition to a HL state there are multiple transitions between LL states, a signature of the hierarchical organization in our data. Fast, counter-clockwise turns represent a locomotor feature which describes a fly’s locomotion in HL state 10. To better visualize this feature, we translated each track such that it began at the origin and rotated the tracks so that the initial velocity vector pointed along the y-axis (Figure 3C₁, see Materials and methods). These transformations make it apparent that the locomotor feature for state 10 is turning left at high speeds (Figure 3C₂).

Figure 3 with 1 supplement see all

Download asset Open asset

Rotated and translated (as in Figure 3C) tracks for each of the 10 HL states are shown in Figure 4. The distribution of the observables for each HL state is also plotted. HL state 1 represents very slow walking with frequent changes in direction. In state 2, flies are either completely stopped or they walk at a speed about twice the speed of the fly in state 1; state 2 represents stop and start locomotion. The subtle, but important differences between state 1 and state 2 show an instance in which the HHMM is successful at extracting an unexpected feature in the velocity profile in a fly’s locomotion. During state 3, the fly is exhibiting a sharp turn that is reflected in the increase in ${\displaystyle {\hat{v}}_{\perp }}$ with a concomitant decrease in ${\displaystyle {\hat{v}}_{||}}$. These three states together represent slow locomotion.

Figure 4

Download asset Open asset

In states 4-7, flies are walking at a medium-speed. In contrast to the clear drop in ${\displaystyle {\hat{v}}_{||}}$ with increases in ${\displaystyle {\hat{v}}_{\perp }}$ in state 3, ${\displaystyle {\hat{v}}_{||}}$ remains strikingly constant irrespective of ${\displaystyle {\hat{v}}_{\perp }}$. These states are different from each other because ${\displaystyle {\hat{v}}_{||}}$ is slightly different.

States 8–10 are high-speed states; each of these states is also characterized by their turn direction. During states 8 and 9, the fly turns right; the fly’s speed is higher during state 9 than during state 8. During state 10, the fly turns left. States 9 and 10 are mirror-symmetric versions of each other.

Flies spend 60% of time performing a locomotor feature for >300 ms, and >10% of their time performing a single locomotor feature for >3 s (Figure 2—figure supplement 2). Thus, flies spend extended time in the same state.

In the absence of ACV, the state occupancy inside and outside the odor-zone are quite similar: The fly spends 30% of its time in state 2 and roughly equal time in all other HL states. Introducing ACV changes the fly’s locomotor behavior both inside and outside the odor-zone, but with opposite effects on the HL state occupancy in the two zones. Inside the odor-zone, in the presence of ACV, the fly spends more time in HL states 1 and 3 at the expense of time spent in HL states 7–10 (Figure 5A). These changes from high-speed states to low-speed states suggest that in the presence of ACV the fly is performing a local search, presumably to find food. Outside the odor-zone (Figure 5B), the fly spends more time in the high-speed states (HL states 8–10), with a decrease in the occupancy of HL state 2 (which includes stopping). Decreased stopping and increased high-speed walking with turning is likely to represent a different search strategy, wherein the fly might be attempting to re-find the odor it has recently lost. We also investigated whether there were changes in the LL state composition of the HL states and found no changes (Figure 5—figure supplement 1). Overall, these results showed that odors affect locomotion not by creating new locomotor features, but by altering the frequency with which existing locomotor features are used.

Figure 5 with 1 supplement see all

Download asset Open asset

The divergent effect of ACV on the probability of HL states inside and outside the odor-zone is consistent with our previous analysis (Jung et al., 2015) and shows that the effect of ACV can be described by the change in the probability of the fly occupying HL states. To assess whether there is a more fine-grained spatial structure to the effect of ACV on a fly’s behavior, we divided the arena into a 60-by-60 grid and measured the ACV-induced changes in the probability of occupying a given HL state at each of the 3600 locations (Figure 6). The probability that a fly is in HL state 1 increases dramatically only at the edge of the odor-zone (Figure 6A), and not throughout the odor-zone where the odor concentration is uniform throughout, showing that the effect of odor on locomotion has a fine-grained spatial structure. Location-specific change in the probability of each HL state are shown in Figure 6B. The fine-grained modulation of locomotion is observed in other states as well - increases in state 2 are largest in an annular region just inside the odor-zone and increases in state 3 are largest at the very center of the arena. A similar specificity is observed in the increase in the probability of HL states outside the odor-zone. Increases in the occupancy of state 8 are uniform across the entire chamber outside the odor-zone; in contrast, the occupancy of states 9 and 10 increases in the region close to the odor-zone.

Figure 6 with 1 supplement see all

Download asset Open asset

The structure that we observe represents the time-average over the entire duration of the odor period, and ignores the time evolution of the behavior (Figure 6—figure supplement 1 and discussion). We were unable to explore the spatio-temporal evolution of behavior because of substantial fly-to-fly differences in locomotion and how it is modulated by odor.

Given that ACV affects the occupancy of HL states, it should be possible to do the reverse, that is decode the presence of ACV based on the distribution of HL states. Surprisingly, a variety of different decoding techniques failed to decode the presence of ACV based on the distribution of HL states. One such method (Figure 7—figure supplement 1) in which we employed logistic regression to classify each one second of every fly’s track into ‘ACV present’ or ‘ACV absent’ failed. Even more surprisingly, population decoding based on HL states did not perform any better than decoding based on the observables (Figure 7—figure supplement 1). One possibility that the logistic regression approach failed is because the average behavior represented in Figure 5 does not accurately encapsulate individual fly behavior. Large fly-to-fly differences, where different individuals have fundamentally different basal locomotion or response to odor, might doom decoding methods aimed at discovering a single set of regressors that captures individual fly behavior.

Consistent with large fly-to-fly differences in behavior, we found that the distances between empirical flies are much larger than the distances between the synthetic flies (Figure 7A). Synthetic flies were generated as described in Figure 7—figure supplement 7–2. It is statistically impossible (p < 10⁻¹³¹) that the observed Euclidean distance represents variations around the same average fly. The same conclusion applied to the fly’s behavior in the other three conditions (before-inside, during-outside and during-inside: Figure 7—figure supplement 3).

Figure 7 with 5 supplements see all

Download asset Open asset

Because the data in Figure 7A is inconsistent with individual flies being variations around a single average fly. We assessed whether the observed variability can be approximated based on a small number of discrete locomotor-types. X-means clustering (see Materials and methods section 6) showed that there are 4 clusters of flies based on their locomotion outside the odor-zone, before odor onset (Figure 7B). Although the identity of flies that cluster together changed, a similar number of clusters was found in each of the four conditions (Before odor/inside odor-zone, during odor/inside odor-zone, before-outside and during-outside, Figure 7 and Figure 7—figure supplement 3). Importantly, X-means clustering on a set of 34 randomly sampled points from a uniform distribution in the probability simplex space that the data reside in found no clusters. The Euclidean distances between synthetic flies drawn from the four different clusters were similar to the distances between empirical flies (Figure 7B and Figure 7—figure supplement 3). Since X-means clustering tends to underestimate the actual number of clusters in data (Pelleg and Moore, 2000), the analyses in Figure 7 and Figure 7—figure supplement 3 suggest that there are at least three to four fly-types based on the frequency with which they use the HL states.

Consistent with the analysis with Euclidean distances above, the KL divergences (Figure 7C, left set of data points) show a large range indicating that some, but not all, flies are well-represented by the population average while others are not. Employing three to four fly-types defined as average distribution of their respective cluster decreased the information loss (Figure 7C, right).

How was the behavior of the flies in the four clusters different from each other? The average occupancy of the HL states for flies in each cluster and one example from each cluster is shown in Figure 7D. Cluster two was distinct from the others because flies move at markedly slower speed and spent >60% of their time in State 2 (Figure 7D) during which the fly was often stopped. Locomotion of flies in the largest cluster (cluster 1) was characterized by an alternation between medium- and slow-speed states. Flies in this cluster employed states 5 and 7 with a high frequency while making radially inward forays into the center of the arena. Similar behavior was observed in cluster 3, except that the flies employed the slower medium-speed states (states 4 and 5). Finally, the fourth cluster of flies demonstrated a different locomotor strategy. Flies in cluster four traversed the arena in concentric circles using the high-speed states (states 8–10). Thus, X-means performed on the HL state distributions appear to identify different locomotor strategies employed by the fly.

How similar is the locomotion of a fly at different times during a trial? To investigate this issue we first examined whether the mean behavior of flies from different clusters are different enough that they can be accurately clustered based on a small sample of the HL states (Figure 7—figure supplement 4A). We limited the analysis to before-outside and during-inside scenarios because the fly spent much of its time in these two scenarios. Only a one-second chunk of data is sufficient for better than chance clustering, and just 30 s of sampling is enough to accurately classify >85% of the flies into their respective clusters. We performed two analyses to test whether a fly’s behavior is persistent: First, we divided the tracks into bins of different length, and asked whether cluster assignments based on small bin sizes is stable (Figure 7—figure supplement 4B). We found that state distribution within each bin was highly predictive of the cluster they belonged to. Second, we repeated the same analysis, but with bins of varying size starting from the first data point or ending at the last data point (Figure 7—figure supplement 4C). These analyses show that within the admittedly short timeframe of our experiments, the cluster assignments are stable.

Apart from the fly-to-fly variability, another reason why decoding based on the average fly fails is that the behavior of the fly before odor onset is only weakly predictive of its behavior during the presence of odor. Some flies exhibited similar behavior in the before odor/outside odor-zone, but were divided into separate clusters in the presence of odor because their locomotion differed (e.g. flies 11 and 33, and 17 and 27;Figure 7—figure supplement 5A, see the distributions of HL states).A similar trend is observed inside the odor-zone (Figure 7—figure supplement 5B). These examples imply that behavior before odor onset is unlikely to be strongly predictive of behavior during the presence of odor. This conclusion is supported by the weak correlation between Euclidean distances between pair of flies in the before and during periods (Figure 7—figure supplement 5A and Figure 7—figure supplement 5B).

The analysis presented above suggests that individual differences explain why the logistic regression approach based on the average HL state distribution across flies failed to decode the presence of ACV from its absence based on the HL states usage by individual flies. If so, individualized logistic regression should be more successful. Logistic regression based on individual flies to decode the presence or absence of odor based on HL state occupancy during a 1s-interval was able to classify odor-no odor at better than chance level for every single fly (Figure 8A, see Materials and methods section seven and Figure 8—figure supplement 1 for details). Moreover, as expected, logistic regression using the HL states performed significantly better than did the observables (Figure 8B) which indicate that HL states are more predictive of the presence of ACV than the observables.

Figure 8 with 1 supplement see all

Download asset Open asset

The analysis in Figure 8A shows that the occupancy of HL states is predictive of the presence or absence of odor when the analysis is performed at the level of individual flies. Does this mean that each fly follows an individualistic strategy? To evaluate whether a fly’s response to ACV cluster into a small number of response types, we once again started with X-means clustering based on the change in state occupancy before and during odor. X-means clustering found five clusters inside the odor-zone and four clusters outside the odor-zone (Figure 8C). Using these clusters as a starting point, we could reconfigure the clusters such that the logistic regression on flies in each cluster performed at a better than chance level for each fly in the cluster (Figure 8C, see Materials and methods section seven for details), thus implying that a fly’s response to odors can be approximated as a choice between few response-types.

Based on their behavior inside the odor-zone, the flies were divided into five clusters, four of these clusters have more than three flies (Figure 9A). Flies in cluster 5, just like the average fly (in Figure 5), slow down inside the odor-zone. Flies in cluster 3 also demonstrate a strategy similar to flies in cluster five except that ACV causes a large decrease in the time a fly spends in the medium-speed states rather than the high-speed states. In contrast to clusters 3 and 5 during which the fly slows down inside the odor-zone, flies in cluster two demonstrate a fundamentally different strategy in which there is a large decrease in state 2 in favor of states 1, 3 and 4. The flies in this cluster go from stop-start locomotion to locomotion in which they either meander at slow speeds or walk slowly with many sharp turns. Finally, for the flies in cluster 1, there is no dramatic change in state. These different strategies represent diametrically different effects of ACV on some HL states – the most striking example is the opposing effects of the odor on HL state 2 occupancies in different clusters – a large decrease in cluster 2, and an increase in clusters 3 and 5. These differences explain the odor-induced increase in usage of all the slow states in the average fly inside the odor-zone, except state 2 (Figure 6A).

Figure 9

Download asset Open asset

Based on how odors modulate their behavior outside the odor-zone, there were four clusters. Behaviors that represent three of these clusters are shown in Figure 9B. Flies in cluster 1 decrease the time they spend in slow-states (state 1–2) and instead spend time in the fast states (states 8–10) resembling the behavior of the average fly. Cluster 2) shows a large decrease in HL state 2 occupancy similar to the behavior of flies in inside odor-zone cluster 2 while exhibiting a large increase in medium-speed states (state 4–6). Cluster 3 showed no dramatic change in state.

The model presented here is a model of locomotion and not the model of locomotion. The choice of observables and model strongly influences the features of the structure that is discovered. Our particular model reveals the structure of locomotion in the velocity space.

In choosing the observables, we employ a common method for describing locomotion, that is we treat the fly as a point object and measure the instantaneous change in the position of this point object; therefore, much of the insights from the model relate to how the fly changes its position in time. Apart from ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$, another similar and more commonly used representation of the change in fly’s position: instantaneous speed and angular speed yielded similar locomotor features (data not shown). Ultimately, we used ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ because this representation is more closely related to movement representation within the insect brain (Green et al., 2017; Heinze, 2017; Turner-Evans and Jayaraman, 2016), and because the measurement errors associated with angular speed are particularly large when flies moves slowly (Gallagher et al., 2013).

A fly’s position can also be described using the actual position of the animal as observables rather than the change in the position, as employed in a recent study in rats (Shan and Mason, 2017) Using the instantaneous position as an observable would reveal different aspects of the structure underlying an animal’s locomotion. Consider the trajectories of flies in Clusters 1 and 3 (Figure 7). They cross similar spatial positions, but are classified into different states because the flies travel at different speeds. In terms of the sequence of position in space, flies in both clusters have a similar behavior – they explore the outer arena border and make occasional radial forays inside the odor-zone. An analysis based on position would likely place these two clusters of flies together whereas our analysis, in the velocity space, placed them in different clusters.

Model architecture is also important. A hierarchical model performed better than a non-hierarchical model. The current model has state durations of <3 s. It is clear to human observers that there is structure in the data that is >3 s long. Flies sometime explore the outer border of the arena using characteristic paths that can last up to a minute. The short duration of states in our model cannot capture structure on these long-time scales. One possibility is choosing a deeper-layered architecture. Given the structured transitions between the HL states in our model, it is likely that if we used a deeper-layered architecture, we would likely uncover structure on a longer timescale.

During both HL states 1 and 2, the fly’s locomotion is quite slow, but in state 2 the fly stops and runs intermittently while in state 1, the fly is continuously in motion, albeit slowly. Similarly, in each of the HL states 4, 5 and 7, ${\displaystyle {\hat{v}}_{||}}$ lies within a narrow range, which is distinct for each of these three states, implying a tight control over forward speed. These locomotor characteristics can persist over 3 seconds (Figure 2—figure supplement 2) – a time period during which a fly takes 30 steps on average (given a step frequency of 10 Hz; Mendes et al., 2013). This tight control over ${\displaystyle {\hat{v}}_{||}}$ over many steps strongly suggests that locomotion unfolds in blocks. The HHMM presented here or the states revealed by it may not reflect the actual states employed by neurons in the brain. In fact, there is an ongoing debate whether behavior and its control is better represented as a continuum than by discrete states. The presence of long-lasting states that are employed repeatedly by all flies in our dataset implies that either locomotion does consists of transition between discrete states, or that these states represent fixed points or peaks of a dynamical system around which the animal spends most of its time (Berman, 2018).

Another surprising result is that the same set of locomotor features describes the behavior of all the flies in the dataset. This result is particularly surprising given that our model explicitly allows each fly its own set of locomotor features. The fact that all flies can be reasonably modeled by the same model implies that within a given environment all flies construct their locomotion from the same building blocks, and differences in locomotion amongst flies or the effect of sensory stimulation can be quantified as changes in the frequency with which these building blocks are employed. An important avenue for future research is to assess whether these locomotor features are fixed or flexible.

In nature, animals encounter odors in a cluttered and dynamic sensory environment (Riffell et al., 2014). Discriminating between odors, navigating towards the chosen odor source and pinpointing the odor source requires a flexible deployment of multiple different motor programs. It is difficult to replicate the complex natural environment in the laboratory. Therefore, laboratory studies are typically aimed at different subsets of the complex environment experienced by animals. In insects, much research has focused on an environment in which it experiences odors in a highly structured odor plume often within a high-contrast visual environment (Budick and Dickinson, 2006; Kennedy, 1983; Vickers, 2000; van Breugel and Dickinson, 2014; Álvarez-Salvado et al., 2018). Recently, similar experiments have been repeated for flies walking towards an odor source (Bell and Wilson, 2016). These experiments model an insect’s behavior under one specific condition wherein the fly tries to locate an odor source at a distance using strong directional information from wind and vision. The experiments described here explore a fly’s behavior in a small, dark circular arena. At most locations in the arena, the air speed was 0.07 m/s; the highest wind-speed was 0.11 m/s (Jung et al., 2015), a value lower than has been employed in most studies. Therefore, non-olfactory directional cues from vision or wind are minimized (Jung et al., 2015). Consistent with this idea, there was no change in the distribution of the flies when wind was completely eliminated.

We find that this behavior near the odor source can be described by changes in the HL states. The clearest evidence that changes in HL states are a good description of the fly’s behavior is the analysis in which we measured the spatial distribution of odor-evoked changes in HL states (Figure 6), and observed a pattern that strongly resembles the odor-zone. This analysis shows that the HL state description is accurate enough to facilitate discovery of arena structure. However, because we averaged HL state distribution over the entire 3 min of odor exposure, the analysis misses some details. The flies first detect odor slightly outside the odor-zone as defined in this study (Jung et al., 2015), and their behavior during the first 10 s after odor encounter differs from their behavior during the rest of the odor period (Figure 6—figure supplement 1A). Moreover, at least some of the spatial structure results from the change in the radial density of the fly as a function of time (Figure 6—figure supplement 1B). A fly’s interaction with odor is dynamic and that HL states are a good analytical framework to extract the spatiotemporal patterning of fly’s behavior by odor. This spatiotemporal pattern likely differs among flies; a full description of this pattern requires a larger dataset and represents an important avenue for future research.

Even single-cell organisms and animals with simple nervous systems display substantial individuality (Jordan et al., 2013; Gallagher et al., 2013). Animals with larger nervous systems are likely to display even greater individuality, in the case of adult flies this individuality was demonstrated in the context of locomotor handedness (Buchanan et al., 2015) in a choice assay. The nature and extent of individuality is harder to assess in more complex behaviors because of the difficulty in assessing individuality in a large behavioral space; differences in behavior can simply be different instantiation of the same behavior, or reflect fundamental differences in behavior. In this study, we find that different flies employ the same locomotor features but use them in vastly different proportions. The observed variability between the flies is inconsistent with a single type of locomotor behavior but can be approximated by invoking 3–4 clusters of flies. Because the clustering framework we employed (X-means) underestimates the number of clusters, and because there were only 34 flies in our dataset, it remains to be seen whether there are a few locomotor-types or a whole continuum of locomotor-types. Another limitation of this study is that we have not yet ascertained whether a fly’s behavior persists over a longer time frame. Despite these limitations, this study makes two important contribution to the study of individuality. First, we develop a statistical framework to study complex behaviors. This method can be extended to examine whether the differences we observe truely represent individuality. Second, in many behavioral studies, researchers focus on the effect of some stimuli on behavior and make conclusions based on the average fly. In this study, we provide a framework for testing whether the description based on an average fly is appropriate, and ways to proceed if such an approach is inadequate.

The diversity of odor responses observed here is consistent with work done on moths where (Willis and Arbas, 1998) but in sharp contrast to recent work on walking Drosophila (Bell and Wilson, 2016) in which the authors reported that attraction to odors results from a stereotypical motor pattern. The authors of that study claimed that the relatively simple response to odors in their study is likely a result of their simple behavioral arena. Another possibility is that in their study there is a strong, directional wind cue. In the presence of a steady wind cue in a narrow arena, it is likely that the flies’ locomotor behavior is dominated by upwind walking and suppresses other elements of their behavior. It is well-established that a fly’s response to odor is strongly influenced by context, as was demonstrated recently by comparing the response to odors in different visual and air flow conditions (Saxena et al., 2018).

When considered from the viewpoint of an individual animal, this variability is hard to understand: A hungry fly in search of food should respond with a singular, hardwired behavior which represents an optimal strategy for locating food. However, species evolve as large populations of individuals, and a successful species should be able to adapt to fluctuating environmental condition. Recent work has shown that - bet hedging - a process by which the same genotype shows considerable phenotypic variation is important for adaptation to fluctuating environments (Kain et al., 2015). Having a diversity of phenotypes ensures that some individuals would thrive in any condition and behavioral variability is a feature not a bug and its careful consideration is critical.

Studying individual behavioral responses is also important to understand the mechanism underlying both the control of locomotion and how odors control locomotion. Analyzing behavior at the population level can provide important insights into an animal’s response but does not provide the resolution necessary for understanding the neural mechanism underlying the moment-by-moment control of behavior at the level of individual flies. In this context, it is instructive to take a closer look at the average response to odors in the light of clusters of response to odors. The average response (Figure 5A) was surprising: the occupancy of all the slow states except state 2 is increased. The lack of increase in occupancy of state 2 results from a cluster of flies in which the occupancy of state 2 is strongly decreased (Figure 9A). A similar effect is observed in the response of the average fly to the medium-speed states – states 4–7. The occupancy of these states decreases in some flies and increases in others. Thus, the average fly is an aggregate of these different clusters of flies, each of which has a distinct response to odor. Disaggregation is an essential first step to understanding neural control of behavior.

The methods used to collect the behavioral data were reported in a previous study (Jung et al., 2015). Briefly, flies were raised in a sparse culture. Flies that were 3–5 days post-eclosion were starved for 14–18 hr. Locomotion of a single fly was recorded for a 3 min period before odor was introduced (before period) and a 3-min period during odor (during period), using a video camera at a rate of 30 frames per second. The coordinates of the fly were extracted using a custom MATLAB program (https://github.com/bhandawat/fly-walking-behavior/tree/master/tracking; Bhandawat, 2017; copy archived at https://github.com/elifesciences-publications/fly-walking-behavior).

The behavioral arena was normalized to a unit circle centered at the origin. The raw coordinates of the centroid of the fly were smoothed using wavelet denoising followed by a locally weighted (lowess) filter. Speed and curvature were defined exactly as in the previous study.

To quantify the behavior of the 34 flies, we computed the speed of the fly along the original direction of movement (${\displaystyle {\hat{v}}_{||}}$) and the speed of the fly perpendicular to the original direction of movement (${\displaystyle {\hat{v}}_{\perp }}$). ${\displaystyle {\hat{v}}_{||}}$ at time t was defined as the component of the velocity at time ${\displaystyle t}$ in the direction of velocity of the fly at time ${\displaystyle t-1}$. ${\displaystyle {\hat{v}}_{\perp }}$ was defined as the component of the velocity at time ${\displaystyle t}$ perpendicular to velocity of the fly at time ${\displaystyle t-1}$ (Figure 1C). These values were calculated as follows:

Values of ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ found to be further than 4 standard deviations away from the average were set to values drawn from a normally distributed distribution (${\displaystyle \sigma =1}$) centered at the 4 standard deviation mark. The resulting ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ were then set as the observables used in fitting a 2-level Hierarchical Hidden Markov model (HHMM).

We employed ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ instead of speed and curvature because curvature is very noisy at low speeds because the calculation of curvature requires division by the third-power of speed. ${\displaystyle {\hat{v}}_{||}}$ and ${\displaystyle {\hat{v}}_{\perp }}$ are directly related to speed and curvature as follows

HMMs are widely used in a variety of fields for modeling time series data. An HMM is a Markov model which assumes that a given sequence of observations may be explained by a set of states that are not observed (or hidden states), and the time independent probability of transitioning between these states. The model processes which produce the observations in an HMM are hidden to the researcher and thus, the goal of fitting an HMM is to uncover the highest likelihood probability model parameters that can generate the data. Baum and others developed the core theory of HMMs (Baum and Petrie, 1966). Since then there has been much exploration of model architecture, and fitting procedure.

HMMs have been shown to be effective in modeling behavior because instantaneous measures of an observable are variable; therefore, behavioral states inferred by the application of simple thresholding to instantaneous measures of the observables are likely to be erroneous. HMM remedies this problem by inferring states based not only on the value of the observable at the current time point but also on the previous and following time points, and allows a more accurate determination of state (this idea is well-explained in Figure 2 in ref 17). Specifically, the assumption of Markov dynamics with a sparse prior on state transitions penalizes the consideration of unlikely state transitions based upon recent history (forward filtering) and future destinations (backward smoothing).

The HHMM is an extension to the HMM which applies hierarchical structure in the form that higher level state is itself an HMM composed of its lower level states (Fine et al., 1998). The approach we take in exploring and fitting the HHMM closely follows the approach developed by Matt Beal in which he applied variational algorithms to fit HHMM to a time series of observables (Beal, 2003).

This section is divided into three parts. First is the description of the model, second is the details of the process by which the model is fit, and third is the thought process behind our model selection.

Time points were sampled using four methods based on bin duration for each fly for the two scenarios with the most data (B_o and D_i) separately (Figure 7—figure supplement 4). In method one, for a given scenario, segments of continuous repeated HL states were randomly sampled and stitched together until the duration of the time bin was fulfilled. In the second method, a window lasting the time bin was sampled from the HL states for the scenario. In method three, the segments were sampled starting with the first time point of the scenarios. In method four, the segments were sampled starting with the last time point of the scenarios and moving backwards in time. After sampling, the average HL state distributions were calculated for each time bin and the Euclidean distance from the distribution to the centroid of each X-means cluster for the given scenario were calculated. The closest cluster was compared to the X-means cluster assignment based on all time points. This process was repeated 100 times to generate a mean percentage of correctly labeled flies based on the subsampling duration. Chance was calculated as the probability of choosing a fly for a given cluster and being in the Voronoi cell of the cluster. This translates to:

where ${\displaystyle x_i}$ is the probability of observing cluster i based on the number of flies in each cluster, ${\displaystyle p_i}$ is a weighting based on the size of Voronoi cell in the simplex space, and K is the total number of clusters.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

We acknowledge the members of Bhandawat lab and Gaby Maimon for critical comments on earlier versions of the manuscript. This research was supported by NIDCD (VB), NINDS (VB) and an NSF CAREER award (VB).

© 2019, Tao et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.