Accept-reject decision-making revealed via a quantitative and ethological study of C. elegans foraging

Reviewer #1 (Public review):

Summary:

This work uses a novel, ethologically relevant behavioral task to explore decision-making paradigms in C. elegans foraging behavior. By rigorously quantifying multiple features of animal behavior as they navigate in a patch food environment, the authors provide strong evidence that worms exhibit one of three qualitatively distinct behavioral responses upon encountering a patch:
(1) "search", in which the encountered patch is below the detection threshold;
(2) "sample", in which animals detect a patch encounter and reduce their motor speed, but do not stay to exploit the resource and are therefore considered to have "rejected" it; and
(3) "exploit", in which animals "accept" the patch and exploit the resource for tens of minutes.
Interestingly, the probability of these outcomes varies with the density of the patch as well as the prior experience of the animal. Together, these experiments provide an interesting new framework for understanding the ability of the C. elegans nervous system to use sensory information and internal state to implement behavioral state decisions.

Strengths:

(1) The work uses a novel, neuroethologically-inspired approach to studying foraging behavior.

(2) The studies are carried out with an exceptional level of quantitative rigor and attention to detail.

(3) Powerful quantitative modeling approaches including GLMs are used to study the behavioral states that worms enter upon encountering food, and the parameters that govern the decision about which state to enter.

(4) The work provides strong evidence that C. elegans can make 'accept-reject' decisions upon encountering a food resource.

(5) Accept-reject decisions depend on the quality of the food resource encountered as well as on internally represented features that provide measurements of multiple dimensions of internal state, including feeding status and time.

Weaknesses:

(1) The authors repeatedly assert that an individual's behavior in the foraging assay depends on its prior history (particularly cultivation conditions). While this seems like a reasonable expectation, it is not fully fleshed out. The work would benefit from studies in which animals are raised on more or less abundant food before the behavioral task.

(2) The authors convincingly show that the probability of particular behavioral outcomes occurring upon patch encounter depends on time-associated parameters (time since last patch encounter, time since last patch exploitation). There are two concerns here. First, it is not clear how these values are initialized - i.e., what values are used for the first occurrence of each behavioral state? More importantly, the authors don't seem to consider the simplest time parameter, the time since the start of the assay (or time since worm transfer). Transferring animals to a new environment can be associated with significant mechanical stimulus, and it seems quite possible that transferring animals causes them to enter a state of arousal. This arousal, which certainly could alter sensory function or decision-making, would likely decay with time. It would be interesting to know how well the model performs using time since assay starts as the only time-dependent parameter.

(3) Similarly, Figures 2L and M clearly show that the probability of a search event occurring upon a patch encounter decreases markedly with time. Because search events are interpreted as a failure to detect a patch, this implies that the detection of (dilute) patches becomes more efficient with time. It would be useful for the authors to consider this possibility as well as potential explanations, which might be related to the point above.

(4) Based on their results with mec-4 and osm-6 mutants, the authors assert that chemosensation, rather than mechanosensation, likely accounts for animals' ability to measure patch density. This argument is not well-supported: mec-4 is required only for the function of the six non-ciliated light-touch neurons (AVM, PVM, ALML/R, PLML/R). In contrast, osm-6 is expected to disrupt the function of the ciliated dopaminergic mechanosensory neurons CEP, ADE, and PDE, which have previously been shown to detect the presence of bacteria (Sawin et al 2000). Thus, the paper's results are entirely consistent with an important role of mechanosensation in detecting bacterial abundance. Along these lines, it would be useful for the authors to speculate on why osm-6 mutants are more, rather than less, likely to "accept" when encountering a patch.

(5) While the evidence for the accept-reject framework is strong, it would be useful for the authors to provide a bit more discussion about the null hypothesis and associated expectations. In other words, what would worm behavior in this assay look like if animals were not able to make accept-reject decisions, relying only on exploit-explore decisions that depend on modulation of food-leaving probability?

Reviewer #2 (Public review):

This study provides an experimental and computational framework to behavioral biology that helps examine and understand how C. elegans make decisions while foraging in environments with patches of food. The authors show that worms actively reject or accept food patches depending on a number of internal and external factors.

The key novelty and strength of this paper is the explicit demonstration of behavior analysis and quantitative modeling to elucidate the decision-making process. In particular, the description of the exploring vs. exploiting phases, and sensing vs. non-sensing categories of C. elegans foraging behavior based on the clustering of behavioral states defined in a multi-dimensional behavior-metrics space, and the implementation of a generalized linear model (GLM) whose parameters can provide quantitative biological interpretations.

While the concept is interesting, there are many flaws in the experimental, analysis, and models that weaken what one can conclude from the work.

Reviewer #3 (Public review):

Summary:

In this study by Haley et al, the authors investigated explore-exploit foraging using C. elegans as a model system. Through an elegant set of patchy environment assays, the authors built a GLM based on past experience that predicts whether an animal will decide to stay on a patch to feed and exploit that resource, instead of choosing to leave and explore other patches.

Strengths:

I really enjoyed reading this paper. The experiments are simple and elegant, and address fundamental questions of foraging theory in a well-defined system. The experimental design is thoroughly vetted, and the authors provide a considerable volume of data to prove their points. My only criticisms have to do with the data interpretation, which I think is easily addressable.

Weaknesses:

(1) Sensing vs. non-sensing

The authors claim that when animals encounter dilute food patches, they do not sense them, as evidenced by the shallow deceleration that occurs when animals encounter these patches. This seems ethologically inaccurate. There is a critical difference between not sensing a stimulus, and not reacting to it. Animals sense numerous stimuli from their environment, but often only behaviorally respond to a fraction of them, depending on their attention and arousal state. With regard to C. elegans, it is well-established that their amphid chemosensory neurons are capable of detecting very dilute concentrations of odors. In addition, the authors provide evidence that osm-6 animals have altered exploit behaviors, further supporting the importance of amphid chemosensory neurons in this behavior.

(2) Search vs. sample & sensing vs. non-sensing

In Figures 2H and 2I, the authors claim that there are three behavioral states based on quantifying average velocity, encounter duration, and acceleration, but I only see three. Based on density distributions alone, there really only seem to be 2 distributions, not 3. The authors claim there are three, but to come to this conclusion, they used a QDA, which inherently is based on the authors training the model to detect three states based on prior annotations. Did the authors perform a model test, such as the Bayesian Information Criterion, to confirm whether 2 vs. 3 Gaussians is statistically significant? It seems like the authors are trying to impose two states on a phenomenon with a broad distribution. This seems very similar to the results observed for roaming vs. dwelling experiments, which again, are essentially two behavioral states.

(4) History-dependence of the GLM

The logistic GLM seems like a logical way to model a binary choice, and I think the parameters you chose are certainly important. However, the framing of them seems odd to me. I do not doubt the animals are assessing the current state of the patch with an assessment of past experience; that makes perfect logical sense. However, it seems odd to reduce past experience to the categories of recently exploited patch, recently encountered patch, and time since last exploitation. This implies the animals have some way of discriminating these past patch experiences and committing them to memory. Also, it seems logical that the time on these patches, not just their density, should also matter, just as the time without food matters. Time is inherent to memory. This model also imposes a prior categorization in trying to distinguish between sensed vs. not-sensed patches, which I criticized earlier. Only "sensed" patches are used in the model, but it is questionable whether worms genuinely do not "sense" these patches.

(5) osm-6

The osm-6 results are interesting. This seems to indicate that the worms are still sensing the food, but are unable to assess quality, therefore the default response is to exploit. How do you think the worms are sensing the food? Clearly, they sense it, but without the amphid sensory neurons, and not mechanosensation. Perhaps feeding is important? Could you speculate on this?

(7) Impact:

I think this work will have a solid impact on the field, as it provides tangible variables to test how animals assess their environment and decide to exploit resources. I think the strength of this research could be strengthened by a reassessment of their model that would both simplify it and provide testable timescales of satiety/starvation memory.

Author response:

We thank the reviewers for their thoughtful comments. We are working to revise our manuscript and address each of the reviewers comments. A summary of our planned revisions and responses to some of the reviewers’ major concerns are included below.

Cultivation Density: Reviewers #1 and #2 suggested that additional studies testing the effects of varying bacterial density during animal development (cultivation) would strengthen our findings. While we agree with the reviewers that this is a very interesting experiment, it is not feasible. Indeed, we attempted this experiment but found it nontrivial to maintain stable bacterial density conditions over long timescales as this requires matching the rate of bacterial growth with the rate of bacterial consumption. Despite our best efforts, we have not been able to identify conditions that satisfy these requirements. We will focus our revised manuscript to include only assertions about the effects of recent experiences.

Transfer Method: Reviewers #1 and #2 expressed concern that the stress of transferring animals to a new plate may have resulted in an increased arousal state and thus a greater probability of rejecting patches. We thank the reviewers for this thoughtful remark and plan to conduct additional analyses to address this hypothesis. We did, however, anticipate this possibility and, to mitigate the stress of moving, we used an agar plug method where animals were transferred using the flat surface of small cylinders of agar. Importantly, the use of agar as a medium to transfer animals provides minimal disruption to their environment as all physical properties (e.g. temperature, humidity, surface tension) are maintained. Qualitatively, we observe no marked change in behavior from before to after transfer with the agar plug method, especially as compared to the often drastic changes observed when using a metal or eyelash pick.

Time Parameter: Related to the transfer method, Reviewer #1 expressed concern that the simplest time parameter (time since start of the assay) might better predict animal behavior. We thank the reviewer for pointing out the need to specifically test whether the time-dependent change in explore-exploit decision-making corresponds better with satiety (time off patch) or arousal (time since transfer/start of assay) state. We will conduct additional analyses to address these alternative hypotheses.

Parameter Initialization: Reviewer #1 pointed out an oversight in our methods section regarding the model parameter values used for the first encounter. We plan to clarify the initialization of parameters in the manuscript. In short, for the first patch encounter where k = 1:

● ρk is the relative density of the first patch.

● τs is the duration of time spent off food since the beginning of the recorded experiment. For the first patch, this is equivalent to the total time elapsed.

● ρh is the approximated relative density of the bacterial patch on the acclimation plates (see Assay preparation and recording in Methods). Acclimation plates contained one large 200 µL patch seeded with OD600 = 1 and grown for a total of ~48 hours. As with all patches, the relative density was estimated from experiments using fluorescent bacteria OP50-GFP as described in Bacterial patch density estimation in Methods.

● ρe is equivalent to ρh.

Sensing vs. non-sensing: Reviewer #3 suggested that the term “non-sensing” may not be ethologically accurate. We thank the reviewer for their comment and agree that we do not know for certain whether the animals sensed these patches or were merely non-responsive to them. We are, however, confident that these encounters lack evidence of sensing. Specifically, we note that our analyses used to classify events as sensing or non-sensing examined whether an animal’s slow-down upon patch entry could be distinguished from either that of events where animals exploited or that of encounters with patches lacking bacteria. We found that “non-sensing” encounters are indeed indistinguishable from encounters with bacteria-free patches where there are no bacteria to be sensed (see Figure 2 - Supplement 7C-D and Patch encounter classification as sensing or non-sensing in Methods). Regardless, we agree with the reviewer that all that can be asserted for certain about these events is that animals do not respond to the bacterial patch in any way that we measured. Therefore, we will replace the term “non-sensing” with “non-responding” to better indicate the ethological interpretation of these events.

Time-dependent changes in sensing vs. non-sensing: Reviewer #1 remarked that the sensation of dilute patches increases with time. We agree with the reviewer that we observe increased responsiveness to dilute patches with time. Although this is interesting, our primary focus was on what decision an animal made given that they clearly sensed the presence of the bacterial patch. Nonetheless, we will add this observation to the discussion as an area of future work to investigate the sensory mechanisms behind this effect.

Classification of sensing vs. non-sensing: Reviewers #2 and #3 expressed concerns about the validity of the two clusters identified using the semi-supervised QDA approach described. We are grateful to the reviewers for pointing out the difficulty in visualizing the clusters and the need for additional clarity in explaining the supervised labeling. We will use additional visualizations and methods to validate the clusters we have discovered. Specifically, we aim to provide additional evidence that the sensing vs. nonsensing data is bi-modal (i.e. a two-cluster classification method fits best). Further, it seems that there may be some confusion as to how we arrived at 3 encounter types (i.e. search, sample, exploit) that we plan to clarify in the manuscript. Specifically, it’s important to note that two methods were used on two different (albeit related) sets of parameters. We first used a two-cluster GMM to classify encounters as explore or exploit. We then used a two-cluster semi-supervised QDA to classify encounters as sensing or non-sensing (to be changed to “non-responding”, see above response) using a different set of parameters. We thus separated the explore cluster into two (sensing and non-sensing exploratory events) resulting in three total encounter types: exploit, sample (explore/sensing), and search (explore/non-sensing). We will clarify this in the text. Additionally, we will clarify the labelling used for “supervising” QDA. Specifically, we made two simple assumptions: 1) animals must have sensed the patch if they exploited it and 2) animals must not have sensed the patch if there were no bacteria to sense. Thus, we labeled encounters as sensing if they were found to be exploitatory as we assume that sensation is prerequisite to exploitation; and we labeled encounters as non-sensing for events where animals encountered patches lacking bacteria (OD600 = 0). All other points were non-labeled prior to learning the model. In this way, our labels were based on the experimental design and results of the GMM, an unsupervised method; rather than any expectations we had about what sensing should look like. The semi-supervised QDA method then used these initial labels to iteratively fit a paraboloid that best separated these clusters, by minimizing the posterior variance of classification.

Accept-reject vs. stay-switch: Reviewers #1 and #2 ask for additional discussion on how the accept-reject decision-making framework differs from the stay-switch framework. We thank the reviewers for alerting us to this gap in our discussion. We intend to clarify that these frameworks ask two different types of questions (i.e. “Do you want to eat it?” versus “If so, how long do you want to eat it for?”). These concepts are well described in canonical foraging theory literature (see Pyke, Pulliam & Charnov 1977 for a review on the subject) and are easily distinguishable for animals that forage using the following framework: 1) search for prey, 2) encounter prey from a distance, 3) identify prey type, 4) decide to pursue (accept-reject decision), 5) pursue and capture the prey, 6) exploit prey, and 7) decide to stop exploiting and start searching again (stay-switch decision). In this case, it is easy to see the distinction between accept-reject and stay-switch decisions. However, in some scenarios, animals must physically encounter prey prior to identification and then must make an accept-reject decision. In these cases where pursuit and capture are not visualized, it is harder to distinguish between accept-reject and stay-switch decisions. In our experiments, we find significant bimodality in encounter duration (see Figure 2H) where short duration (exploratory) encounters appear to represent a lower bound where animals spend the minimum amount of time possible on a patch (less than 2 minutes), which we interpret as a rejection of the patch. On the other hand, exploitatory encounters span a large range of durations from 2 to 60+ minutes which we interpret as an initial acceptance of the patch followed by a series of stay-switch decisions which determine the overall duration of the encounter. While one could certainly model our data using only stay-switch decision-making, we ascertain that an encounter of minimal duration is better interpreted ethologically as a rejection than as an immediate switch decision. We will revise the text to further extrapolate upon our point of view on this somewhat philosophical distinction and what it predicts about C. elegans behavior.

Sensory mutant behavior: Reviewers #1 and #3 ask for further speculation on the observed behavior of osm-6 and mec-4 animals. We will further elaborate on our findings, how they relate to previous studies, and what they suggest about the mechanisms behind these foraging decisions.

Model design: Reviewer #3 suggested several alterations to the behavioral model. While the proposed model seems entirely reasonable and could aid in elucidating the time component of how prior experience affects decision-making, we chose the present model based on our experience with model selection using these data. Indeed, as the reviewer suggested, we did a great number of analyses involving model selection including model selection criteria (AIC, BIC) and optimization with regularization techniques (LASSO and elastic nets). We found that the problem of model selection was compounded by the enormous array of highly correlated variables we had to choose from. Additionally, we found that both interaction terms and non-linear terms of our task variables could be predictive of accept-reject decisions but that the precise set of terms selected depended sensitively on which model selection technique was used and generally made rather small contributions to prediction. The diverse array of results and combinatorial number of predictors to possibly include failed to add anything of interpretable value. We therefore chose to take a different approach to this problem. Rather than trying to determine what the “best” model was we instead asked whether a minimal model could be used to answer a set of core questions. Indeed, our goal was not maximal predictive performance but rather to distinguish between the effects of different influences enough to determine if encounter history had a significant, independent effect on decision making. We thus chose to only include task variables that spanned the most basic components of behavioral mechanisms to ask very specific questions. For example, we selected a time variable that we thought best encapsulated satiety. While we could have included many additional terms, or made different choices about which terms to include, based on our analyses these choices would not have qualitatively changed our results. Further, we sought to validate the parameters we chose with additional studies (i.e. food-deprived and sensory mutant animals). We regard our study as an initial foray into demonstrating accept-reject decision-making in nematodes. The exact mechanisms and, consequently, the best model design is therefore beyond the scope of this study. Lastly, Reviewer #3 criticized the use of only sensed patches in the model. While we acknowledge that we are not certain as to whether the “non-sensing” encounters are truly not sensed, we find qualitatively similar results when including all exploratory patches in our analyses. In fact, when all encounters are used, we find stronger correlations between our task variables and the accept-reject decision. However, we take the position that sensation is necessary for decision-making and thus believe that while our model’s predictive performance may be better using all encounters, the interpretation of our findings is stronger when we only include sensing events.