1. Neuroscience
  2. Physics of Living Systems
Download icon

Variability in locomotor dynamics reveals the critical role of feedback in task control

  1. Ismail Uyanik  Is a corresponding author
  2. Shahin Sefati
  3. Sarah A Stamper
  4. Kyoung-A Cho
  5. M Mert Ankarali
  6. Eric S Fortune
  7. Noah J Cowan  Is a corresponding author
  1. Department of Electrical and Electronics Engineering, Hacettepe University, Turkey
  2. Laboratory of Computational Sensing and Robotics, Johns Hopkins University, United States
  3. Department of Biological Sciences, New Jersey Institute of Technology, United States
  4. Department of Mechanical Engineering, Johns Hopkins University, United States
  5. Department of Electrical and Electronics Engineering, Middle East Technical University, Turkey
Research Article
  • Cited 1
  • Views 1,550
  • Annotations
Cite this article as: eLife 2020;9:e51219 doi: 10.7554/eLife.51219

Abstract

Animals vary considerably in size, shape, and physiological features across individuals, but yet achieve remarkably similar behavioral performances. We examined how animals compensate for morphophysiological variation by measuring the system dynamics of individual knifefish (Eigenmannia virescens) in a refuge tracking task. Kinematic measurements of Eigenmannia were used to generate individualized estimates of each fish’s locomotor plant and controller, revealing substantial variability between fish. To test the impact of this variability on behavioral performance, these models were used to perform simulated ‘brain transplants’—computationally swapping controllers and plants between individuals. We found that simulated closed-loop performance was robust to mismatch between plant and controller. This suggests that animals rely on feedback rather than precisely tuned neural controllers to compensate for morphophysiological variability.

eLife digest

People come in different shapes and sizes, but most will perform similarly well if asked to complete a task requiring fine manual dexterity – such as holding a pen or picking up a single grape. How can different individuals, with different sized hands and muscles, produce such similar movements? One explanation is that an individual’s brain and nervous system become precisely tuned to mechanics of the body’s muscles and skeleton. An alternative explanation is that brain and nervous system use a more “robust” control policy that can compensate for differences in the body by relying on feedback from the senses to guide the movements.

To distinguish between these two explanations, Uyanik et al. turned to weakly electric freshwater fish known as glass knifefish. These fish seek refuge within root systems, reed grass and among other objects in the water. They swim backwards and forwards to stay hidden despite constantly changing currents. Each fish shuttles back and forth by moving a long ribbon-like fin on the underside of its body. Uyanik et al. measured the movements of the ribbon fin under controlled conditions in the laboratory, and then used the data to create computer models of the brain and body of each fish. The models of each fish’s brain and body were quite different.

To study how the brain interacts with the body, Uyanik et al. then conducted experiments reminiscent of those described in the story of Frankenstein and transplanted the brain from each computer model into the body of different model fish. These “brain swaps” had almost no effect on the model’s simulated swimming behavior. Instead, these “Frankenfish” used sensory feedback to compensate for any mismatch between their brain and body.

This suggests that, for some behaviors, an animal’s brain does not need to be precisely tuned to the specific characteristics of its body. Instead, robust control of movement relies on many seemingly redundant systems that provide sensory feedback. This has implications for the field of robotics. It further suggests that when designing robots, engineers should prioritize enabling the robots to use sensory feedback to cope with unexpected events, a well-known idea in control engineering.

Introduction

Animals routinely exhibit dramatic variations in morphophysiology between individuals but can nevertheless achieve similar performance in sensorimotor tasks (Sponberg et al., 2015; Bullimore and Burn, 2006). Further, individual animals can experience rapid changes in their own morphophysiological features, such as extreme weight changes that occur during and between bouts of feeding. For example, mosquitoes can consume more than their body weight (Van Handel, 1965) and hummingbirds can consume up to 20% of their body weight (Hou et al., 2015) in a single feeding. How neural control systems accommodate these changes is not known.

The behavioral performance of an individual animal is determined via an interplay between its ‘controller’ and ‘plant’ (Kiemel et al., 2011; van der Kooij and Peterka, 2011; Cowan et al., 2014; Dickinson et al., 2000; Hedrick et al., 2009). The plant typically includes musculoskeletal components that interact with the environment to generate movement (Hedrick and Robinson, 2010; Sefati et al., 2013; Maladen et al., 2009). The controller typically includes sensory systems and neural circuits used to process information to generate motor commands (Cowan and Fortune, 2007; Kiemel et al., 2011; Lockhart and Ting, 2007; Roth et al., 2014). From the perspective of control theory, one might expect the dynamics of the controller to be precisely tuned to the dynamics of the plant, resulting in an optimal control law that reduces variability in task performance (Todorov, 2004; Franklin and Wolpert, 2011; Bays and Wolpert, 2007). Were this the case, variations across individuals in morphophysiological features of their plants should manifest in commensurate differences in each animal’s controller. Alternatively, the central nervous system (CNS) may be implementing robust feedback control that attenuates morphophysiological variability at the behavioral level without the need for precise tuning.

Investigating these relationships requires separate estimates for plants and controllers. However, the classical input–output system identification of behavioral tasks—using only the sensory input and the behavioral output—is limited to generating closed-loop control models of behavioral responses. Data-driven system identification of the underlying neural controllers or locomotor plants requires additional observations such as a measurement of the control output. Electromyograms (EMGs) are the most commonly used proxy for the output of the neural controller. EMGs allow separate data-driven estimates of the controller and plant (Kiemel et al., 2011; van der Kooij and Peterka, 2011) but require understanding the coordination strategy across multiple groups of muscles that interact in fundamentally nonlinear ways (Ting and Macpherson, 2005).

We studied refuge tracking in a species of weakly electric fish Eigenmannia virescens (Figure 1A), a system that permits identification of input–output dynamics as well as the locomotor plant via behavioral observations alone. Like an ‘aquatic hummingbird’, Eigenmannia precisely hover in place, making rapid and nuanced adjustments to its position in response to the movement of the refuge in which it is hiding (Rose and Canfield, 1993; Roth et al., 2011; Uyanik et al., 2019b; Figure 1—video 1). During refuge tracking, individual Eigenmannia generate fore-aft thrust forces using undulatory motions of their ventral ribbon fin. Undulations are initiated at the rostral and caudal ends of the fin resulting in counter propagating waves that travel towards each other (Sefati et al., 2013; Ruiz-Torres et al., 2013). The two traveling waves meet at a position along the ribbon fin known as the nodal point (Figure 1—video 2). In a task in which the fish maintains position in a stationary refuge, Eigenmannia shift the rostrocaudal position of the nodal point as a function of steady-state swimming speed (Sefati et al., 2013; Figure 1—figure supplement 1), providing a behavioral proxy for the controller’s output, without reliance on EMGs.

Figure 1 with 4 supplements see all
Experimental analysis of refuge tracking behavior.

(A) Eigenmannia virescens maintains its position, y(t), in a moving refuge. r(t) is the position of the moving refuge. (B) A feedback control system model for refuge tracking. The feedback controller C(s) maps the error signal, e(t), to a control signal, u(t). The plant P(s), in turn, generates fore-aft movements y(t). We used measurements of the nodal point as u(t) to estimate P(s) in each fish, which was subsequently used to infer C(s). (C) Eigenmannia virescens partitions its ribbon fin into two counter-propagating waves that generate opposing forces (Sefati et al., 2013; Ruiz-Torres et al., 2013). The rostral-to-caudal and caudal-to-rostral waves collide at the nodal point (green line). Movements of the nodal point serve as a proxy for the control signal, u(t). Gray lines indicate the positions of the peaks and troughs of the traveling waves over time. In this trial, the refuge was following a sinusoidal trajectory at 2.05 Hz. (D) Movement of the nodal point u(t), position of the fish y(t), and position of the refuge r(t) of 13 trials at 0.95 Hz. Semi-transparent lines represent data from individual trials, green and blue lines are the means.

We measured tracking performance and moment-to-moment position of the nodal point in three fish during a refuge tracking task. Despite the fact that tracking performance of the three fish were similar, there were nevertheless large variations in the movement of the nodal point, reflecting morphophysiological differences between individuals. We used computational techniques, specifically data-driven system identification of feedback control models, to explore how neural control systems cope with individual variability in locomotor dynamics.

Results

We measured the performance of three fish in a refuge tracking task by comparing the position of the refuge r(t) with the position of the fish y(t) (Figure 1A). These measurements are used in a feedback control model in which ‘error signal’ e(t) is defined as the difference between r(t) and y(t) (Figure 1B). To separately estimate the controller C(s) and the plant P(s), a measurement of the control signal u(t) is necessary. In Eigenmannia, the position of the nodal point can be measured over time (Figure 1C, Figure 1—video 2). During refuge tracking behavior, we observed that the position of the nodal point appears to have a linear relationship to y(t) making it a candidate readout of u(t) (Figure 1D). Using u(t) and y(t), we estimated P(s) for each fish and calculated their corresponding controllers C(s). We used C(s) and P(s) of each fish to computationally manipulate the interplay between the controller, plant and the sensory feedback.

Estimating a data-driven plant model

We estimated a data-driven model for the plant dynamics P(s) of each fish. For the purpose of visualizing the plant models graphically, it is useful to treat the plant, P(s), as a filter through which motor commands are processed. At a given frequency, ω, the filter’s behavior can be represented as a complex number, called a phasor, that is obtained by evaluating the transfer function P(s) at that frequency, namely P(jω), where j=-1.

The locus of phasors as a function of frequency is called the frequency response function. We estimated the frequency responses of the locomotor dynamics of each fish using the position of the fish y(t) and its nodal point u(t). To visualize the variability across plants, we used a Nyquist plot. On such a plot, the gain and phase of the response of the plant at a given frequency are represented in polar form, where the x axis is the real part and the y axis is the imaginary part—both dimensionless. The gain is the distance from the origin 0+j0, and the phase is the angle measured counterclockwise from the real axis. Nyquist plots of each individual's estimated plant revealed substantial differences between locomotor plants of individual fish (Figure 2A). Despite these differences, the frequency responses shared a common structure: the locomotor dynamics of each fish acted as a low-pass filter on the movements of the nodal point. This common structure facilitated the application of parametric modeling, reducing the complexity of analysis while enabling computational manipulations of the model system. We used the physics-based parametric model of locomotor dynamics of Eigenmannia described by Sefati et al. (2013) for the plant (see Materials and methods for derivation):

(1) P(s)=kms2+bs

Here, m, k, and b represent mass, gain, and damping, respectively, and s is complex frequency in Laplace domain (see, e.g. Roth et al., 2014). We estimated the parameters in the parametric plant model for each fish based on their individualized frequency responses via numerical optimization (see Materials and methods) (Figure 2B). Frequency responses for the estimated parametric models are illustrated in Figure 2C (see black lines in Figure 2A for corresponding Nyquist plots). Finally, we estimated the plant for a ‘merged’ fish, in which the data from the three fish were concatenated as a single fish. The differences in the frequency responses between individuals resulted in substantial differences (about twofold) in estimated model parameters (Table 1). Moreover, the merged fish has plant dynamics that differ from each of the individual fish (Figure 2A, bottom), highlighting the need to use individualized plants for the analysis of the control system of each fish.

System identification of plant dynamics.

(A) Nyquist plots of the estimated plant models for each individual fish and the merged fish. Blue, green and cyan point clouds correspond to bootstrapped data estimates for the low (0.55 Hz), medium (0.95 Hz) and high (2.05 Hz) frequencies, respectively. Black circles and cross marks represent the mean for measured and bootstrapped data, respectively. Black lines represent the response of the estimated model with diamonds indicating their values at the test frequencies. Each point on the black lines correspond to the complex-valued frequency response of the plant model at a specific frequency. For example, the purple dot on the lower-right panel corresponds to the merged-plant-model’s response at 0.7 Hz. The gain and phase associated with this response are shown on the associated Bode plots given in (C). (B) Reconciliation of physics-based and data-driven models. The solid lines represent the natural feedback control system used by the fish for refuge tracking. The dashed lines represent copies of signals used for parametric system identification. P^(s) represents the parametric transfer function for the plant dynamics of the fish with ‘unknown’ system parameters. The parametric system identification estimates these parameters via minimizing the difference (modeling error) between the actual output of the fish y(t) and the prediction of the model. (C) Gain and phase plots of the frequency response functions of the estimated parametric models for each fish and the merged fish corresponding to the black lines in (A).

Table 1
Estimated parameters of the plant model for each fish as well as the merged fish.
Fish 1Fish 2Fish 3Merged
k(N/m)0.32280.22000.16220.2385
b(Ns/m)0.04170.01860.02180.0269
m(kg)0.00250.00250.00250.0025

Examining the effects of feedback on behavioral variability

Despite the differences in plant dynamics, fish produced remarkably similar tracking performance, consistent with previously published reports (Cowan and Fortune, 2007; Roth et al., 2011). This behavioral robustness could be achieved via precise tuning between the controller and plant dynamics of each fish. Alternatively, the central nervous system (CNS) may be implementing robust feedback control without the need for precise tuning. To test these hypotheses, we built feedback control models that permit computational manipulation of the relationships between controller and plant. Specifically, we swapped the controllers and plants between fish using these computational models (Figure 3). If each fish required precise tuning for consistent behavior, we would expect to see increased variability for the swapped models. Alternatively, a robust feedback controller might be insensitive to mismatch between C(s) and P(s) pairs.

Computational brain swapping between fish.

We computationally swapped the individually tailored controllers between fish. The feedback control diagrams illustrate computational brain swaps. The controller from the light gray fish, C2(s), is used to control the plant of the dark gray fish, P1(s), and vice versa.

We used the second order model proposed by Cowan and Fortune (2007) to represent the input–output behavioral response of the fish:

(2) G(s)=Aωn2s2+2ζωns+ωn2

We estimated the model parameters for each fish using the position of the fish y(t) and the refuge r(t). In other words, we generated individualized parametric transfer functions that capture the input–output behavioral performance of each fish. Parameters varied by about 15–20% (Table 2).

Table 2
Estimated parameters of the second order input-output model for each fish as well as the merged fish.
Fish 1Fish 2Fish 3Merged
A0.530.600.520.57
ξ0.580.550.520.55
ωn(rad/s)8.559.517.888.50

We investigated how the variability in plant dynamics (parameters varied by about twofold; Table 1) is mitigated at the level of behavior (parameters varied by 15–20%; Table 2). Specifically, we inferred a controller for each fish using models of their respective plant dynamics and input–output behavioral responses. Given the plant P(s) and behavioral performance G(s) of each individual fish, we can infer the controller for each fish using the following equation (see Materials and methods and Cowan and Fortune, 2007):

(3) C(s)=G(s)(1-G(s))P(s)

Estimates of controllers and plants for individual fish allowed us to implement computational manipulations in the system models. Given a model of each fish’s plant and controller, we computed the transfer function of the closed-loop system using each fish’s own controller when matched with its own plant (‘matched’, Figure 4A), via the equation (see Equation 11, Materials and methods):

(4) Gmatched,i(s)=Pi(s)Ci(s)1+Pi(s)Ci(s),

for i=1,2,3. Then, to test the hypothesis that the animals rely on precise tuning between their plants and controllers, we substituted the controller of each fish with the plant dynamics of other fish (see ‘swapped’ in Figure 4A), that is a simulated brain transplant, namely

(5) Gswapped,i,j(s)=Pi(s)Cj(s)1+Pi(s)Cj(s)

whereby each controller j=1,2,3 is paired with another fish’s plant ij, for a total of 6 ‘swapped’ cases. If the controllers and plants need to be co-tuned, then we would expect a significant increase in variability in the swapped models.

Effects of feedback on behavioral variability.

At the top of each column is a control diagram that corresponds to all figures in that column. (A) Frequency-response gain and phase plots of the estimated input–output models for the matched (orange), swapped (gray), and merged (green) fish in a feedback control topology. (B) Variability of the matched (orange) and swapped (gray) models. Dashed line is the behavioral variability observed in the tracking data across fish. (C) Same as in (B) but for the loop gain, P(s)C(s). (D) Same as in (C) but for the loop gain, P(s)C(s).

To quantify such changes in variability and to evaluate the fitness of a given computational model for explaining biological data, we defined two metrics termed ‘model variability’ and ‘behavioral variability’. Model variability quantifies the variability of the complex-valued frequency responses of matched or swapped models across a range of frequencies. Behavioral variability, on the other hand, provides a conservative estimate of the variability observed across trials in the real fish. See Model variability and Bootstrap estimate of behavioral variability in Materials and methods.

Using these metrics, we computed the variability of the matched models and compared against the behavioral variability observed in real fish. Unsurprisingly, variability of the matched models (orange region in Figure 4B) remained well below the behavioral variability across the entire frequency range of interest (dashed line in Figure 4B). Surprisingly, however, the model variability remained below the behavioral variability even for swapped models (Figure 4B, gray region versus dashed line). These results highlight the fact that sensory feedback can attenuate the variability of closed-loop models despite mismatch between the controller and plant pairs. In other words, feedback models do not require precise tuning between the controller and plant to achieve the low variability we observed in the behavioral performance of the animals.

Having established that variability of the closed-loop models is robust to the relations between the plant and controller in a feedback system, we examined the role of feedback. This was achieved by examining the loop gain, that is the dynamic amplification of signals that occurs in feedback systems (Figure 4C). The only difference between the loop gain model and the closed-loop model is the absence and presence of feedback, respectively. For our model, this was calculated as the product of the plant and controller in both matched and swapped cases. This removal of feedback revealed dramatic variability in the loop gain at frequencies below about 1 Hz (Figure 4D). This variability was well above the behavioral variability observed in fish. In contrast, at frequencies above 1 Hz, the model variability was slightly reduced. These results indicate that sensory feedback attenuates behavioral variability in the biologically relevant range of tracking frequencies at a cost of slightly increased variability at high frequencies.

Parametrizing the range of neural controllers

As predicted by Cowan and Fortune (2007), each of the feedback controllers obtained above for the averaged fish responses had high-pass filtering characteristics despite the differences in their dynamics (Figure 5A). What is the range of neural controllers that, when used in this feedback control topology, leads to behavior that is indistinguishable from the real fish? In other words, how well tuned to the plant does the neural controller need to be to achieve these behavioral performances?

Feedback controllers satisfying the behavioral variability.

(A) Bode plots for the estimated controllers for the matched (orange) and merged (green) fish under feedback control. Each controller exhibits high-pass filter behavior across the frequency range of interest. Gray shaded regions represent the range of controllers that produce behavioral responses consistent with behavioral variability. (B) The range of behavioral input–output transfer functions consistent with behavioral variability.

We used the Youla-Kučera parametrization to obtain a range of controllers that generate similar behavioral responses (Kučera, 2011). Specifically, this parametrization provided a parametric transfer function describing all stabilizing controllers for the merged plant:

(6) C(s)=Q(s)1-Pm(s)Q(s)

Here, Pm(s) is the transfer function of the merged plant and Q(s) is any stable and proper function.

Equation 6 parametrizes all stabilizing controllers for the merged plant Pm(s). However, we were interested in finding the subset of controllers that yields indistinguishable behavioral performances from real fish. To achieve this, we computed the range for the input–output system dynamics, G(s), of real fish response. Specifically, we calculated the bounds for the gain and phase responses of the 1000 input–output transfer function models estimated while computing the behavioral variability (see Materials and methods). The gray-shaded areas in Figure 5B serve as the range of frequency responses that are consistent with behavioral variability of the real fish.

For each of these 1000 transfer functions G(s) that were consistent with the behavioral variability of the fish, we selected Q(s)=G(s)/Pm(s) to generate 1000 corresponding controllers using Equation 6. The bounds for gain and phase responses of these 1000 controllers (the gray shaded regions) show the breadth of controllers that, when implemented within the feedback control topology, produce behavioral outputs consistent with the performance of the real fish (see Figure 5A). Note that the controllers calculated in Equation 3 also satisfy the structure of Equation 6 when Q(s)=G(s)/P(s) for the associated plant dynamics P(s) of each fish; in other words, the controller estimates are guaranteed to be stabilizing for the associated plants.

These results indicate that the neural controllers need not be precisely tuned to their associated plant dynamics. We found a wide range of controllers that the fish could implement to generate consistent behavioral performance. We note that each of these controllers had high-pass filtering characteristics.

Discussion

Feedback-based task control allows animals to cope with dramatic but nevertheless common variations of their plant dynamics between individuals. Further, individuals can experience variations in plant dynamics over time such as instantaneous increases in weight during feeding (Van Handel, 1965; Hou et al., 2015), muscle fatigue during repetitive behaviors (Enoka and Stuart, 1992), or carrying heavy objects (Zollikofer, 1994). These changes likely result in variable mismatch between the neural controller and the locomotor plant of individual animals. This mismatch is similar to that induced by swapping plants and controllers across individuals, suggesting that moment-to-moment variability can also be eliminated through sensory feedback.

Deciphering the interplay between the task plant, behavioral performance, and neurophysiological activity requires understanding the impacts of the closed-loop control topology. Given the range of morphophysiological features observed across individuals within a species, our results suggest that there is also a range of controller dynamics—ultimately manifest as neurophysiological activity—that each individual could use to achieve consistent biologically relevant behavioral performances. As a consequence, we expect to see more variation at the neurophysiological level than is revealed by task performance for behaviors that rely on closed-loop control.

Reconciling data-driven and physics-based models of locomotor dynamics

A key contribution of this work is the identification of a data-driven plant model for the locomotor dynamics of a freely behaving animal based on behavioral observations only. To achieve this, we adopted a grey-box system identification approach that seeks to reconcile a physics-based parametric transfer function model with a non-parametric data-driven model (i.e., the frequency-response function).

Developing a model from first principles, for example Newton’s laws, is sometimes an effective modeling approach for describing the dynamics of a physical system. For instance, a widely used model in legged locomotion is the spring-loaded inverted pendulum (SLIP) model for describing running dynamics in the sagittal plane (Blickhan and Full, 1993; Full and Koditschek, 1999). While physics-based models have proven to be successful in modeling the dynamics of biological and mechanical movements, there are limitations. Physics-based approaches for modeling behaviors at lower levels (e.g., the spiking activity of all motor neurons) may lead to a very complex model that does not accurately capture high-level behavior.

Data-driven system identification approaches are used to directly identify a dynamical model based on empirical data (Ljung, 1998; Kiemel et al., 2011; Uyanik et al., 2019a). In general, data-driven system identification may take a black-box approach in which only a general model structure is assumed (say, an ODE or frequency response function). However, data-driven techniques typically generate numerical transfer function estimates to represent animal behavior.

Alternatively, the so-called grey-box approach that we adopt in this paper integrates the structure of a specific physics-based model but leaves its parameters free, relying on data-driven system identification to fit those parameters. In this case, prior knowledge about the underlying dynamical model informs and constrains data-driven system identification. Grey-box identification can provide a bridge between top-down, data-driven modeling and bottom-up, physics-based modeling. We utilized the parametric dynamical model of Sefati et al. (2013) for the plant but estimated the model parameters using data-driven system identification techniques. Our results show that the data-driven estimates for the plant dynamics match the structure of this model (Figure 2A).

Effects of variability in plant dynamics

Our results reveal two complementary perspectives on variability in plant dynamics. On the one hand, estimates of the closed-loop controllers were highly sensitive to the dynamics of the plant of individual fish. This was an inevitable consequence of our strategy for estimating the controllers—inferring the controllers from the plant and closed-loop dynamics. On the other hand, our closed-loop control models were robust to variability of either the plant or controller, indicating that precise tuning is not needed for this behavior. A control-theoretic sensitivity analysis demonstrates that these results are not unique to this example but rather are a general property of feedback control systems, see Csete and Doyle (2002) for a review.

Specifically, consider the frequency dependent sensitivity function of the feedback controller C(s) with respect to plant P(s), in the closed-loop topology:

(7) S(CP)(jω)=CP=[G(jω)G(jω]1]1P2(jω)

The sensitivity of the controller to the plant dynamics is a frequency dependent function. It depends on the gain and phase of both the measured closed-loop transfer function G(jω) as well as the plant model P(jω). At low frequencies, fish track extremely well and thus G(jω)-10. At high frequencies, the low-pass plant P(jω) is small. Combining these factors, we expect the sensitivity |S(jω)| to be large across frequencies. In other words, there is an inescapable sensitivity to plant dynamics when the controllers are estimated using this computational strategy.

We conducted a complementary analysis to compute the sensitivity of the closed-loop tracking response G to perturbations in the combined controller and plant dynamics PC. We treated the controller–plant pair PC as a single variable and obtained the frequency-dependent sensitivity function as

(8) S(GPC)(jω)=GPC=1(1+PC(jω))2.

At low frequencies, G(jω) has nearly unity gain and thus PC(jω) goes to ∞. As a result, sensitivity S(GPC) approaches zero. At high frequencies, PC(jω) goes to zero and thus sensitivity is bounded around 1. Thus, despite the fact that the controller estimates are sensitive to plant variations, the closed-loop transfer function (in the presence of sensory feedback) is robust against variability of controller–plant pairs.

Feedback and variability in neural systems

These findings suggest that a fish could implement a range of controllers in its nervous system for refuge tracking. These controllers must have high-pass filtering characteristics, but their details may be inconsequential. This has two implications for neurophysiological analysis of neural control systems. First, neurophysiological activity within control circuits in open-loop experiments (e.g. playback and anesthetized/immobilized animals) need not appear to be well tuned for the control of a behavioral performance. This poor tuning, which may manifest in variability that appears across levels of functional organization—from variability in neural activity within neurons, variability in tuning across neurons, and variability across individuals—is refined via feedback during behaviors in which the feedback loops are intact. Second, there must be mechanisms by which the controllers are slowly (at time constants greater than that necessary for the moment-to-moment control of the behavior) tuned to the dynamics of the animal’s locomotor plant. For instance, adaptation of cerebellar activity in relation to mismatch between intended versus actual motor performances contribute to the retuning of neural controllers (Morton and Bastian, 2006; Bell et al., 1997; Pisotta and Molinari, 2014).

Feedback is mediated both through the effects of behavior on sensory receptors and via descending pathways in the brain. Behavior generally results in concomitant activation of receptor types across the animal, which can include, for example, simultaneous stimulation of stretch receptors embedded in muscles and visual receptors in eyes. Correlations in feedback-related activity across sensory modalities likely contribute to robust control (Roth et al., 2016). Internal feedback pathways, interestingly, have been recently shown to synthesize sensory filtering properties of behaviorally-relevant stimuli. Descending neural feedback is used to dynamically synthesize responses to movement (Metzen et al., 2018; Huang et al., 2018; Clarke and Maler, 2017).

How do other animal systems manage variability, broadly speaking, to achieve consistent output? In pyloric neural circuits of crustaceans, the oscillatory output of the system is consistent despite dramatic variations in the dynamics of cellular and membrane properties of neurons within these circuits (Goaillard et al., 2009; Marder and Taylor, 2011). How these circuits maintain consistent output despite underlying variability remains an open question (Hamood and Marder, 2014) but likely relies on feedback regulation that is intrinsic to the neural network itself.

The mechanisms by which systems maintain consistent output can be assessed through behavioral analysis (Krakauer et al., 2017) of responses to systematic perturbations (Cowan et al., 2014). For example, perturbations have been used in many species including flying insects (Bender and Dickinson, 2006; Matthews and Sponberg, 2018), walking sticks (Dallmann et al., 2019; Diederich et al., 2002), and humans (Lee and Perreault, 2019) to reveal how control systems manage mechanical and sensory variation. These analyses and others show that animals rely on sensory feedback to modulate moment-to-moment movement to maintain consistent task performances.

Ultimately, understanding how robustness emerges in closed loop requires investigating the interplay between plants and controllers, which are inextricably linked (Cowan and Fortune, 2007Tytell et al., 2011Tytell et al., 2018). A commonly implemented strategy, for example, is the use of low-pass plant dynamics. This strategy can avoid instabilities that arise from long-latency feedback, an inescapable feature of biological control systems (Sponberg et al., 2015; Madhav and Cowan, 2020). Specifically, delay introduces phase lag that increases with frequency. As the phase angle of the loop gain PC approaches −180°, the likelihood of instability increases. A low-pass plant can mitigate this instability by ensuring that the gain of |PC|1 near this ‘phase crossover frequency.’ In turn, the animal exhibits low behavioral sensitivity, namely S(GPC)1 in Equation 8. In short, a well-tuned neuromechanical plant can simplify feedback control by rendering the dynamics passively self-stabilizing (Hedrick et al., 2009; Sefati et al., 2013), while nevertheless maintaining behavioral flexibility (Cowan et al., 2014).

Materials and methods

All experimental procedures with fish were reviewed and approved by the Johns Hopkins University and Rutgers University Animal Care and Use Committees and followed guidelines for the ethical use of animals in research established by the National Research Council and the Society for Neuroscience.

Adult Eigenmannia virescens, a species of weakly electric Gymnotiform fish, were obtained through commercial vendors and housed in laboratory tanks. Tanks were maintained at a temperature of approximately 27°C and a conductivity between 50 − 200 μS . We transferred individual fish to the experimental tank about 1 day before experiments for acclimation. Three fish were used in this study.

Experimental apparatus

Request a detailed protocol

The experimental apparatus is similar to that used in previous studies (Stamper et al., 2012; Biswas et al., 2018; Uyanik et al., 2019b). A refuge machined from a PVC pipe with a length of 15 cm and 5.08 cm diameter was placed in the experimental tank with the fish. The bottom face of the refuge was removed to allow video recording from below. Six windows, 0.625 cm in width and spaced within 2.5 cm intervals, were machined onto each side to provide visual and electrosensory cues. During experiments, we actuated the refuge using a linear stepper motor with 0.94 μm resolution (IntelLiDrives, Inc Philadelphia, PA, USA) driven via a Stepper motor controller (Copley Controls, Canton, MA, USA). MATLAB (MathWorks, Natick, MA, USA) scripts were used to control the movement of the refuge and to capture video. Video data were captured using a pco.1200s high speed camera (Cooke Corp., Romulus, MI, USA) with a Micro-Nikkor 60 mm f/2.8D lens (Nikon Inc, Melville, NY, USA). All videos used for data analysis were shot at 30 frames per second with 1280 × 1024 pixel resolution. Some videos of ribbon fin motion were shot at 100 frames per second.

Experimental procedure

Request a detailed protocol

Refuge movement consisted of single sine waves of amplitude 0.1 cm and of frequencies 0.55, 0.95, and 2.05 Hz. The amplitude of refuge movements was chosen because fish rely on counter propagating waves for tracking in this regime (Roth et al., 2011). At higher amplitudes, fish often will use a uni-directional wave in the ribbon fin for locomotion. The frequencies were selected to be within the normal tracking regime as determined in previous studies (Stamper et al., 2012; Biswas et al., 2018; Uyanik et al., 2019b). Trials were randomized with respect to frequency. Each trial lasted 60 seconds. The stimulus amplitude was linearly ramped up over the first ten seconds to prevent startle responses from the fish. During the experimental phase, the stimulus frequency and amplitude were maintained for 40 seconds. Finally, the stimulus amplitude was ramped down during the final ten seconds. Trials were separated by a minimum break of 2 minutes.

Derivation of plant model

Request a detailed protocol

Sefati et al. (2013) developed and tested a second-order, linear, ordinary differential equation that describes how changes in fore-aft position of the animal, y(t), relate to changes in position of the nodal point, u(t):

(9) md2ydt2+bdydt=ku(t)

Here, m, k, and b represent mass, gain, and damping, respectively. This equation follows from Equation S13 in the Supporting Information of Sefati et al. (2013). Note that the present paper uses slightly different nomenclature; in particular u(t), y(t), and b in the present paper correspond to ΔL, x(t), and β, respectively, in Sefati et al. (2013).

The Laplace transform provides a computationally convenient means to represent dynamics of linear, time-invariant systems such as the one in Equation 9 (Roth et al., 2014). Taking the Laplace transform of both sides of Equation 9, neglecting initial conditions, and algebraically simplifying, we arrive at the plant model in Equation 1:

(10) Y(s)U(s)=kms2+bsP(s)

Inferring controller using plant and closed-loop dynamics

Request a detailed protocol

Given the feedback control topology in Figure 1, the closed-loop dynamics relating the movement of the refuge to the movement of the fish are given in the Laplace domain by the following equation:

(11) G(s)=P(s)C(s)1+P(s)C(s)

This equation is also shown in Equation 7 of Cowan and Fortune (2007) with a slightly different nomenclature; in particular P(s) and G(s) in the present paper correspond to G(s) and H(s), respectively, in Cowan and Fortune (2007).

Given G(s) and P(s), one can compute the complementary controller C(s) using Equation 11 as

(12) C(s)=G(s)(1-G(s))P(s).

Reconciling data-driven and physics-based approaches to estimate the locomotor dynamics

Request a detailed protocol

The position of the refuge and fish were tracked for each video using custom software (Hedrick, 2008). The videos were analyzed to extract 3 to 10 seconds segments, where the fish used counter propagating waves for refuge tracking. Then, the nodal point was hand clicked in these video segments: 18,000 nodal point measurements were made over a total of 106 segments of data.

The physics-based plant model in Sefati et al. (2013) was previously validated with quasi-static open-loop experiments. Here we reconciled the physics-based plant model from Sefati et al. (2013) (Equation 1) with the data that were collected in tracking experiments.

For each frequency of refuge movement, M segments of nodal point data were extracted. Each segment of data consists of the following measurements: nodal point shift {u1m,u2m,,unm} and fish position {y1m,y2m,,ynm} as a function of time {t1m,t2m,,tnm}, where n is the number of samples and m={1,2,,M}.

We estimated the magnitude and phase of the plant model for each frequency of refuge movement. The average value of nodal point shift and fish position were computed from M data segments per fish for each frequency of refuge movement. We aligned each data segment based on the phase of refuge signals. The segments are not completely overlapping: we selected the largest time window with at least 50 percent overlap of data segments. A sine wave function of the following form was fitted to the average nodal point data, uavg(t), and average fish position data, yavg(t), as

(13) uavg(t)=Ausin(2πfit+ϕu)+Bu,
(14) yavg(t)=Aysin(2πfit+ϕy)+By,

where input–output pairs (Au,Ay), (ϕu,ϕy) and (Bu,By) correspond to magnitudes, phases and DC offsets in polar coordinates, respectively. Note that this fitting was done separately for each refuge frequency, fi={0.55,0.95,2.05} Hz.

After computing the magnitude and phase for both the average nodal shift and fish position, we estimated the magnitude and phase for the plant transfer function at ωi=2πfi as

(15) |P^(jωi)|=AyAu,
(16) P^(jωi)=ϕyϕu,

We obtained a non-parametric estimate of the plant transfer function for each frequency ωi, that is P^(jωi) by estimating magnitude and phases. We used P^(jωi) to estimate the parameters of the transfer function model given in Equation 1. In this model, there are three unknown parameters, namely m, k, and b. However, for the fitting purposes we reduced the number of unknown parameters to two by normalizing the ‘gain’ (k) and ‘damping’ (b) by the ‘mass’ (m). The normalized plant transfer function in Fourier domain takes the form

(17) P(jω)=k/m(jω)2+(b/m)(jω)=k/m-ω2+(bjω/m).

where j=-1. For an ideal deterministic system, for each frequency P(jωi)=P^(jωi), where P^(jωi) corresponds to the non-parametrically computed frequency response function. For this reason, estimates of transfer function parameters were made by minimizing a cost function using gradient descent method:

(18) J(k/m,b/m)=i=13|P(jωi)-P^(jωi)|2.

Bootstrap estimate of behavioral variability

Request a detailed protocol

We estimated behavioral variability using bootstrap estimates derived from individual experimental trials at the three test frequencies. Across all three fish, we made 37 observations of the frequency response at fi=0.55 Hz, 35 observations at fi=0.95 Hz, and 34 observations at fi=2.05 Hz, namely:

(19) {G^1(j2π0.55),G^2(j2π0.55),,G^37(j2π0.55)}{G^1(j2π0.95),G^2(j2π0.95),,G^35(j2π0.95)}{G^1(j2π2.05),G^2(j2π2.05),,G^34(j2π2.05)}

To estimate the behavioral variability at frequencies that were not explicitly tested, we used a parametric approach. Specifically, we constructed N=1000 triplets by randomly selecting one frequency response function from each of the test frequencies in Equation 19. For each of the 1000 triplets, we estimated a transfer function, Gbootstrap,i(s), i=1,,N of the form in Equation 2 using Matlab’s transfer function estimation method ‘tfest’.

Let xi and yi be real and imaginary parts of the complex-valued frequency response function, namely Gbootstrap,i(jω0)=xi+jyi, where ω0 is a frequency in the range 0.2 Hz to 2.05 Hz. The covariance matrix for the estimated frequency response function in the complex domain was calculated as

(20) Covω0=[σxxω0σxyω0σyxω0σyyω0,]

where

(21) σxxω0=1N1i=1N(xiμx)2,
(22) σyyω0=1N1i=1N(yiμy)2,
(23) σxyω0=σyxω0=1N1i=1N(xiμx)(yiμy).

Here, μx and μy are mean values of xi and yi, i{1,2,,N}, respectively. The final bootstrap estimate of behavioral variability was calculated as the largest singular value of the central covariance matrix, Covω0. In addition, the range of the gain and phase of these 1000 transfer function models was plotted in Figure 5B.

Model variability

Request a detailed protocol

The variability across ‘matched’ and ‘swapped’ models was calculated for both the closed-loop transfer function G(s) (Figure 4B) and the loop gain P(s)C(s) (Figure 4D). We evaluated each of the three fish-specific controllers and plant transfer functions at frequencies between 0.2 Hz and 2.05 Hz; for each frequency ω0 in this range, we have C1(jω0), C2(jω0), and C3(jω0) and P1(jω0), P2(jω0), and P3(jω0).

To calculate the matched closed-loop variability, we first calculated the real (xi) and imaginary (yi) parts of Ci(jω0)Pi(jω0)/(1+Ci(jω0)Pi(jω0)). Using these values, the matched closed-loop variability was calculated as the largest singular value of the central covariance matrix of these ordered pairs. The matched loop-gain variability was calculated similarly, using the real and imaginary parts of Ci(jω0)Pi(jω0). For each of these calculations, N=3, because there are three sets of matched models. The closed-loop and loop-gain swapped variability was calculated identically, except using the N=6 swapped permutations of control–plant pairs. Figure 4B,D illustrates the variability of the matched and swapped models both for closed-loop control and loop gain.

References

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
    Templates and anchors: neuromechanical hypotheses of legged locomotion on land
    1. RJ Full
    2. DE Koditschek
    (1999)
    The Journal of Experimental Biology 202:3325–3332.
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
    System Identification: Theory for the User (Second)
    1. L Ljung
    (1998)
    Prentice Hall.
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
    Longitudinal tracking responses of Eigenmannia and Sternopygus
    1. GJ Rose
    2. JG Canfield
    (1993)
    Journal of Comparative Physiology. A, Sensory, Neural, and Behavioral Physiology 173:698–700.
    https://doi.org/10.1007/BF00213075
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
    Stepping patterns in ants-influence of load
    1. C Zollikofer
    (1994)
    The Journal of Experimental Biology 192:119–127.

Decision letter

  1. Ronald L Calabrese
    Senior and Reviewing Editor; Emory University, United States
  2. Len Maler
    Reviewer; University of Ottawa, Canada

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

The authors ask whether the neural controller (used when an animal engages in precise behavior) is optimized for each animal’s particular motor plant properties or whether a generic controller is sufficient and instead robust feedback mechanisms ensure that the generic controller works well despite variations in the motor plant. The authors have found a very clever way to address this problem in the refuge tracking behavior of an electric fish. The CNS controller controls the location of a 'nodal point' along the ribbon fin of these fish. Forward- and backward-directed oscillations of the ribbon fin (opposing forces) meet at the nodal point and the fish can precisely control the location of the nodal point in order to achieve both stability and maneuverability during locomotion. The movement of the nodal point serves as a linear estimate of the controller output of the fish as it tracks the moving refuge. Kinematic measurements were used to generate individualized estimates of each fish's locomotor plant and controller revealing substantial variability between fish. To test the impact of this variability on behavioral performance, these models were used to perform simulated 'brain transplants' – computationally swapping controllers and plants between individuals. The authors conclude that simulated closed-loop performance was robust to mismatch between plant and controller. This seminal observation suggests that animals rely on feedback rather than precisely tuned neural controllers to compensate for morphophysiological variability.

Decision letter after peer review:

Thank you for submitting your article "Variability in locomotor dynamics reveals the critical role of feedback in task control" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by Ronald Calabrese as the Senior and Reviewing Editor. The following individual involved in review of your submission has agreed to reveal their identity: Len Maler (Reviewer #1).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

This is an impressive paper and builds on a series of impressive papers. The quality of the experiments and analyses is very high. The authors' perspective is from the engineering approach in which the motor activity of, e.g., a robot or weakly electric fish is divided into the mechanics (motor plant) and 'controller' (sensory system and CNS). Furthermore, the mechanics will vary across different animals due to their size etc. The question posed by the authors is whether the controller is optimized for each animal for its particular motor plant properties vs. whether a generic controller is sufficient and robust feedback mechanisms ensure that the generic controller works well despite variations in the motor plant. This is a very general question and one that is hard or impossible to address in most animal models. As carefully argued by the authors, devising separate data-driven models of controller and plant is very difficult for muscle-based plants simply because most motor actions (e.g., stepping) require coordination of many muscles across joints.

The authors, based on prior papers, have found a very clever way to circumvent this problem in the refuge tracking behavior of a weakly electric fish (Eigenmannia). Their earlier studies showed that the CNS controller operated by controlling the location of a 'nodal point' along the ribbon fin of these fish. Forward- and backward-directed oscillations of the ribbon fin (opposing forces) meet at the nodal point and the fish can precisely control the location of the nodal point in order to achieve both stability and maneuverability during locomotion. The position of the nodal point varies as a function of swim speed. The authors argue that the movement of the nodal point can serve as a linear estimate of the controller output of the fish as it tracks the moving refuge. Kinematic measurements of Eigenmannia were used to generate individualized estimates of each fish's locomotor plant and controller revealing substantial variability between fish. To test the impact of this variability on behavioral performance, these models were used to perform simulated 'brain transplants' – computationally swapping controllers and plants between individuals. The authors conclude that simulated closed-loop performance was robust to mismatch between plant and controller. This suggests that animals rely on feedback rather than precisely tuned neural controllers to compensate for morphophysiological variability.

Essential revisions:

1) There is a consensus that the manuscript is too dense with many points needing to be clarified as detailed in the attached reviews (Reviewer #2 in particular). We do think that this paper should be in a general journal like eLife, partially because it is important that neurobiologists become at least familiar with the concepts and, to a limited extent, to the methods of control theory as practiced by engineers. Right now, the manuscript just won't work for most neurobiologists. But it can be written in a much clearer way. The import of the equations should be explained in the text and the derivations placed in a supplementary information section for specialists.

2) Might the model be tested directly/experimentally by perturbing the motor plant, for example weighting the fish by overfeeding or with a weight belt (Reviewer #3)? There is a realization that these approaches might be unsuitable, but the experimenters are resourceful, and some perturbation might be found, e.g. clipping some of the ribbon fin (caudal quarter maybe).

Reviewer #1:

This is an impressive manuscript and built on a series of impressive papers. The quality of the experiments and analyses are very high and I have no serious concerns about the conclusions drawn with one caveat (see below).

The authors’ perspective is from the engineering approach in which the motor activity of, e.g., a robot or weakly electric fish is divided into the mechanics (motor plant) and 'controller' (sensory system and CNS). Furthermore, the mechanics will vary across different animals due to their size etc. The question posed by the authors is whether the controller is optimized for each animal with respect to its particular motor plant properties vs. whether a generic controller is sufficient and robust feedback mechanisms ensure that the generic controller works well despite variations in the motor plant.

This is a very general question and also one that is hard or impossible to address in most animal models. As carefully argued by the authors, devising separate data-driven models of controller and plant is very difficult for muscle-based plants simply because most motor actions (e.g., stepping) require coordination of many muscles across joints.

The authors, based on prior papers, have found a very clever way to circumvent this problem in the refuge tracking behaviour of a weakly electric fish (Eigenmannia). Their earlier studies showed that the CNS controller operated by controlling the location of a 'nodal point' along the ribbon fin of these fish. Forward- and backward-directed oscillations of the ribbon fin (opposing forces) meet at the nodal point and the fish can precisely control the location of the nodal point in order to achieve both stability and maneuverability during locomotion. The position of the nodal point varies as a function of swim speed. The authors argue, and I agree, that the movement of the nodal point can serve as a linear estimate of the controller output of the fish as it tracks the moving refuge.

Given this assumption and the authors’ prior papers, the experiments were well justified and well carried out. The analysis is, I expect, also well carried out. But here I think that the authors simply skipped too many steps in their derivations. This is the major problem that I think must be addressed in a revision.

1) "We adopted the physics-based parametric model of locomotor dynamics of Eigenmannia described by Sefati et al., 2013", this is the 'controller dynamics'. This is Equation 1 in the Results and gives P(s). I think this is derived from Equation 4 is Sefati et al., 2013 (derived very nicely in the supplementary information). But, I fail to see the derivation. It is true that I did not expend a lot of effort trying to derive Equation 3 from Equation 4; however, I think that a typical reader should not have to make this attempt. The authors should make the derivation explicitly, perhaps in a supplementary information section (as was done in Sefati et al., 2013).

2) The authors then use a previous paper (Cowan and Fortune, 2007) to represent the behavioural response of the fish (Equation 2); this is well justified and gives G(s). The next point is the most critical and most novel. The authors then 'back calculate' the parameters for the controller, P(s), by using data driven behavioural model encapsulated in G(s). This results in Equation 3. At this point, Cowan and Fortune is cited and I got confused. Their closed loop transfer function, H(s), has an entirely different form from Equation 3. I am presumably missing something here and I think the authors really have to spell out in detail how the do the 'back calculation' and how it connects to Cowan and Fortune's results. Again, this would likely be a fairly detailed derivation and, as in the Sefati et al. paper, could be presented in supplementary information.

3) Lastly, I was struck by the parallel between the authors' conclusions and that of Marder and colleagues. Dr. Marder and colleagues have shown, over many papers, that the output of the STG pattern generator can be produced by many different (with respect to parameters) STG models. There is no unique STG pattern generator and the average parameters don't match any real STG. A paper by O'Leary and Marder (2016, Current Biology) advances a similar argument to the one in this manuscript – that "The high variability in conductance densities in these neurons [9, 10] appears to contradict the possibility that robustness is achieved through precise tuning of key temperature-dependent processes."

This paper uses Ca-dependent global feedback (Ca entry depends on activity) to maintain robust STG activity over a wide range of temperatures, despite the individual ion channels not being tuned for this temperature range. Although the systems are entirely different the principle seems, to me, to be the same. It would be great to add this to the Discussion.

One overall conclusion of this paper is that "Physics-based approaches for modeling behaviors at lower levels (e.g., the spiking activity of all motor neurons) may lead to a very complex model that does not accurately capture high-level behavior." I fully agree and I also am convinced that the feedback tuning mechanisms of this manuscript and the O'Leary and Marder paper are the way to go for modelling complex systems – when one is dealing with even the electrosensory system of Eigenmannia, there are too many parameters to be estimated for a realistic approach to neural models. This is a powerful conclusion and giving it the wider context – crab STG and Eigenmannia electrosensory system and ribbon fin.

Reviewer #2:

In this paper the authors build on their prior work examining the neuromechanical control of locomotion in Eigenmannia to attack an important and understudied question in motor control – how neural control strategies vary based on individual differences in the motor plant across individuals. To do so, the authors combine detailed behavioral measurements with control-theoretic analysis and simulations to argue that animals use feedback (sensory) signals, rather than precisely-tuned feedforward neural controllers, to robustly control behavior.

This is a potentially highly impactful paper, however, I have three major concerns as listed below. Also, I should provide the caveat that I am neither an expert in models of fish locomotion nor in the kind of control-theoretic analyses used here. Therefore I must leave the technical evaluation of these aspects to other reviewers.

1) Comprehensibility of this paper to a general audience of biologists. This is a highly technical paper which the authors are trying to squeeze into 4 main-text figures and the absolute minimum of explanatory text. Because of this choice, I think that this paper will not be comprehensible to most eLife readers (in its present state this seems much more appropriate for PLOS Comp Bio or some more specialized computational journal), thereby reducing its potential impact. I think the authors need to expand the text (and possibly include a couple of explanatory figures) to allow readers without control systems expertise to understand the paper. For example, a reader not already familiar with Nyquist or Bode plots will not be able to understand any of the key evidence for the authors' arguments presented in Figures 3 or 4. In the main text, the authors often cite their prior work rather than explaining essential components of their approach. This includes small things like not defining/explaining the variables in Equation 2 as well as explaining the key strengths of their model system – it took me several readings to understand what was meant by the nodal point providing a "linear proxy" for u(t).

2) Dissociating the role of feedback control from that of plant dynamics. The authors present the study as attempting to distinguish between two alternatives – that behavioral robustness arises from either precise feedforward neural control or well-tuned feedback control. However it's not clear to me whether or how the authors have ruled out a third possibility – that the physics of the plant are such that behavioral output (forward-backward movement in the refuge task) is simply insensitive to variations in the motor command, at least over the range of motor command variations actually produced by the fish. In fact the result shown in Figure 3C would appear to be consistent with this possibility. The authors offer this interpretation:

"Surprisingly, however, the variability remained below the trial-to-trial variability (Figure 3C, gray region versus dashed line). These results highlight the fact that sensory feedback can attenuate the output variability despite mismatch between the controller and plant pairs. In other words, feedback models do not require precise tuning between the controller and plant to achieve the low variability we observed in the behavioral performance of the animals."

It seems to me that an equally plausible explanation is not that sensory feedback is compensating for "mismatch" errors, but rather that the physical plant is simply insensitive to errors. How can the authors rule this out? An analysis quantifying how much the natural variations in u(t) actually cause variations in y(t) would seem critical to answer this. It's possible that this issue is already implicitly addressed in the analyses provided, but if so I can't figure this out and I'd like the authors to clarify.

3) Expanding on a couple of Discussion points

The authors conclude the Discussion as follows:

"The dynamic synthesis of filters based on current sensory information may be a mechanism that shapes neural controllers on two time scales: for the moment-to-moment task dynamics and over the longer term for the maintenance of behavioral performances."

I'd suggest that the authors expand on this idea a bit; at present it's not very clear to me how they think their simulation results (e.g. that the synthesis of the high-pass controller and low-pass plant yields a robust system) relate to the question of multiple timescales.

Also I'd encourage the authors to expand on possible biological implementations of their findings, and speculate a bit on how those implementations might be similar or different from those known in other model systems. For instance, the controller described here is able to produce similar results (i.e., tracking the same stimulus albeit with greater variability) with different parameters (i.e., switched case) and is reminiscent of multifunctional neural circuits such as those characterized in the crustacean pyloric system. In addition, the characterization of how feedback reduces variability is similar to how the role of feedback and CPG entrainment is studied in multiple species including cats, stick insects, and crayfish. To be clear I'm not suggesting that the authors review all of these (or even any of them in great depth), but I'm sure readers will wonder how the authors’ theories relate to ideas from other systems.

Reviewer #3:

I've been thinking a lot about this paper and while I argue that it should eventually be published in eLife, I think (for this broad readership journal) it could use some expansion regarding pedagogy on the control aspects. I find it very challenging to decide if I believe that the "simulated swapping" tells me anything fundamental about the system, or if it is just a computational exercise. I think it is not, but it is hard for me to evaluate given all the assumptions going into "gray box" model. What if the black box model isn’t the correct one? In particular, since the authors likely have room in terms of word count, and certainly have space in terms of numbers of figures, I think a bit more explanation of the methods to demystify the story would be helpful.

Based on the first paragraph of the Introduction, I would have expected to see how the controller changes when fish lose or gain weight (or some manipulation). But I am not convinced by their statement "These changes likely result in variable mismatch between the neural controller and the locomotor plant of individual animals. This mismatch is similar to that induced by swapping plants and controllers across individuals, suggesting that moment-to-moment variability can also be eliminated through sensory feedback." Is this true? Why not do the (obviously harder) experiment of indeed letting the fish eat a lot (or not) and measuring the controller? That way, the authors don't have to do a virtual swap. Even better, why not add weighted "jackets" to the fish as a real "plant" manipulation? I think if they could show this, the paper would become outstanding (instead of "just" very cool as it is now).

This phrase in the Abstract "virtually indistinguishable between conspecifics" is a bit overwrought. What does "virtually indistinguishable" mean?

https://doi.org/10.7554/eLife.51219.sa1

Author response

Essential revisions:

1) There is a consensus that the manuscript is too dense with many points needing to be clarified as detailed in the attached reviews (reviewer #2 in particular). We do think that this paper should be in a general journal like eLife, partially because it is important that neurobiologists become at least familiar with the concepts and, to a limited extent, to the methods of control theory as practiced by engineers. Right now, the manuscript just won't work for most neurobiologists. But it can be written in a much clearer way. The import of the equations should be explained in the text and the derivations placed in a supplementary information section for specialists.

Initially, we went too far in avoiding being “tutorial”, but we now recognize​ that we missed the mark and that our approaches require far more explanation to be accessible to a general biology audience. We address the details of this Essential Revision in our detailed responses to the individual reviewers.

2) Might the model be tested directly/experimentally by perturbing the motor plant, for example weighting the fish by overfeeding or with a weight belt (reviewer #3)? There is a realization that these approaches might be unsuitable, but the experimenters are resourceful, and some perturbation might be found, e.g. clipping some of the ribbon fin (caudal quarter maybe).

The current paper uses computational models to test the idea that the​ controllers are robust to perturbations in the plant dynamics. However, as the reviewers suggested, one might directly test this idea on the animal itself by adding a weight jacket, overfeeding, or clipping the ribbon fin. Indeed, such experiments are described in Roth, Sponberg, and Cowan, 2014 (see p57, first column, last paragraph therein).

We have considered trying similar experiments over the last 10 years and have not yet found the silver bullet. First, adding weight to these fish is extremely hard for at least two reasons: (a) any added mass would have to be neutrally buoyant (such as a water balloon), limiting the materials that can be used, and (b) these are very delicate animals, and so suturing or otherwise attaching objects of significant mass to the animal is extremely challenging without potential injury to the animal. Second, we have considered more dramatic measures such as fin clipping, especially since animals often arrive to the lab with damage to their ribbon fins. However, in both of these cases, one would want to perform the mechanical alterations “on the fly” to avoid the longer-time scales of sensorimotor adaptation. Lastly, we are unaware of examples where our fish will eat a significant portion of their body mass in one feeding.

However, the issue raised by the reviewers is both fundamental and not well explained in the original manuscript. We have added a new discussion point on this subject (subsection “Feedback and Variability in Neural Systems”, fourth paragraph), where we point to studies that perform analogous plant perturbation experiments in other model organisms.

Reviewer #1:

[…]

Given this assumption and the authors prior papers, the experiments were well justified and well carried out. The analysis is, I expect, also well carried out. But here I think that the authors simply skipped too many steps in their derivations. This is the major problem that I think must be addressed in a revision.

1) "We adopted the physics-based parametric model of locomotor dynamics of Eigenmannia described by Sefati et al., 2013", this is the 'controller dynamics'. This is Equation 1 in the Results and gives P(s). I think this is derived from Equation 4 is Sefati et al., 2013 (derived very nicely in the supplementary information). But, I fail to see the derivation. It is true that I did not expend a lot of effort trying to derive Equation 3 from Equation 4; however, I think that a typical reader should not have to make this attempt. The authors should make the derivation explicitly, perhaps in a supplementary information section (as was done in Sefati et al., 2013).

We have added the derivation and some clarifications to the Materials and​ methods. Specifically, we included a new subsection in Materials and methods, “Derivation of Plant Model”, which explains how P(s) is derived.

2) The authors then use a previous paper (Cowan and Fortune, 2007) to represent the behavioural response of the fish (Equation 2); this is well justified and gives G(s). The next point is the most critical and most novel. The authors then 'back calculate' the parameters for the controller, P(s), by using data driven behavioural model encapsulated in G(s). This results in Equation 3. At this point, Cowan and Fortune is cited and I got confused. Their closed loop transfer function, H(s), has an entirely different form from Equation 3. I am presumably missing something here and I think the authors really have to spell out in detail how the do the 'back calculation' and how it connects to Cowan and Fortune's results. Again, this would likely be a fairly detailed derivation and, as in the Sefati et al. paper, could be presented in supplementary information.

Yes, this was particularly confusing as some of the variables, including​ H(s), were renamed between previous publications and this manuscript. These differences have been resolved, and the essential components of the derivation added to the Materials and methods subsection “Inferring Controller using Plant and Closed-Loop Dynamics”.

Specifically, the closed-loop transfer function in the current manuscript, G(s) in Equation 2 is identical to H(s) in Equation 2 of Cowan and Fortune, 2007. Both manuscripts use the same approach: back-calculation of controllers from the closed-loop transfer function and plant dynamics. However, in the current manuscript we derived the controller for a general closed-loop transfer function G(s) and plant dynamics P(s) to obtain the solution given in Equation 3. This approach differs from Cowan and Fortune, 2007, where they solved for controllers for hypothetical​​ (not experimentally estimated) ​plant dynamics. Nevertheless, although the final equations for the controllers look different, they originate from the same derivations.

3) Lastly, I was struck by the parallel between the authors' conclusions and that of Marder and colleagues. Dr. Marder and colleagues have shown, over many papers, that the output of the STG pattern generator can be produced by many different (with respect to parameters) STG models. There is no unique STG pattern generator and the average parameters don't match any real STG. A paper by O'Leary and Marder (2016, Current Biology) advances a similar argument to the one in this manuscript – that "The high variability in conductance densities in these neurons [9, 10] appears to contradict the possibility that robustness is achieved through precise tuning of key temperature-dependent processes."

This paper uses Ca-dependent global feedback (Ca entry depends on activity) to maintain robust STG activity over a wide range of temperatures, despite the individual ion channels not being tuned for this temperature range. Although the systems are entirely different the principle seems, to me, to be the same. It would be great to add this to the Discussion.

One overall conclusion of this paper is that "Physics-based approaches for modeling behaviors at lower levels (e.g., the spiking activity of all motor neurons) may lead to a very complex model that does not accurately capture high-level behavior." I fully agree and I also am convinced that the feedback tuning mechanisms of this manuscript and the O'Leary and Marder paper are the way to go for modelling complex systems – when one is dealing with even the electrosensory system of Eigenmannia, there are too many parameters to be estimated for a realistic approach to neural models. This is a powerful conclusion and giving it the wider context – crab STG and Eigenmannia electrosensory system and ribbon fin.

Thanks for suggesting the STG work. We are always interacting with people​ in the STG field, but somehow we didn’t make this connection. This is related to reviewer 2’s comments, and the two have been folded together in two new paragraphs at the end of the Discussion (subsection “Feedback and Variability in Neural Systems”, third and fourth paragraphs).

Reviewer #2:

[…]

This is a potentially highly impactful paper, however, I have three major concerns as listed below. Also, I should provide the caveat that I am neither an expert in models of fish locomotion nor in the kind of control-theoretic analyses used here. Therefore I must leave the technical evaluation of these aspects to other reviewers.

1) Comprehensibility of this paper to a general audience of biologists. This is a highly technical paper which the authors are trying to squeeze into 4 main-text figures and the absolute minimum of explanatory text. Because of this choice, I think that this paper will not be comprehensible to most eLife readers (in its present state this seems much more appropriate for PLOS Comp Bio or some more specialized computational journal), thereby reducing its potential impact. I think the authors need to expand the text (and possibly include a couple of explanatory figures) to allow readers without control systems expertise to understand the paper. For example, a reader not already familiar with Nyquist or Bode plots will not be able to understand any of the key evidence for the authors' arguments presented in Figures 3 or 4. In the main text, the authors often cite their prior work rather than explaining essential components of their approach. This includes small things like not defining/explaining the variables in Equation 2 as well as explaining the key strengths of their model system – it took me several readings to understand what was meant by the nodal point providing a "linear proxy" for u(t).

We added subsections “Derivation of Plant Model” and “Inferring Controller​ using Plant and Closed-Loop Dynamics” to the Materials and methods that provide step-by-step derivations and applications of the equations.

In addition, we added and revised sections in the Materials and methods related to calculating variability in the model systems. These new subsections are “Bootstrap Estimate of Behavioral Variability” and “Model Variability.”

Other changes were made throughout to improve the clarity of the modeling and analysis; see for example, subsection “Estimating a Data-Driven Plant Model”.

We also reorganized the figures to improve comprehensibility of the paper as follows: the revised Figure 1 now shows the schematic of the system, a feedback diagram, and example data for ribbon-fin location. The revised Figure 2 now shows the plant model, including how model parameters were estimated. The revised Figure 3 focuses on the “computational brain transplant”. The revised Figure 4 presents the primary computational results on the role of feedback on variability. The Figure 5 in the current manuscript is identical to Figure 4 in the previous submission.

As background, we had two major concerns about describing the math when we were writing the original manuscript. First, we were concerned that too much emphasis on the equations would obscure the critical point that this work is driven by experimental data obtained in animals. That data is a central feature of the paper that is essential for all of the subsequent modeling and analyses. Second, we were concerned that the manuscript might become a form of tutorial on the application of control theory, which was not the primary objective of this manuscript.

That said, our long-term goal to help bring these concepts and approaches into the general lexicon of biologists who study movement and locomotion. In addressing the reviewers’ concerns, we think that the revised manuscript does a far better job of achieving that long-term goal while avoiding the pitfalls that concerned us before.

2) Dissociating the role of feedback control from that of plant dynamics. The authors present the study as attempting to distinguish between two alternatives – that behavioral robustness arises from either precise feedforward neural control or well-tuned feedback control. However it's not clear to me whether or how the authors have ruled out a third possibility – that the physics of the plant are such that behavioral output (forward-backward movement in the refuge task) is simply insensitive to variations in the motor command, at least over the range of motor command variations actually produced by the fish. In fact the result shown in Figure 3C would appear to be consistent with this possibility. The authors offer this interpretation:

"Surprisingly, however, the variability remained below the trial-to-trial variability (Figure 3C, gray region versus dashed line). These results highlight the fact that sensory feedback can attenuate the output variability despite mismatch between the controller and plant pairs. In other words, feedback models do not require precise tuning between the controller and plant to achieve the low variability we observed in the behavioral performance of the animals."

It seems to me that an equally plausible explanation is not that sensory feedback is compensating for "mismatch" errors, but rather that the physical plant is simply insensitive to errors. How can the authors rule this out? An analysis quantifying how much the natural variations in u(t) actually cause variations in y(t) would seem critical to answer this. It's possible that this issue is already implicitly addressed in the analyses provided, but if so I can't figure this out and I'd like the authors to clarify.

The reviewer is correct that the issue concerning the potential for the plant itself to be insensitive to errors was addressed implicitly in the original manuscript. Namely, if the plant were somehow able to reduce variability, it would manifest in the loop gain, since the loop gain is P(s)C(s). Indeed, the plants for the individual and merged fish are shown in Figure 2 and as can be seen, their gains vary by nearly a factor of 2 (from about 5 to about 10 at the lowest frequency). In addition, a related issue was mentioned in the original manuscript in describing parameter variability: “parameter estimates for the plant models varied two fold, but the parameter estimates for the closed-loop models varied around 15-20%“. Lastly, the plant gains are high (above 2 for low to mid frequencies) which means they will amplify errors. Indeed this is one of the features of robust feedback: open-loop sensitivity provides closed-loop robustness (see e.g. Csete and Doyle, 2002).

3) Expanding on a couple of discussion points

The authors conclude the Discussion as follows:

"The dynamic synthesis of filters based on current sensory information may be a mechanism that shapes neural controllers on two time scales: for the moment-to-moment task dynamics and over the longer term for the maintenance of behavioral performances."

I'd suggest that the authors expand on this idea a bit; at present it's not very clear to me how they think their simulation results (e.g. that the synthesis of the high-pass controller and low-pass plant yields a robust system) relate to the question of multiple timescales.

We agree that the current study in no way addresses longer time scales of​ adaptation and is only focused on moment-to-moment control. Thus, this discussion point was not well fleshed out, and we decided to eliminate it altogether.

Also I'd encourage the authors to expand on possible biological implementations of their findings, and speculate a bit on how those implementations might be similar or different from those known in other model systems. For instance, the controller described here is able to produce similar results (i.e., tracking the same stimulus albeit with greater variability) with different parameters (i.e., switched case) and is reminiscent of multifunctional neural circuits such as those characterized in the crustacean pyloric system. In addition, the characterization of how feedback reduces variability is similar to how the role of feedback and CPG entrainment is studied in multiple species including cats, stick insects, and crayfish. To be clear I'm not suggesting that the authors review all of these (or even any of them in great depth), but I'm sure readers will wonder how the authors theories relate to ideas from other systems.

We agree this is an interesting connection, and is related to reviewer 1’s comments, which we address in tandem by adding two paragraphs to the end of the Discussion (subsection “Feedback and Variability in Neural Systems”, third and fourth paragraphs). Specifically, the work in stick insects includes perturbations to the mechanical plant (angle of the walking surface) with no changes in the kinematics. This is akin to the suggestion above for new experiments that perturb the animal’s plant.

Reviewer #3:

I've been thinking a lot about this paper and while I argue that it should eventually be published in eLife, I think (for this broad readership journal) it could use some expansion regarding pedagogy on the control aspects. I find it very challenging to decide if I believe that the "simulated swapping" tells me anything fundamental about the system, or if it is just a computational exercise. I think it is not, but it is hard for me to evaluate given all the assumptions going into "gray box" model. What if the black box model isn’t the correct one? In particular, since the authors likely have room in terms of word count, and certainly have space in terms of numbers of figures, I think a bit more explanation of the methods to demystify the story would be helpful.

We have expanded our description of the modeling and computational methods. See response to reviewer #2, issue #1.

Based on the first paragraph of the Introduction, I would have expected to see how the controller changes when fish lose or gain weight (or some manipulation). But I am not convinced by their statement "These changes likely result in variable mismatch between the neural controller and the locomotor plant of individual animals. This mismatch is similar to that induced by swapping plants and controllers across individuals, suggesting that moment-to-moment variability can also be eliminated through sensory feedback." Is this true? Why not do the (obviously harder) experiment of indeed letting the fish eat a lot (or not) and measuring the controller. That way, the authors don't have to do a virtual swap? Even better, why not add weighted "jackets" to the fish as a real "plant" manipulation? I think if they could show this, the paper would become outstanding (instead of "just" very cool as it is now).

We have thought carefully about the types of experiments that we could​ attempt – including feeding, adding neutral buoyancy ballast, fin clips, and virtual reality to modulate feedback. Ultimately we want to make neurophysiological recordings from the neurons that comprise the neural controller in freely swimming fish that are performing tracking while systematically perturbing the locomotor plant.

To address this concern, we have added references and discussion about other species in which these sorts of manipulations have been made. See subsection “Feedback and Variability in Neural Systems”, third and fourth paragraphs.

This phrase in the Abstract "virtually indistinguishable between conspecifics" is a bit overwrought. What does "virtually indistinguishable" mean?

This imprecise statement has been replaced.

https://doi.org/10.7554/eLife.51219.sa2

Article and author information

Author details

  1. Ismail Uyanik

    1. Department of Electrical and Electronics Engineering, Hacettepe University, Ankara, Turkey
    2. Laboratory of Computational Sensing and Robotics, Johns Hopkins University, Baltimore, United States
    3. Department of Biological Sciences, New Jersey Institute of Technology, Newark, United States
    Contribution
    Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology
    For correspondence
    uyanik@ee.hacettepe.edu.tr
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3535-5616
  2. Shahin Sefati

    Department of Mechanical Engineering, Johns Hopkins University, Baltimore, United States
    Contribution
    Conceptualization, Data curation, Software, Formal analysis, Validation, Visualization, Methodology
    Competing interests
    No competing interests declared
  3. Sarah A Stamper

    Department of Mechanical Engineering, Johns Hopkins University, Baltimore, United States
    Contribution
    Conceptualization, Data curation, Formal analysis
    Competing interests
    No competing interests declared
  4. Kyoung-A Cho

    Department of Mechanical Engineering, Johns Hopkins University, Baltimore, United States
    Contribution
    Data curation
    Competing interests
    No competing interests declared
  5. M Mert Ankarali

    Department of Electrical and Electronics Engineering, Middle East Technical University, Ankara, Turkey
    Contribution
    Software, Formal analysis
    Competing interests
    No competing interests declared
  6. Eric S Fortune

    Department of Biological Sciences, New Jersey Institute of Technology, Newark, United States
    Contribution
    Conceptualization, Resources, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Project administration
    Contributed equally with
    Noah J Cowan
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-6447-5425
  7. Noah J Cowan

    1. Laboratory of Computational Sensing and Robotics, Johns Hopkins University, Baltimore, United States
    2. Department of Mechanical Engineering, Johns Hopkins University, Baltimore, United States
    Contribution
    Conceptualization, Resources, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Project administration
    Contributed equally with
    Eric S Fortune
    For correspondence
    ncowan@jhu.edu
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2502-3770

Funding

National Science Foundation (1557895)

  • Noah J Cowan

National Science Foundation (1557858)

  • Eric S Fortune

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

This work was supported by a collaborative National Science Foundation (NSF) Award to Noah J Cowan (1557895) and Eric S Fortune (1557858). Figures and Figure Supplements incorporate illustrations drawn by Eatai Roth. We also thank him for his invaluable feedback on the manuscript.

Ethics

Animal experimentation: All experimental procedures used for this study were reviewed and approved by Johns Hopkins (protocol: FI19A178) and Rutgers (protocol: 999900774) Animal Care and Use committees and followed the guidelines given by the National Research Council and the Society for Neuroscience.

Senior and Reviewing Editor

  1. Ronald L Calabrese, Emory University, United States

Reviewer

  1. Len Maler, University of Ottawa, Canada

Publication history

  1. Received: August 20, 2019
  2. Accepted: January 21, 2020
  3. Accepted Manuscript published: January 23, 2020 (version 1)
  4. Version of Record published: February 25, 2020 (version 2)

Copyright

© 2020, Uyanik et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,550
    Page views
  • 206
    Downloads
  • 1
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Human Biology and Medicine
    2. Neuroscience
    Jianxi Zhu et al.
    Research Article
    1. Neuroscience
    Nathan Weisz et al.
    Research Article Updated