High-speed decisions used by animals to catch objects.

(A) Male hoverflies use a rapid open loop-strategy to catch passing females: When sighting a female moving at speed ‘v’ at fixed distance ‘D’ the male rapidly turns in a direction in which it will, always accelerating at a fixed rate ‘a’, be led to a point at which it can intercept the female. Here, information is taken only once. To make this decision, the male uses hardwired circuitry that assumes fixed values for D, v and a, which allows to simplify the complex problem considerably (6). (B) To secure downed prey hunting archerfish also use an open-loop strategy: Based only on visual information sampled after prey starts moving the fish turn to the later impact point and then (corresponding points in time indicated by an asterisk) take off at a speed that makes them arrive at full speed very slightly after their prey. The turns are produced in a so-called C-start maneuver and can be initiated in as little as 40 ms. The hunting context does not allow similar simplifications as in the hoverfly and fish can respond from a large range of distances and for any orientation. They are also, for instance, not limited to targets of an expected size or speed. (C) When prey is made to slide purely horizontally on a glass plate – without actually falling downwards – then the fish still elicit turn decisions. These are not aimed at the fly’s actual position and also not in the direction of the fly’s movement, but at the ‘virtual’ impact point (VIP), where ballistically falling prey would have landed given its initial speed, direction and height.

Establishing a technique that allows to replace ballistics.

To define arbitrary relations between input and the rewarded turns, as required in this study, we first established and calibrated a virtual reality approach. (A) Sketch of setup with motion presented on a screen and feeders operated to deliver food in time at the VIPs. Evidence that the decisions are not influenced by the feeders: Same errors to VIPs with feeders present or absent. Inset illustrates how error of turn decision is defined (for details see Fig. S1). No rapid turns are elicited when feeders are operated without movement on screen. When motion directions are chosen to create a conflict between turning either to the VIPs or to the feeder positions (offset either 11.2 or 22.3 cm), then turns minimized the error to the VIPs but not to the feeder positions. Data are represented as medians. (C-G) Comparing the nature of the turn decisions with real and virtual impact points. Same errors (C) to virtual and real impact point, same latency (D), same kinematics (E, duration of bending phase). (F, G) Under virtual conditions accurate starts are also possible across wide ranges of distance (F) and turning angle (G). See Figs. S1, S2, Movies S1, S2 and text for detailed statistics. ***p<0.001, n.s. not significant.

The high-speed decision learns a new rule of how to connect input and output.

(A) Sketch of how we employed virtual reality to challenge the fish with a new rule of how they must turn based on initial target motion. For all objects shown, the trajectories were always straight during the time it takes the fish to make its decision but then were deflected as illustrated in the inset. Reward is then given at the corresponding new ‘deflected’ impact point but no longer at the ballistic impact point (that ballistics would predict from the initial movement). (B) Despite its apparent simplicity, learning to adjust the turns to this type of motion would require non-trivial corrections to the previous (ballistic) turning angles. These corrections depend on the fish’s initial position. Furthermore, a different such map would be needed for each set of input variables (see Fig. S3). (C, D) Median errors of the turns relative to the new and to the old ballistic (D) VIPs in successive training stages 1 to 9 (each with 60 evaluated turn decisions), with food always delivered at the new VIPs. Colored background highlights when turns were oriented towards the new (C) or to the old ballistic (D) VIPs. (E, F) Cumulative density functions (CDFs) at the stages indicated by colored circles in (C, D) to show how systematic error to the new landing point decreased (E) whereas errors to the ballistic landing points increased (F). (G) Errors made in several critical tests – from left to right: Deflection of trajectory visible and feeders present at the new VIPs (n=240), interspersed tests with only the initial straight trajectory (but not the later deflection) shown (n=68), interspersed tests (rate: 1 in 8) with also only the initial straight trajectory shown but direction of motion offset from the direction of the feeders by either 15° (n=197) or 72° (n=145) to create a clear conflict between turning to the feeder position or to the VIP derived according to the new rule. These controls show that the fish had indeed learned to base their turn decision on the initial movement. (H, I) After training the fish were able to again respond appropriately over a wide range of distances (H) and required angles of turn (I). Data are represented as median (C, D, G). ***p<0.001, **p<0.01, n.s. not significant.

The new rule is represented in a way that allows immediate generalization.

(A) Idea for generalization tests that are possible because training to the ‘deflected’ trajectories (Fig. 3) was only for one level of target height. When faced with a larger than training height, fish would only be able to turn to the corresponding impact point P if they had acquired a more general rule of how to connect input (height, direction, speed) to the rewarding turn. Had they only substituted those input-output experiences that were no longer rewarded, then they should still use the previously rewarded input-output relations and turn to the ballistic impact point A (red, see text). Had they substituted the old input-output relation for all height levels with the trained new ones, then they should turn to point B (blue, see text) that would be appropriate for the training height. (B, C) When the fish were shown the deflected trajectories at larger (B) or lower (C) height than experienced during training, then already their first turns minimized the errors to the VIPs that are appropriate for the new rule at the untrained new height level with no indication that errors would initially be large and then decrease. (D, E) Closer analysis, using cumulative density functions (CDFs), for the errors made to the predicted points based on hypotheses introduced in (A), both for the larger (D) and for the lower (E) height (both n=30). (F) Evidence that learning the new rule had not used prior assumptions on target size. Errors of the turns made in the first 30 tests in which absolute target size was more than three-fold (13 mm) than the target size (4 mm) encountered throughout the training to the new rule. (G) CDFs of the first 30 tests with a low target speed (1.425 m/s) that the fish never encountered during training to the new rule. Again, the fish immediately chose turns to minimize the error to the predicted VIP based on the new rule but not the error to the points predicted by hypotheses A and B.

The high-speed decision can conditionally use two different rules of how to connect input and output.

(A) After successful training to infer the new (deflected) virtual impact points (VIPs), fish were randomly shown either the disks with the deflected trajectories (reward at deflected VIPs) or new non-disk objects that moved on straight trajectories and were rewarded at the corresponding ballistic VIPs. (B, C) Error in the turns as determined in interspersed tests that only showed the straight initial movement both for non-disk objects (triangle, B) and for the familiar disk objects (C). Objects were chosen randomly. ‘Pre’ (filled symbols) denotes baseline before fish were exposed to a total of 500 presentations of the non-disk objects with reward at their ballistic VIPs. Errors were determined in subsequent test phases. As the fish changed their turn decision to minimize the error to the non-disk VIPs (B) they continued to aim to the deflected VIPs whenever they encountered moving disks (C) showing that they had not reversed to generally using ballistics again. (D-F) Interspersed tests that only showed the same initial linear trajectory but with either a disk (n=60) or a triangle moving (n=187). Graphs on right side to illustrate choice situation as seen from above: For the same set of input variables (speed, height, direction) the fish must turn to the ballistic VIP when a triangle is moving, but to the deflected VIP when a circle is moving. Detailed analysis shows that turns were appropriate to the correct VIP with no difference in error (D), latency (E) or kinematics (bending duration) (F). (G) Individual fish were able to select the appropriate turn to the deflected VIPs when encountering moving disks and to the ballistic VIP when encountering triangles. Aims were equally appropriate and did not differ in variability. Data are represented as medians (B, C, G). n.s. not significant.

The clue indicating which rule to use needs not to be known before target movement starts.

Schematic illustration of the idea to examine whether shape information was required before movement information, giving the decision circuit some time to adjust which rule was subsequently to be used. Either the randomly chosen object or a neutral symbol (cross) was shown for 2s and then the chosen object (disk or triangle) started moving. In all tests, only initial straight movement was shown. Errors were the same, regardless of whether object identity was revealed before or only during the very brief decision time (see Movies S6, S7). Data are represented as medians. n.s. not significant.

Analyzing the accuracy of the turn decisions.

Related to Figs 1 and 2. (A) Illustration of the two phases of the C-start that produced the turn: Initial bending into the shape of a letter ‘C’ (so-called stage 1 or bending phase) and subsequent straightening phase (so-called stage 2) at the end of which the aim of the fish is measured and the accuracy of the turn is evaluated. The maneuver takes place directly below the water surface and turns the fish from its previous (before) to its appropriate new (after) orientation, typically to the later landing point of falling prey. (B) Quantifying the accuracy of the C-start decision: As soon as the fish is straightened (end of stage 2 of the C-start), the orientation of its stiff front end is determined (using the bases of the two pectoral fins and the tip of the snout as markers). A line is then considered along this orientation and the minimum distance from the later landing point (real or calculated virtual) is determined. In case the line determined by the orientation at stage 2 cuts the projection of the (real or virtual) trajectory before the landing point, the error is defined negative, and positive otherwise. Note that this definition is needed to allow a direct comparison of the errors made across all possible initial distances and orientations of the responding fish relative to the various prey trajectories.

Apparatus for changing the rule that connects data on initial target movement to the turn the fish must make.

Related to Fig. 2. (A) Photo of feeder with slider that can be moved at the appropriate time by an electromagnet to allow the passage of a piece of food within the tube. (B) Top view of experimental tank with 4 feeders and LCD screen. (C) View from below through transparent bottom of the tank with LCD screen and background. (D) Side view of the tank (sized 1m x 1m x 0.6 m) with highspeed camera below.

Learning the new rule requires position-dependent corrections to the ballistic turns with the pattern of corrections different for each input constellation.

Related to Fig. 3. It is important to understand that the apparently simple motion model (Fig. 3A) translates to rather complex corrections of the turns the fish must make. To illustrate this, we show the corrections the fish must make to its prior ballistic choices for only two different directions of initial target movement. (A) Definition of terms used. Sketch of top view of a responding fish that turns in response to either the motion shown in red or in blue on the screen above. The respective virtual (deflected) impact points are shown in red and in blue. The corresponding ballistic virtual impact points are shown in grey. ‘Correction’ is the angle that would have to be added to the turns toward the ballistic impact point. For two different directions of target movement (red versus blue) a fish located at the same spot would need to apply a different correction as indicated. shows the corrections required at each position in the tank (sized 100 cm x 100 cm) to ensure that turns at the rewarded new ‘deflected’ impact points. Note that this map of corrections applies for one direction of target movement only, with the (’deflected’) virtual impact point at x=89 cm, y=68 cm. Corrections > +18 deg and < -18 deg are assigned the values +18 and -18 deg, respectively. (C) Simply changing the direction of initial target motion requires a different map of corrections. Shown are the corrections the fish would have to apply, at each position, in response to a target that moves in a different initial direction, leading to another virtual impact point (x=11 cm, y=62 cm. (D) Quantitative analysis of the difference in the maps of corrections (B, C) that are required for the two different initial directions of target movement. This illustrates that accounting only for two possible types of initial prey movement (speed and height left constant) would require an extensive look-up table with stored corrections for every position and condition of target movement. Trained fish that randomly faced many different types of initial motion were able to respond appropriately to each of them from any location, without an increase in delay (see Fig. 3) and could apply the rule the extracted to generalize to untrained conditions (see Fig. 4).