Two highspeed decisions used by animals to catch objects.

(A) Male hoverflies use a rapid open-loop strategy to catch passing females: When sighting a female that moves at speed ‘v’, at sighting distance ‘D’ from the male, then the male rapidly turns so that a constant acceleration ‘a’ will automatically make him arrive at the point of interception at the appropriate time. In this decision, the male takes information only once and uses hardwired circuitry that assumes fixed values for D, v, and a. While this simplifies the complex interception considerably it would make it impossible to intercept targets with other values of D, v, or a (29). (B) Archerfish down aerial prey with shots of water and also use an open-loop strategy to arrive simultaneously with their prey as soon as it hits the water surface. The responding fish, shooters, and bystanders alike, sample visual information on the speed, height and direction of the initial falling motion and turn right towards where ballistically falling prey would later land. These turns are part of a maneuver, called C-start (34,40), that also lends the fish the level of speed that, when kept, make it arrive just in time. Depending on visual contrast, water temperature, height of prey and the distance from the future landing point, fully accurate turns can be initiated in as little as 40 ms (11). In contrast to the conditions in the male hoverfly, the ecological constraints in archerfish do not allow similar simplifying assumptions and archerfish can respond appropriately from a large range of distances and initial orientation and when encountering targets of unusual size or speed (as will be shown below).

When prey is prevented from falling, archerfish turn to the virtual ballistic impact point.

(A) Sketch of an experiment in which archerfish are occasionally challenged with prey that is prevented from falling but that only slides horizontally on a glass plate. As long as such tests are rare, archerfish still elicit turn decisions even though the fly does not fall and will never ever come down. Remarkably the aim of these starts is towards the appropriate ‘virtual’ impact point (VIP). To show this, one uses ballistics, to calculate for each test with a sliding fly where the fly would have landed, given its initial values of motion (initial horizontal speed, direction and height). Using the results accumulated from many such tests shows that, for any combination of the initial motion values, the fish turn toward the expected ballistic landing point. The decision circuit thus maps the combination of initial motion values to a turn towards the expected ballistic landing point. (B) Sketch to show how the accuracy of turns toward the appropriate VIPs can be quantitatively assayed for each of the various combinations of sliding speed, direction, and height. An ecologically relevant measure of the turn’s accuracy is how close the chosen orientation would bring the fish to the VIP, i.e. how close following the chosen initial direction would bring the fish to the point of reward. The sketch also illustrates that this measure allows pooling responses with widely different initial conditions, i.e., wide differences in the initial distance and required turn angle for each VIP. (A, B) This experiment provides the basis of all experiments in this paper. It shows that turns can (i) be elicited by purely horizontal motion, and that (ii) these turns are towards the virtual ballistic impact points, i.e. the points predicted from initial motion, using ballistics.

Critical tests that directly show the suitability of the virtual reality setup that is used in the present paper.

To exclusively work with virtual stimuli, as required in the present study, a system was needed in which the fish would respond to virtual stimuli as in the natural situation. (A) Sketch of the successful setup with motion presented on an LCD screen and feeders operated to deliver food in time at the virtual impact points (VIPs), the points at which ballistically falling prey would impact, given its initial motion (see Fig. 2). (B) Evidence that the decisions are not influenced by the presence and operation of the conspicuous feeders: Same errors to VIPs with feeders present or removed. No rapid turns are elicited when feeders are operated without movement shown on the screen. In tests in which the feeders are not at the position of the VIPs but offset from them (by either 11.2 or 22.3 cm), turns minimized the error to the VIPs but not to the feeder positions. Data are represented as medians, inset recalls how errors were defined (see Figs. 2 and S1). (C-G) Direct comparison of the turn decisions with real and virtual impact points. Same errors (C) to virtual and real impact point, same response latency (D), same kinematics (E, duration of bending phase). (F, G) As with real objects, accurate starts to virtual landing points are also possible across wide ranges of distance (F) and turning angle (G). See Figs. S1, S2 for additional information, Movies S1, S2 for examples showing the turn decision, and how they were evaluated. See text for detailed statistics. ***p<0.001, n.s. not significant.

The highspeed decision is not hardwired to ballistics but can be trained to a new rule.

(A) Sketch of the movement pattern used to introduce new virtual impact points (VIPs) that are still strictly defined by initial motion but systematically differ from the ballistic VIPs. Trajectories were initially straight but – after the time it typically takes the fish to make their decision – then changed their direction as illustrated. Once this new movement pattern was introduced, reward was given exclusively at the corresponding new ‘deflected’ VIPs and no longer at the ballistic VIP. (B) Learning to adjust the turns to initial prey motion requires non-trivial corrections to the previous turning angles (to the ballistic VIPs). These corrections depend on the fish’s initial position and a different map is needed for each set of motion input variables (as illustrated in Fig. S3). (C, D) Median errors of the turns relative to the new (C) and to the old ballistic (D) VIPs in successive training blocks 1 to 9 (each with 60 evaluated turn decisions), with food always delivered at the new VIPs. Colored background highlights when turns were oriented towards the new (C) and when to the old ballistic (D) VIPs. (E, F) Cumulative density functions (CDFs) at the blocks indicated by colored circles in (C, D) to show how the systematic error to the new VIPs systematically decreased (E) whereas errors to the ballistic landing points increased (F). (G) Errors made in several critical tests – from left to right: the deflection was visible and feeders were present at the new VIPs, just as during training (n=240), interspersed tests with only the initial straight trajectory shown (i.e. prey vanished before the deflection would occur) (n=68), interspersed tests (rate: 1 in 8) with also only the initial straight trajectory shown but direction of motion offset from the direction of the feeders by either 15° (n=197) or by 72° (n=145) to test if fish turned to the new VIP or to the feeder positions. (H, I) After training the fish were able to again respond appropriately over a wide range of distances (H) and required angles of turn (I). (J, K) Absence of speed-accuracy tradeoff of the re-programmed highspeed decisions. Plot of magnitude of error in aim versus response latency in the critical tests shown in (G) with only initial pre-deflection prey trajectory and offset of either 15° (J) or 72° (K). Note absence of significant correlation between error magnitude and response latency (p>0.3; R2<0.005). Data are represented as median in (C, D, G). ***p<0.001, **p<0.01, n.s. not significant.

The new rule is represented in a way that allows immediate generalization.

(A) Idea for generalization tests that are possible because training to the ‘deflected’ trajectories (Fig. 4) was only for one level of target height. When faced with a larger than training height, fish would only be able to turn to the corresponding impact point P if they had acquired a more general rule of how to connect input (height, direction, speed) to the rewarding turn. Had they only exchanged those input-output experiences that were no longer rewarded, then they should still use the previously rewarded input-output relations and turn to the ballistic impact point (red, ‘’ballistic’’; see text). Had they substituted the old input-output relation for all height levels with the trained new ones, then they should turn to point ‘’substitute’’ (blue, see text) that would be appropriate for the training height. (B, C) When the fish were shown the deflected trajectories at larger (B) or lower (C) height than experienced during training, then already their first turns minimized the errors to the VIPs that are appropriate for the new rule at the untrained new height level with no indication that errors would initially be large and then decrease. (D, E) Closer analysis, using cumulative density functions (CDFs), for the errors made to the predicted points based on hypotheses introduced in (A), both for the larger (D) and for the lower (E) height (both n=30). (F) Evidence that learning the new rule had not used prior assumptions on target size. Errors of the turns made in the first 30 tests in which absolute target size was more than three-fold (13 mm) than the target size (4 mm) encountered throughout the training to the new rule. (G) CDFs of the first 30 tests with a low target speed (1.425 m/s) that the fish never encountered during training to the new rule. Again, the fish immediately chose turns to minimize the error to the predicted VIP based on the new rule but not the error to the points calculated based on predictions ‘’ballistic’’ or ‘’substitute’’.

The highspeed decision can conditionally use two different rules of how to connect input and output.

(A) After successful training to infer the new (deflected) virtual impact points (VIPs), fish were randomly shown either the disks with the deflected trajectories (reward at deflected VIPs) or new non-disk objects that moved on straight trajectories and were rewarded at the corresponding ballistic VIPs. (B, C) Error in the turns as determined in interspersed tests that only showed the straight initial movement both for non-disk objects (triangle, B) and for the familiar disk objects (C). Objects were chosen randomly. ‘Pre’ (filled symbols) denotes baseline before fish were exposed to a total of 500 presentations of the non-disk objects with reward at their ballistic VIPs. Errors were determined in subsequent test phases. As the fish changed their turn decision to minimize the error to the non-disk VIPs (B) they continued to aim to the deflected VIPs whenever they encountered moving disks (C) showing that they had not reversed to generally using ballistics again. (D-F) Interspersed tests that only showed the same initial linear trajectory but with either a disk (n=60) or a triangle moving (n=187). Graphs on right side to illustrate choice situation as seen from above: For the same set of input variables (speed, height, direction) the fish must turn to the ballistic VIP when a triangle is moving, but to the deflected VIP when a circle is moving. Detailed analysis shows that turns were appropriate to the correct VIP with no difference in error (D), latency (E) or kinematics (bending duration) (F). (G) Individual fish were able to select the appropriate turn to the deflected VIPs when encountering moving disks and to the ballistic VIP when encountering triangles. Aims were equally appropriate and did not differ in variability. Data are represented as medians (B, C, G). n.s. not significant.

The clue that signals which rule must be used is not required before prey motion starts.

In this series of experiments one of two differently shaped types of moving prey are shown, as in Fig. 6. Either a triangle that required the fish to turn to the appropriate ballistic VIPs, or a disk, that required the fish to turn not according to ballistics but based on the deflected rule (see Fig. 4). One of the two objects were randomly selected. In one type of experiments (’2s before’), the decision circuit was given time to adjust to the rule that was needed by displaying the object in stationary position for 2s prior to motion onset. In the other type (’only during decision phase’) no prior information was given. Instead, a neutral symbol (a cross) was shown 2s before prey motion and then the chosen object (disk or triangle) appeared and immediately was moving. In all tests, only the initial straight movement was shown (i.e., no deflection was visible in the moving disks). Errors were the same, regardless of whether object identity was revealed before or only during the very brief decision time (see Movies S6, S7). Data are represented as medians. n.s. not significant.

The accuracy of the archerfish turn decisions can be precisely quantified.

(A) Illustration of the so-called C-start that produces the archerfish’s turn: Initial bending into the shape of a letter ‘C’ (so-called stage 1 or bending phase) and subsequent straightening phase (so-called stage 2). The maneuver takes place directly below the water surface and turns the fish from its previous (before) to its appropriate new (after) orientation, the later landing point of real falling prey, or – as introduced in Fig. 2 – the virtual landing point, calculated from the prey’s initial motion, based on ballistics. The rapid maneuver is monitored using digital highspeed video (500 frames per second). (B) As soon as the fish is straightened (i.e., at the end of stage 2 of the C-start) and before it actually starts swimming, the aim of the fish relative to the real or virtual impact point is determined to quantify the accuracy of the turn decisions. For this, the orientation of the stiff front end of the fish is determined (using the bases of the two pectoral fins and the tip of the snout as markers). A line is then considered along this orientation, and the minimum distance from the later landing point (real or calculated virtual) is determined. This way it is possible to measure how close continuing the initial aim would lead the fish to the impact point. If the line determined by the orientation at stage 2 cuts the projection of the (real or virtual) trajectory before the impact point, then the error is defined negative, otherwise positive. Note that the definition allows a direct comparison of the errors made across all possible initial distances and orientations of the responding fish relative to the various prey trajectories.

The apparatus that passed all critical tests (Fig. 3) for studying the archerfish turn decisions under virtual reality (VR) conditions.

Establishing and critically testing a VR setup was required in this study, because we needed to stop rewarding the fish at the points predicted from ballistics and initial prey movement. Instead, we wanted to introduce a different rule that predicts the point of reward from initial motion. Our aim was to explore if the turn decisions could be reprogrammed. (A) Photo of one of the feeders used to reward the fish in time at a virtual impact point. An electromagnet moves a slider at the appropriate time to allow the passage of a piece of food within the tube. During training and in the majority of tests, the feeders are at the position of a virtual impact point. In the crucial critical tests, however, the feeders are offset from the predicted reward points. (B) Top view of experimental tank with 4 feeders and a rectangular LCD screen, on which motion is shown within a circular area in the center. As shown in the critical experiments reported in Fig. 3 the setup is fully suitable to produce turn decision that are not statistically different in any aspect from those elicited using real falling prey. (C) View from below through the transparent bottom of the tank with the LCD screen and background above. This is the view in all movies that accompany the paper. (D) Side view of the tank (sized 1m x 1m x 0.6 m) with the highspeed camera (500 frames per second) below.

Learning the new rule (Fig. 4) requires complex position-dependent corrections, that differ with every different input constellation.

The apparently simple new rule to connect the turn to initial prey motion (Fig. 4A) requires a complex pattern of corrections the fish must make relative its previous turns to the ballistic virtual impact points. To illustrate this, graphs display the corrections the fish must make to its prior ballistic choices for two different directions of initial prey movement. (A) Definition of terms used. Sketch of top view of a responding fish (black circle and line denote head and body axis, respectively) that turns in response to either the motion shown in red or in blue on the screen above. The respective virtual (deflected) impact points of the new relation used in Fig. 4A are shown in red and in blue. The corresponding ballistic virtual impact points are shown in grey, and the turns needed to align the fish towards them are also indicated in grey. ‘Correction’ is the angle that would have to be added to these grey turns. Note that for two different directions of prey movement (red versus blue) a fish located at the same spot would need to apply different corrections. (B) shows the pattern of corrections for every position in the tank (sized 100 cm x 100 cm) for just one direction of initial prey movement, with the (’deflected’) virtual impact point at x=89 cm, y=68 cm. Corrections > +18 deg and < -18 deg are assigned the values +18 and -18 deg, respectively. (C) Changing the direction of initial prey movement requires a different map of corrections (virtual impact point at x=11 cm, y=62 cm). (D) Analysis of the difference in the maps of corrections (of B, C) for two different initial directions of prey movement. This shows that even only two possible directions of initial prey movement (with speed and height constant) would require an extensive look-up table with stored corrections for every position and condition of target movement. Trained fish that randomly faced many different types of initial motion were able to respond appropriately to each of them from any location, without an increase in response latency (see Fig. 4) and could even generalize to untrained conditions (see Fig. 5).