Insects visually solve an array of complex problems including the detection and tracking of fast moving prey (Wiederman et al., 2017), long-distance navigation (Wehner, 2020), and even three-dimensional (3D) depth estimation (Nityananda et al., 2018). These capabilities are realised using a sensory apparatus that is fundamentally different from those of mammals. Therefore revealing the functional properties of the insect visual system offers insights for biologists as well as inspiration for engineers looking to develop novel artificial imaging systems (Land and Fernald, 1992; Land, 1997; Arendt, 2003; Song et al., 2013).

Arthropods possess two primary visual sensors known as compound eyes. Each eye is constructed from a patchwork of self-contained light-sensing structures known as ommatidia, each featuring a lens, a light guide, and a cluster of photosensitive cells (Figure 1a). Ommatidia are physically interlocked with their neighbours, together forming a bulbous outer structure (the compound eye itself) that can vary in size, shape, and curvature, offering a range of adaptations for particular tasks and environments (Land and Nilsson, 2002; Figure 1b). The properties of the lens and photo-receptors are fixed for individual ommatidia but can vary across regions of an individual’s eye (Meyer and Labhart, 1993) as well as between individuals of different castes (Collett and Land, 1975) and species (Land, 1989). This arrangement of independent, interlocked light sensing elements and lenses differs greatly from the mammalian system that utilises a single lens to project a high-resolution image onto a retina.

Figure 1

Download asset Open asset

Creation of compound eye models (CEMs), in both hardware or software, is a well-established mechanism to explore the information provided by compound eyes and assess their impact on behaviour. Insights derived from this methodology include demonstration of the benefits of a wide field of view (FOV) and low resolution for visual navigation (Zeil et al., 2003; Vardy and Moller, 2005; Mangan and Webb, 2009; Wystrach et al., 2016), and the role played by non-visible light sensing for place recognition (Möller, 2002; Stone et al., 2006; Differt and Möller, 2016) and direction sensing (Lambrinos et al., 1997; Gkanias et al., 2019). Yet, simulated CEMs tend to suffer from a common design shortcoming that limits their ability to accurately replicate insect vision. Specifically, despite differences in the sampling techniques used (e.g. see Mangan and Webb, 2009; Baddeley et al., 2012 for custom sampling approaches, Neumann, 2002; Basten and Mallot, 2010 for rendering based cubemapping CEMs, and Polster et al., 2018 for ray-casting methods), all contemporary CEMs sample from a single viewpoint. In contrast, the distributed arrangement of ommatidia on distinct 3D eye surfaces provides insects with a multi-viewpoint system that generates different information for different eye shapes.

To facilitate exploration of such features, e.g., the placement of ommatidia on arbitrary surfaces, an ideal rendering system would allow light to be sampled from different 3D locations through individually configured ommatidia replicating the complex structure of real compound eyes. High-fidelity compound-vision rendering engines with some of these features (though notably slower than real time) were developed previously (Giger, 1996; Collins, 1998; Polster et al., 2018) but were not widely adopted. Difficulties arising as a result of the computational complexity (and so execution time) of CEMs are diminishing as dedicated ray-casting hardware emerges that allows for the capture of visual data from multiple locations in parallel at high speed (e.g. Nvidia [Santa Clara, California, United States] RTX series GPUs [Purcell et al., 2005; Burgess, 2020]). Such systems present an ideal tool tferreo replicate insect vision in unprecedented accuracy at the speeds needed for effective exploration of the compound eye design space itself. As outlined in Millward et al., 2020, a next-generation insect eye renderer should:

1. Allow for the arrangement of an arbitrary number of ommatidia at arbitrary 3D points.

2. Allow for the configuration of individual ommatidial properties (e.g. lens acceptance angle).

3. Perform beyond real time to allow exploration of the design space.

This paper presents a ray-casting-based renderer, CompoundRay, that leverages modern hardware-accelerated ray-tracing (RT) graphics pipelines and fulfils all three of these criteria, allowing researchers in the fields of compound vision and biorobotics to quickly explore the impact varying eye designs have on an autonomous agent’s ability to perceive the world.

The analysis that follows first assesses the performance of CompoundRay with respect to the three criteria defined in the ‘Introduction’, before showing its utility in an example experiment. Performance is benchmarked in two 3D environments: a lab environment inspired by those used in insect cognitive experiments (Ofstad et al., 2011); and a 3D scan of a real-world natural environment covering an area of a number of square kilometers. The natural environment was constructed using structure-from-motion photogrammetry using photos captured from a drone over rural Harpenden, UK (3D model subject to upcoming publication, available via contact of Dr. Joe Woodgate, Queen Mary University of London).

Criterion 1: arbitrary arrangements of ommatidia

Criterion 1 states that the renderer must support arbitrary arrangements of ommatidia within 3D space. Figure 6 shows two different environments rendered using three differing eye designs, from a simple spherical model with spherical uniformly distributed ommatidia that more closely aligns with current compound eye rendering methods, to a hypothetical arbitrary arrangement similar to that of the now long-extinct Erbenochile erbeni. While Figure 6a&b is still possible to simulate using traditional projection-based methods due to their ommatidial axes converging onto one central spot, 6 c demonstrates an eye with multiple unique focal points across an irregular surface which must be rendered from the viewpoint of each ommatidium independently.

Figure 6

Download asset Open asset

Criterion 2: inhomogeneous ommatidial properties

Criterion 2 states that it should be possible to specify heterogeneous ommatidial optical properties across an eye, such as the enlarged facets as found in robberflies (Wardill et al., 2017). As demonstrated in Figure 7, the renderer achieves this by allowing for shaping of the acceptance cone of each ommatidium. In doing so, Figure 7 also demonstrates the importance of heterogeneous ommatidial configurations.

Figure 7

Download asset Open asset

Figure 7a&b shows a non-uniform eye design with homogeneous, globally identical, ommatidial acceptance angles. It can be seen that in Figure 7b, where the acceptance angle is lower, aliasing occurs as objects are only partially observed through any given ommatidium, causing blind spots to form, introducing noise into the image. Conversely, in Figure 7a, where the acceptance angle is large, the image becomes blurry, with each ommatidium oversampling portions of its neighbour’s cone of vision. Figure 7c, however, demonstrates a heterogeneous ommatidial distribution, in which the acceptance angles toward the dorsal and ventral regions of the eye are larger compared with the angles of those around the eye’s horizontal acute zone. This mirrors what is seen in nature, where ommatidial FOV is seen to vary in order to encompass the entirety of an insect’s spherical visual field (Land and Nilsson, 2002). As can be seen in the generated image, this is important as it appropriately avoids creating blind-spots in the dorsal and ventral regions while also avoiding oversampling (blurring) in the horizontally acute region, resulting in a much clearer picture of the observed environment (before the visual data is passed into any insect visual processing neural circuits) while also minimising the required number of ommatidia. The data presented in Figures 6 and 7 clearly demonstrates differences in visual information provided by different ommatidial layouts, reinforcing the need for realistic eye modelling methods.

Criterion 3: speed

Minimum sample count

Criterion 3 states that to rapidly explore the information content of various eye designs any such rendering system should be able to perform at real time or faster. However, speed of the system is dependent on the number of rays required to render a given scene from a given eye; higher total ray counts will require more compute power. As discussed in the section ‘Modelling individual ommatidia’, per-ommatidial sampling choice can have a dramatic impact on the image generated, meaning that lower ray counts – while increasing speed – will decrease the accuracy of the image produced. As a result, it is important to be able to define some minimum number of samples required for any given eye in any given environment in a reliable and repeatable manner. Previous works Polster et al., 2018 have attempted to establish a baseline number of samples required per ommatidium before additional samples become redundant only via qualitative visual analysis of the results produced. Here we perform quantitative analysis of images produced by the renderer in order to more definitively define this baseline (Figure 8).

Figure 8

Download asset Open asset

Unlike previous works, the temporally stochastic nature of CompoundRays’ sampling approach can be used to aid the measurement of impact that sampling rate has on the final image. By measuring the per-ommatidium spread (here, standard deviation) of the Euclidean distance of the received colour as plotted in RGB colour space over a range of frames captured from a static compound camera, the precision of the light sampling function approximation can be measured. To control for the varying FOV of each ommatidium (which impacts the number of samples required, as larger FOVs require more sampling rays to accurately capture the ommatidium’s sampling domain), we normalize the measure of spread recorded at each ommatidium against its degree of coverage, in steradians, expressing the solid angle subtended by the ommatidium’s FOV at the eye’s centre. That is, a steradian measures the solid angle within a sphere, in the same same way that a radian measures 2D angles within a circle.

Figure 8 shows this measure of variance within the compound eye across a range of sample-rays-per-ommatidium, from 1 to 700. As the number of samples increases, the variance decreases logarithmically. We propose iteratively increasing samples until the maximum variance falls below a defined threshold value. Here, we use a threshold of 1%, meaning that the maximum expected standard deviation of the most deviant ommatidium on an eye (normalised to one steradian) should be no greater than 1% of the maximum difference between two received colours, here defined as the length of vector [255, 255, 255]. This method allows for standardised, eye-independent configuration of per-ommatidial sampling rate.

However, the required number of samples per eye is highly positively correlated with the visual frequency and intensity of the simulated environment at any given point, with locations exhibiting high contrast seen from a distance requiring higher sampling rates in order to properly resolve. This can be seen in the skyline in Figure 8, which remains dominant throughout the majority of the sampling rates, consistently forming the source of the highest spread. Figure 9 shows how the perceived variance changes over two example environments. The varying environment presents an obstacle in our attempts to standardise sampling rates, as deriving a suitable minimum sampling rate at one point in an environment – e.g., where the visual frequency was, on average, low – could lead to aliasing occurring in more visually complex areas of the same environment, potentially introducing bias for or against certain areas if analysis on the generated data were sensitive to the variance introduced.

Figure 9

Download asset Open asset

Ultimately, the varying visual complexity poses a challenge. In an ideal scenario maximum per-ommatidium frame-to-frame variation caused by visual complexity should be a controllable factor. In the examples presented in this paper, we worked to minimise this variation to a point at which the most variable ommatidial response had a standard deviation of 1% or less of the total maximal difference in colour. One way of achieving this would be to keep a running measure of the standard deviation of each ommatidium and dynamically adapt sampling rates to ensure that no ommatidium had such a high sampling spread (adaptive antialiasing). In this work, however, we define a minimum sampling count for a given environment by performing a search for that environment’s point of highest visual complexity (indicated by highest per-ommatidium deviation) – with this location found, sampling rate can be increased until deviation decreases to acceptable limits (in this case, within 1%). Figure 9 shows plots of the relative variation in standard deviation over the lab and (partial) natural environments.

Performance

Figure 10 shows the rendering speed, in frames per second, against the number of sample rays being emitted in each frame when running using a selection of high-end consumer-grade graphics cards and our two sample environments. Marked on the graph are the total per-frame sample counts required by common insect eyes, chosen to encourage a maximum per-ommatidium standard deviation of 1%.

Figure 10

Download asset Open asset

As can be seen, performance improves significantly when the renderer is able to utilise the next-generation RT hardware available in the newer RTX series of NVidia graphics cards, consistently outperforming older cards in the larger natural environment and outperforming in the smaller lab environment when using more than 30,000 rays. Counterintuitively, the previous generation of NVidia graphics cards was found to outperform the RTX series in the (relatively small) lab environment at lower ray counts (below 30,000). This appears to be due to static overhead involved with the use of the RTX series’ RT cores, which may also be the cause of the small positive trend seen between 1000 and 8000 samples when using the 2080Ti. Another potentially counterintuitive result that can be seen here is the higher performance of the dragonfly eye in the natural (high polygon) environment versus in the low-polygon laboratory environment. This is because the laboratory environment produces significantly higher visual contrast (emphasised in Figure 9) due to its stark black/white colour scheme, resulting in a requirement of almost double the number of rays per steradian to target 1% standard deviation, resulting in higher ray counts overall, particularly in models with a high number of ommatidia.

Example experiment: Apis mellifera visual field comparison

Leveraging the speed of CompoundRay, an example data-driven comparative analysis was performed between a realistic eye model and two simplified derivatives. We note that the data outcomes should be considered as pilot data, with the focus being on proving the tool and its usability. As laid out in the section ‘Example inter-eye comparison method’, this experiment used an MLP network to estimate the relative position of a 2 mm sphere within a 50 mm³ test area using only the visual information provided from each of the eye models, with neural network performance acting as a measure of eye configuration utility. The models were split into the ‘real’ eye model (Figure 5i), an eye pair based directly on the measurements of Baird and Taylor, 2017; the ‘split’ eye model (Figure 5ii), an eye pair retaining per-ommatidial headings (ommatidial axial directions), but sampled from the centre points of each eye (as per approaches common in insect binocular vision study); and finally the ‘single’ eye model, an eye pair sampled from a single point between the two eyes (Figure 5iii). We hypothesise that the ‘real’ eye design will outperform the ‘split’ and ‘single’ eye models by increasing margins, as for each of the latter two designs the data relating to the surface shape and relative positions of the eyes is increasingly reduced.

In terms of total error over the validation set, we observed that – as expected – the ‘real’ eye design appears to offer a higher utility (lower total error when trying to locate the relative position of the sphere), ultimately decreasing its network’s validation error quicker and to a lower minimum after 100 epochs when compared to both the ‘split’ and ‘single’ eye designs (Figure 11a). Interestingly, despite an initial slow in learning rate, the ‘single’ eye design eventually achieved a similar utility score on the validation set than that of the ‘split’ eye design.

Figure 11

Download asset Open asset

To qualitatively assess the performance of each eye across the entire 50 mm³ sampling cube, a further 1,000,000 samples were taken at regular intervals forming an error volume for each eye design by measuring the error between the trained neural network’s relative position and the real relative position (the error volume for the ‘real’ eye is pictured in Figure 5c). In this case, to demonstrate versatility, CompoundRay was used in real time to generate these views, rather than simply being used in an offline manner to generate these benchmarking datasets.

The most obvious region of high error is that outside of the insect’s FOV. As these eyes were designed with uniform average acceptance angles, this is larger than might be expected of a real bee eye. Comparing the ‘real’ eye to its ‘single’ and ‘split’ counterparts by means of comparing the absolute difference between their error volumes (creating the difference volumes shown from top-down in Figure 11b&c), patterns emerge. Seemingly, the most significant region of difference between the ‘real’ eye model and the ‘single’ eye model can be seen around the edges of the eye’s field of vision. As this difference follows the entirety of the edge of the eye’s field of vision, this appears to be likely due to parallax – the ‘real’ eye samples from two points, allowing it slightly extended range of vision compared to that of the ‘single’ eye model, resulting in slightly different ranges of FOV between the two, with this difference increasing the further from the eye the point is. In the ‘real’ vs ‘split’ comparison image, however, there are two regions of higher error (highlighted in red circles in Figure 11c) toward the forward-facing regions of the eye’s vision. These indicate a region of improved visual information when using a full eye surface as compared to simple two-point sampling of the environment.

This paper has introduced a new compound eye perspective rendering software, CompoundRay, that addresses the need for a fast, compound eye perspective rendering pipeline (Stürzl et al., 2015; Taylor and Baird, 2017; Millward et al., 2020). In particular, the tool supports arbitrary instantiation of heterogeneous ommatidia at any place in space and can perform rendering tasks at speeds in the order of thousands of frames per second using consumer-grade graphics hardware. We have demonstrated the utility afforded by the software and highlighted the importance of using higher-fidelity compound eye perspective rendering systems by demonstrating the visual differences introduced by altering these traits. It has also set a grounding for reasonable use of the software to ensure reproducibility and reduce biasing introduced as a result of the variety present in simulated environments and compound eye designs while warning of potential dangers that might arise during data analysis. With the introduction of the CompoundRay rendering library, it is possible to simulate in a timely manner the visual experience of structurally diverse compound eyes in complex environments, digitally captured or otherwise manually constructed. CompoundRay is provided as an open-sourced library (https://github.com/BrainsOnBoard/compound-ray).

This work lays the technical foundations for future research into elements of compound eye design that have previously been given little consideration. In particular, CompoundRay can act as the previously missing step to integrate recently catalogued real eye designs (Baird and Taylor, 2017; Bagheri et al., 2020) into mapped insect environments (Stürzl et al., 2015; Risse et al., 2018) to explore the information provided to insects in their natural surroundings. Similarly, the ability to configure ommatidial properties individually and with respect to eye surface shape enables new investigations into the benefits of asymmetric compound eyes such as those found in robber flies (Wardill et al., 2017) and crabs (Zeil et al., 1986). Furthermore, we see opportunities to model the recently reported microsaccadic sampling in fruit flies (Juusola et al., 2017), as well as aid with similar vision-centric neurological studies (Viollet, 2014; Wystrach et al., 2016; Kemppainen et al., 2021).

Our example study demonstrated the utility of CompoundRay for this strand of research. Beyond its ability to replicate insect vision in unprecedented detail, its image generation speeds facilitate use of contemporary data-driven methods to explore the eye design space. For example, we used over 3.3 million training images, which CompoundRay rendered in only 2 hr. The insights gained from more thorough analysis of the insect visual perspective—and its design relative to visual feature extraction—will help guide the development of artificial visual systems by considering not only visual post-processing steps but also the intrinsic structure and design of the image retrieval system itself. CompoundRay is explicitly designed to be highly configurable to allow members of the research community to pursue their specific research questions.

In our example study, we compared eye designs using an MLP network as a utility function, bypassing the need to directly compare the images generated by two differing eye designs. This was done as the task of directly comparing two dissimilar eye designs – either analytically using a comparison program or qualitatively via human observation – is non-trivial. Compound eyes are inherently different from our own, and the CompoundRay rendering system allows for large-scale data collection on arbitrary compound surfaces. We note that with this comes the risk of anthropomorphising the data received when directly observing it, or placing bias on one area over another when comparing programmatically. At its base level, the eye data gathered is most ‘pure’ when considered as a single vector of ommatidial samples, much in the same way that the neural systems of any insect will interpret the data. In contrast, all the images that we have presented here have been projected onto a 2D plane in order to be displayed. Great care must be taken when considering these projections, as they can introduce biases that may be easy to overlook. For instance, in Figure 6a&b, orientation-wise spherical projection mapping was used to artificially place the ommatidia across the 2D spaces, forming panoramic Voronoi plots of observed light at each ommatidium. In these projections, much as is seen in 2D projections of Earth, the points at the poles of the image are artificially inflated, giving a bias in terms of raw surface area affected toward ommatidia present in the ventral and dorsal regions of the eye. Not only do these visual distortions warp the image produced, but if that image is used as the basis for algorithmic comparison (e.g., as the input to an image difference function [Philippides et al., 2011]) then that too, will be effected by these projection biases. In the case where direct comparison between two eyes with entirely dissimilar surfaces is attempted, the problems can be magnified further, as the projections of each eye will introduce biases that may interact in unexpected ways.

These projection biases highlight how fundamentally difficult it is to directly compare eye designs that do not share an underlying surface structure, unless those surfaces happen to be affine transformations of one another (they are affinely equivalent). For any surfaces that are affinely equivalent, the surface itself can be used as a projection surface. In this way, for instance, the outputs of two cubic eyes could be compared to each other by performing all comparison calculations on the cube’s surface, interpolating between points where an ommatidia exists on one eye but not another. It is trivial to directly compare compound eye images from eyes with the same structure in terms of ommatidial placement, and eyes with surfaces that are affinely equivalent can also be compared; however, further precautions must be taken when comparing the images from two or more eyes that do not have compatible projection surfaces so as not to unduly bias a portion of one surface over any other. If direct comparison between two eye designs is required, and their surfaces are not affine transformations of each other, then intermediate surfaces to perform calculations over must be found within the interpolation from one surface to the other, with a significant amount of care taken to reduce as much as possible projection artefacts forming between the two. However, we recommend instead taking a proxy metric as demonstrated here (using a localisation task) to measure differences between eye-accessible information content.

By basing CompoundRay around the conceptually simple approach of ray-casting and adhering to open standards, the rendering system has been designed with open development in mind. In the future, the library could be extended to further enhance biological realism by adding non-visible spectral (e.g. ultraviolet) lighting (Möller, 2002; Stone et al., 2006; Differt and Möller, 2015) and polarisation (e.g.Gkanias et al., 2019) sensitivities, as well as more realistic photo-receptor response characteristics (e.g. Song et al., 2009). Similarly, new technologies could be rapidly evaluated by simulating the input to novel imaging systems (e.g. dynamic; Viollet and Franceschini, 2010 or heterogeneous pixel arrays) and processing pipelines (e.g. event-based retinas; Gallego et al., 2022). In addition, environmental features such as shadow casting, reflection, and refractions present obvious extensions as well the introduction of features such as adaptive anti-aliasing and increasing the projections available to users. Currently, CompoundRay returns all output images in an eight-bit colourspace, which aligns with most contemporary display equipment. However, we note that the underlying GPU implementation operates on 32-bit floating-point variables which if exposed to the high-level Python API could improve the system’s ability to represent visual sensors that experience a wider bandwidth of light (hyperspectral visual systems). It is hoped that as further efforts are conducted to map the eye structures of insects, CompoundRay will serve as a key tool in uncovering the intricacies of these eye designs in a modern, data-driven way.

The manuscript is a computational study, with all modelling code and data accessible on GitHub at https://github.com/ManganLab/eye-renderer. Use of the natural environment was kindly provided by Dr. JoeWoodgate, Queen Mary University of London and is subject to upcoming publication. As such, instead included in the CompoundRay repository is a stand-in natural 3D terrain model. As all models are used for demonstrative purpose, this stand-in model offers little difference to the natural model used, bar it's subjectively lower-quality aesthetics.

1. Conference

   1. Appel A

   (1968) Some techniques for shading machine renderings of solids the April 30--May 2, 1968

   spring joint computer conference.

   https://doi.org/10.1145/1468075.1468082

   • Google Scholar
2. 1. Arendt D

   (2003)

   Evolution of eyes and photoreceptor cell types

   The International Journal of Developmental Biology 47:563–571.

   • PubMed
   • Google Scholar
3. 1. Baddeley B
   2. Graham P
   3. Husbands P
   4. Philippides A

   (2012) A model of ant route navigation driven by scene familiarity

   PLOS Computational Biology 8:e1002336.

   https://doi.org/10.1371/journal.pcbi.1002336

   • PubMed
   • Google Scholar
4. 1. Bagheri ZM
   2. Jessop AL
   3. Kato S
   4. Partridge JC
   5. Shaw J
   6. Ogawa Y
   7. Hemmi JM

   (2020) A new method for mapping spatial resolution in compound eyes suggests two visual streaks in fiddler crabs

   The Journal of Experimental Biology 223:jeb210195.

   https://doi.org/10.1242/jeb.210195

   • PubMed
   • Google Scholar
5. 1. Baird E
   2. Taylor G

   (2017) X-Ray micro computed-tomography

   Current Biology 27:R289–R291.

   https://doi.org/10.1016/j.cub.2017.01.066

   • PubMed
   • Google Scholar
6. 1. Basten K
   2. Mallot HA

   (2010) Simulated visual homing in desert ant natural environments: efficiency of skyline cues

   Biol Cybern 102:413–425.

   https://doi.org/10.1007/s00422-010-0375-9

   • PubMed
   • Google Scholar
7. 1. Burgess J

   (2020) Rtx on—the NVIDIA Turing GpU

   IEEE Micro 40:36–44.

   https://doi.org/10.1109/MM.2020.2971677

   • Google Scholar
8. 1. Christensen P
   2. Fong J
   3. Shade J
   4. Wooten W
   5. Schubert B
   6. Kensler A
   7. Friedman S
   8. Kilpatrick C
   9. Ramshaw C
   10. Bannister M
   11. Rayner B
   12. Brouillat J
   13. Liani M

   (2018) RenderMan: an advanced path-tracing architecture for movie rendering

   ACM Transactions on Graphics 37:1–21.

   https://doi.org/10.1145/3182162

   • Google Scholar
9. 1. Collett TS
   2. Land MF

   (1975) Visual control of flight behaviour in the hoverflysyritta pipiens L

   Journal of Comparative Physiology ? A 99:1–66.

   https://doi.org/10.1007/BF01464710

   • Google Scholar
10. Book

    1. Collins S

    (1998) Reconstructing the visual field of compound eyes

    In: Rocamora SP, editors. Eurographics Rendering Workshop. Springer Wien. pp. 81–92.

    https://doi.org/10.1007/978-3-7091-6858-5

    • Google Scholar
11. 1. Differt D
    2. Möller R

    (2015) Insect models of illumination-invariant skyline extraction from UV and green channels

    Journal of Theoretical Biology 380:444–462.

    https://doi.org/10.1016/j.jtbi.2015.06.020

    • PubMed
    • Google Scholar
12. 1. Differt D
    2. Möller R

    (2016) Spectral skyline separation: extended landmark databases and panoramic imaging

    Sensors 16:10.

    https://doi.org/10.3390/s16101614

    • PubMed
    • Google Scholar
13. 1. Franz MO
    2. Schölkopf B
    3. Mallot HA
    4. Bülthoff HH

    (1998) Where did I take that snapshot? scene-based homing by image matching

    Biol Cybern 79:191–202.

    https://doi.org/10.1007/s004220050470

    • Google Scholar
14. 1. Gallego G
    2. Delbruck T
    3. Orchard G
    4. Bartolozzi C
    5. Taba B
    6. Censi A
    7. Leutenegger S
    8. Davison AJ
    9. Conradt J
    10. Daniilidis K
    11. Scaramuzza D

    (2022) Event-based vision: A survey

    IEEE Transactions on Pattern Analysis and Machine Intelligence 44:154–180.

    https://doi.org/10.1109/TPAMI.2020.3008413

    • PubMed
    • Google Scholar
15. Thesis

    1. Giger AD

    (1996)

    PhD thesis: Honeybee vision: analysis of pattern orientation

    The Australian National University.

    • Google Scholar
16. 1. Gkanias E
    2. Risse B
    3. Mangan M
    4. Webb B

    (2019) From skylight input to behavioural output: a computational model of the insect polarised light COMPASS

    PLOS Computational Biology 15:e1007123.

    https://doi.org/10.1371/journal.pcbi.1007123

    • PubMed
    • Google Scholar
17. Report

    1. Golmant N
    2. Vemuri N
    3. Yao Z
    4. Feinberg V
    5. Gholami A
    6. Rothauge K
    7. Mahoney MW
    8. Gonzalez J

    (2018)

    On The Computational Inefficiency Of Large Batch Sizes For Stochastic Gradient Descent Technical report

    ICLR.

    • Google Scholar
18. 1. Greiner B
    2. Ribi WA
    3. Warrant EJ

    (2004) Retinal and optical adaptations for nocturnal vision in the halictid bee megalopta genalis

    Cell and Tissue Research 316:377–390.

    https://doi.org/10.1007/s00441-004-0883-9

    • PubMed
    • Google Scholar
19. 1. Harris CR
    2. Millman KJ
    3. van der Walt SJ
    4. Gommers R
    5. Virtanen P
    6. Cournapeau D
    7. Wieser E
    8. Taylor J
    9. Berg S
    10. Smith NJ
    11. Kern R
    12. Picus M
    13. Hoyer S
    14. van Kerkwijk MH
    15. Brett M
    16. Haldane A
    17. Del Río JF
    18. Wiebe M
    19. Peterson P
    20. Gérard-Marchant P
    21. Sheppard K
    22. Reddy T
    23. Weckesser W
    24. Abbasi H
    25. Gohlke C
    26. Oliphant TE

    (2020) Array programming with numpy

    Nature 585:357–362.

    https://doi.org/10.1038/s41586-020-2649-2

    • PubMed
    • Google Scholar
20. 1. Juusola M
    2. French AS

    (1997) Visual acuity for moving objects in first- and second-order neurons of the fly compound eye

    Journal of Neurophysiology 77:1487–1495.

    https://doi.org/10.1152/jn.1997.77.3.1487

    • PubMed
    • Google Scholar
21. 1. Juusola M
    2. Dau A
    3. Song Z
    4. Solanki N
    5. Rien D
    6. Jaciuch D
    7. Dongre SA
    8. Blanchard F
    9. de Polavieja GG
    10. Hardie RC
    11. Takalo J

    (2017) Microsaccadic sampling of moving image information provides Drosophila hyperacute vision

    eLife 6:e26117.

    https://doi.org/10.7554/eLife.26117

    • PubMed
    • Google Scholar
22. Conference

    1. Kajiya JT

    (1986) The rendering equation the 13th annual conference

    ACM SIGGRAPH Computer Graphics.

    https://doi.org/10.1145/15922.15902

    • Google Scholar
23. Preprint

    1. Kemppainen J
    2. Scales B
    3. Haghighi KR
    4. Takalo J
    5. Mansour N
    6. McManus J
    7. Leko G
    8. Saari P
    9. Hurcomb J
    10. Antohi A
    11. Suuronen JP
    12. Blanchard F
    13. Hardie RC
    14. Song Z
    15. Hampton M
    16. Eckermann M
    17. Westermeier F
    18. Frohn J
    19. Hoekstra H
    20. Lee CH
    21. Huttula M
    22. Mokso R
    23. Juusola M

    (2021) Binocular Mirror-Symmetric Microsaccadic Sampling Enables Drosophila Hyperacute 3D-Vision

    bioRxiv.

    https://doi.org/10.1101/2021.05.03.442473

    • Google Scholar
24. 1. Labhart T
    2. Nilsson DE

    (1995) The dorsal eye of the dragonfly sympetrum: specializations for prey detection against the blue sky

    Journal of Comparative Physiology A 176:437–453.

    https://doi.org/10.1007/BF00196410

    • Google Scholar
25. 1. Lambrinos D
    2. Kobayashi H
    3. Pfeifer R
    4. Maris M
    5. Labhart T
    6. Wehner R

    (1997) An autonomous agent navigating with a polarized light COMPASS

    Adaptive Behavior 6:131–161.

    https://doi.org/10.1177/105971239700600104

    • Google Scholar
26. Book

    1. Land MF

    (1989) Variations in the structure and design of compound eyes

    In: Stavenga DG, Hardie RC, editors. In Facets of Vision. Berlin Heidelberg, Berlin, Heidelberg: Springer. pp. 90–111.

    https://doi.org/10.1007/978-3-642-74082-4_5

    • Google Scholar
27. 1. Land MF
    2. Fernald RD

    (1992) The evolution of eyes

    Annual Review of Neuroscience 15:1–29.

    https://doi.org/10.1146/annurev.ne.15.030192.000245

    • PubMed
    • Google Scholar
28. 1. Land MF

    (1997) Visual acuity in insects

    Annual Review of Entomology 42:147–177.

    https://doi.org/10.1146/annurev.ento.42.1.147

    • PubMed
    • Google Scholar
29. Book

    1. Land MF
    2. Nilsson DE

    (2002)

    Animal Eyes

    Oxford University Press.

    • Google Scholar
30. Conference

    1. Li Z
    2. Liu F
    3. Yang W
    4. Peng S
    5. Zhou J

    (2021) A survey of convolutional neural networks: analysis, applications, and prospects

    IEEE Transactions on Neural Networks and Learning Systems.

    https://doi.org/10.1109/TNNLS.2021.3084827

    • Google Scholar
31. 1. Lippmann RP

    (1987) An introduction to computing with neural nets

    IEEE ASSP Magazine 4:4–22.

    https://doi.org/10.1109/MASSP.1987.1165576

    • Google Scholar
32. 1. Mangan M
    2. Webb B

    (2009) Modelling place memory in crickets

    Biol Cybern 101:307–323.

    https://doi.org/10.1007/s00422-009-0338-1

    • PubMed
    • Google Scholar
33. 1. Meyer EP
    2. Labhart T

    (1993) Morphological specializations of dorsal rim ommatidia in the compound eye of dragonflies and damselfies (Odonata)

    Cell & Tissue Research 272:17–22.

    https://doi.org/10.1007/BF00323566

    • Google Scholar
34. Conference

    1. Millward B
    2. Maddock S
    3. Mangan M
    4. Sheffield Robotics, The University Of Sheffield
    5. Department of Computer Science, The University Of Sheffield

    (2020) Towards Insect Inspired Visual Sensors for Robots

    UKRAS20 Conference.

    https://doi.org/10.31256/Do2Ik3H

    • Google Scholar
35. 1. Möller R

    (2002) Insects could exploit UV-green contrast for landmark navigation

    Journal of Theoretical Biology 214:619–631.

    https://doi.org/10.1006/jtbi.2001.2484

    • PubMed
    • Google Scholar
36. Book

    1. Neumann TR

    (2002) Modeling insect compound eyes: space-variant spherical vision

    In: Poggio TA, editors. Biologically Motivated Computer Vision. Springer. pp. 360–367.

    https://doi.org/10.1007/3-540-36181-2

    • Google Scholar
37. 1. Nityananda V
    2. Bissianna G
    3. Tarawneh G
    4. Read J

    (2016) Small or far away? size and distance perception in the praying mantis

    Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 371:20150262.

    https://doi.org/10.1098/rstb.2015.0262

    • PubMed
    • Google Scholar
38. 1. Nityananda V
    2. Tarawneh G
    3. Henriksen S
    4. Umeton D
    5. Simmons A
    6. Read JCA

    (2018) A novel form of stereo vision in the praying mantis

    Current Biology 28:588–593.

    https://doi.org/10.1016/j.cub.2018.01.012

    • PubMed
    • Google Scholar
39. 1. Ofstad TA
    2. Zuker CS
    3. Reiser MB

    (2011) Visual place learning in Drosophila melanogaster

    Nature 474:204–207.

    https://doi.org/10.1038/nature10131

    • PubMed
    • Google Scholar
40. 1. Parker SG
    2. Bigler J
    3. Dietrich A
    4. Friedrich H
    5. Hoberock J
    6. Luebke D
    7. McAllister D
    8. McGuire M
    9. Morley K
    10. Robison A
    11. Stich M

    (2010) OptiX: a general purpose ray tracing engine

    ACM Transactions on Graphics 29:1–13.

    https://doi.org/10.1145/1778765.1778803

    • Google Scholar
41. 1. Philippides A
    2. Baddeley B
    3. Cheng K
    4. Graham P

    (2011) How might ants use panoramic views for route navigation?

    The Journal of Experimental Biology 214:445–451.

    https://doi.org/10.1242/jeb.046755

    • PubMed
    • Google Scholar
42. Preprint

    1. Polster J
    2. Petrasch J
    3. Menzel R
    4. Landgraf T

    (2018) Reconstructing the Visual Perception of Honey Bees in Complex 3-D Worlds

    arXiv.

    https://arxiv.org/abs/1811.07560

    • Google Scholar
43. Conference

    1. Purcell TJ
    2. Buck I
    3. Mark WR
    4. Hanrahan P

    (2005) Ray tracing on programmable graphics hardware

    ACM SIGGRAPH 2005 Courses.

    https://doi.org/10.1145/1198555.1198798

    • Google Scholar
44. 1. Risse B
    2. Mangan M
    3. Stürzl W
    4. Webb B

    (2018) Software to convert terrestrial lidar scans of natural environments into photorealistic meshes

    Environmental Modelling & Software 99:88–100.

    https://doi.org/10.1016/j.envsoft.2017.09.018

    • Google Scholar
45. Book

    1. Robinet F
    2. Arnaud R
    3. Parisi T
    4. Cozzi P

    (2014)

    GlTF: designing an open-standard runtime asset format

    In: Engel W, editors. GPU Pro 5: Advanced Rendering Techniques. CRC Press. pp. 18–522.

    • Google Scholar
46. 1. Roth SD

    (1982) Ray casting for modeling solids

    Computer Graphics and Image Processing 18:109–144.

    https://doi.org/10.1016/0146-664X(82)90169-1

    • Google Scholar
47. 1. Scarselli F
    2. Gori M
    3. Tsoi AC
    4. Hagenbuchner M
    5. Monfardini G

    (2009) The graph neural network model

    IEEE Transactions on Neural Networks 20:61–80.

    https://doi.org/10.1109/TNN.2008.2005605

    • PubMed
    • Google Scholar
48. 1. Schwarz S
    2. Narendra A
    3. Zeil J

    (2011) The properties of the visual system in the Australian desert ant melophorus bagoti

    Arthropod Structure & Development 40:128–134.

    https://doi.org/10.1016/j.asd.2010.10.003

    • PubMed
    • Google Scholar
49. Book

    1. Song Z
    2. Coca D
    3. Billings S

    (2009) Biophysical Modeling of a Drosophila Photoreceptor. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    Springer.

    https://doi.org/10.1007/978-3-642-74082-4_5

    • Google Scholar
50. 1. Song YM
    2. Xie Y
    3. Malyarchuk V
    4. Xiao J
    5. Jung I
    6. Choi KJ
    7. Liu Z
    8. Park H
    9. Lu C
    10. Kim RH
    11. Li R
    12. Crozier KB
    13. Huang Y
    14. Rogers JA

    (2013) Digital cameras with designs inspired by the arthropod eye

    Nature 497:95–99.

    https://doi.org/10.1038/nature12083

    • PubMed
    • Google Scholar
51. Conference

    1. Stone T
    2. Mangan M
    3. Ardin P
    4. Webb B

    (2006) Sky segmentation with ultraviolet images can be used for navigation

    Robotics.

    https://doi.org/10.15607/RSS.2014.X.047

    • Google Scholar
52. 1. Stürzl W
    2. Grixa I
    3. Mair E
    4. Narendra A
    5. Zeil J

    (2015) Three-Dimensional models of natural environments and the mapping of navigational information

    Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 201:563–584.

    https://doi.org/10.1007/s00359-015-1002-y

    • PubMed
    • Google Scholar
53. Conference

    1. Taylor GJ
    2. Baird E

    (2017)

    INSECT VISION: SEGMENTATION TO SIMULATIONS

    In 3rd International Conference on Tomography of Materials and Structures.

    • Google Scholar
54. 1. van Praagh JP
    2. Ribi W
    3. Wehrhahn C
    4. Wittmann D

    (1980) Drone bees fixate the Queen with the dorsal frontal part of their compound eyes

    Journal of Comparative Physiology 136:263–266.

    https://doi.org/10.1007/BF00657542

    • Google Scholar
55. Book

    1. Van Rossum G
    2. Drake FL

    (2009)

    Python 3 Reference Manual

    CreateSpace, Scotts Valley, CA.

    • Google Scholar
56. 1. Vardy A
    2. Moller R

    (2005) Biologically plausible visual homing methods based on optical flow techniques

    Connection Science 17:47–89.

    https://doi.org/10.1080/09540090500140958

    • Google Scholar
57. 1. Viollet S
    2. Franceschini N

    (2010) A hyperacute optical position sensor based on biomimetic retinal micro-scanning

    Sensors and Actuators A 160:60–68.

    https://doi.org/10.1016/j.sna.2010.03.036

    • Google Scholar
58. 1. Viollet S

    (2014) Vibrating makes for better seeing: from the fly’s micro-eye movements to hyperacute visual sensors

    Frontiers in Bioengineering and Biotechnology 2:9.

    https://doi.org/10.3389/fbioe.2014.00009

    • PubMed
    • Google Scholar
59. 1. Wardill TJ
    2. Fabian ST
    3. Pettigrew AC
    4. Stavenga DG
    5. Nordström K
    6. Gonzalez-Bellido PT

    (2017) A novel interception strategy in a miniature robber fly with extreme visual acuity

    Current Biology 27:854–859.

    https://doi.org/10.1016/j.cub.2017.01.050

    • PubMed
    • Google Scholar
60. Book

    1. Wehner R

    (2020) Desert Navigator

    Harvard University Press.

    https://doi.org/10.4159/9780674247918

    • Google Scholar
61. Conference

    1. Whitted T

    (1979) An improved illumination model for shaded display

    the 6th annual conference.

    https://doi.org/10.1145/800249.807419

    • Google Scholar
62. 1. Wiederman SD
    2. Fabian JM
    3. Dunbier JR
    4. O’Carroll DC

    (2017) A predictive focus of gain modulation encodes target trajectories in insect vision

    eLife 6:e26478.

    https://doi.org/10.7554/eLife.26478

    • PubMed
    • Google Scholar
63. 1. Wystrach A
    2. Dewar A
    3. Philippides A
    4. Graham P

    (2016) How do field of view and resolution affect the information content of panoramic scenes for visual navigation? A computational investigation

    Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 202:87–95.

    https://doi.org/10.1007/s00359-015-1052-1

    • PubMed
    • Google Scholar
64. 1. Zeil J
    2. Nalbach G
    3. Nalbach HO

    (1986) Eyes, eye stalks and the visual world of semi-terrestrial crabs

    Journal of Comparative Physiology A 159:801–811.

    https://doi.org/10.1007/BF00603733

    • Google Scholar
65. 1. Zeil J
    2. Hofmann MI
    3. Chahl JS

    (2003) Catchment areas of panoramic snapshots in outdoor scenes

    Journal of the Optical Society of America. A, Optics, Image Science, and Vision 20:450–469.

    https://doi.org/10.1364/josaa.20.000450

    • PubMed
    • Google Scholar

Author details

1. Blayze Millward

   Department of Computer Science, University of Sheffield, Sheffield, United Kingdom

   Contribution

   Conceptualization, Software, Investigation, Visualization, Methodology, Writing - original draft

   For correspondence

   b.f.millward@sheffield.ac.uk

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-9025-1484

2. Steve Maddock

   Department of Computer Science, University of Sheffield, Sheffield, United Kingdom

   Contribution

   Supervision, Validation, Writing - review and editing

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-3179-0263

3. Michael Mangan

   Department of Computer Science, University of Sheffield, Sheffield, United Kingdom

   Contribution

   Conceptualization, Supervision, Funding acquisition, Methodology, Writing - review and editing

   Competing interests

   No competing interests declared

• 1,240

  views

• 145

  downloads

• 4

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.