Automating an insect biodiversity metric using distributed optical sensors: an evaluation across Kansas, USA cropping systems

Reviewer #1 (Public Review):

The article offers a comparative study between various methodologies to evaluate the abundance, richness, and diversity of insects from data obtained in a large-scale field experiment. The experiment is impressive in view of the number of locations, its spatial coverage, the number of instruments or methods used, and the data collected appears rich and worthy of multiple publications. The paper focuses on the validation of a novel approach based on optical sensors. These sensors collect the backscattered light from flying insects in their field of view and can retrieve the wingbeat frequency and the body-to-wing backscattering ratios.
Unfortunately, the paper is poorly written and hard to read, with a lack of clear sections, and an overall confusing structure. The methods, metrics, and data analysis are not properly and thoroughly described, making it sometimes difficult to evaluate the validity of the approach.
Most importantly, the methodology to retrieve the richness and diversity from optical sensors seems flawed. While the scope and scale of the experiment is valuable, I do not believe that this article supports the authors' claim. The main criticisms are described in more detail below.

1. The Material and Method section is poorly structured. The article focuses on a series of metrics to evaluate biodiversity from three independent methods: optical sensors, malaise traps, and net sweeping. The authors need to provide a clear and thorough description of what the metrics to be studied are, and how those metrics are evaluated for each method. While it is the main focus of the paper, the term "biodiversity metrics" is never properly defined, it is used in the singular form in both the title and abstract, then in its plural form in the rest of the paper, making the reader further doubt what exactly it means. It is then discussed using the correlation value retrieved when studying richness, so is the biodiversity metric the same as richness? Studying biodiversity remains a complex and sometimes contentious subject and this term, especially when measured by three different methods, is far from obvious. The term "community metrics" is defined as abundance, richness, and diversity; is that the same as biodiversity metrics? In any case, the method section should thoroughly describe how each of those metrics is calculated from the raw data collected by each method. This information is somewhat there, but in a very unorganized way, making it difficult to read. I would recommend organizing this section with multiple and clear sections: 1) describing the metrics that are meant to be studied, 2) the location, dates and time, type of crops, and other general information about the experiment, 3) description and methods around optical sensors, 4) description and methods around malaise traps, 5) description and methods around the sweeping. The last 3 sections should describe how it retrieves the previously defined metrics, potentially using equations.

2. Regarding the calculation of the body-to-wing ratio, sigma is described as a "signal" line 195, then is described as intensity counts in Figure 2; isn't it really the backscattering optical cross-section? It changes significantly over time during the signal, so how is one value of sigma calculated? Is it the average of the whole insect event? The maximum?

3. The "ecosystem services" paragraph is really confusing and needs to be rewritten.

4. Like for the method section, the result section should be structured around the comparison of each metric, abundance, richness, and diversity, or any other properly defined metrics described in the method, so that the result section is consistent with the method section.

5. The abundance is not correlated; interestingly, malaise traps and sweeping are even less correlated which further supports the claims by the authors that new and improved methods are needed. This part of the results could be further developed. A linear fit could be added to Figure 4.

6. Richness and diversity are the most problematic. Again, the method is poorly described, with pieces of explanation spread out throughout the paper, but my understanding is the following: the optical sensor retrieves two features from each insect signal, wbf, and BWR. Clustering is made using DBSCAN which has 2 parameters: minimum number of signals, and merge distance. It is important to note that these two parameters will greatly influence the number of clusters found by DBSCAN. The richness obtained by optical sensors is defined as the number of clusters and the diversity is evaluated from it as well. Hence, both diversity and richness are greatly dependent on the chosen parameters. The DBSCAN parameters are chosen by maximizing the Spearman correlation between richness obtained by the optical sensors and richness by the capture methods. I see a major problem here: if you optimize the parameters, that directly impact the retrieved diversity and richness by optical sensors, to have the best correlation with either the richness or diversity of the other methods, you will automatically create a correlation between the richness and diversity retrieved by the optical sensors and alternative methods. The p-value in Figure 6 does not represent the probability of the correlation hypothesis being false anymore, since the whole process is based on artificially forcing the correlation from the start.

7. In addition, the clustering method provides values higher than 80, which is quite unrealistic with just 2 features, wbf and BWR. It is clear from many studies using optical sensors that the features from optical sensors are subject to variability. Wbf has naturally some variances within the same species, not to mention temperature dependency. Backscattering cross sections will also heavily function on the insect's orientation (facing or sideways) while crossing the cone of light, and, even though it is a ratio, the collection efficiency of the instrument telescope and scattering efficiency of the target will be impacted by the position of the insects within the cone of light, which will also impact the variability on the BWR. While those features can still be used, obtaining 80 clusters from two variables with such statistical fluctuations is simply not credible. Additional features could help, such as the two wavelengths mentioned in the description of the optical sensor but are never mentioned again.

The conclusion then states that the study serves as the first field validation. I disagree; the abundance doesn't correlate, and the richness and diversity evaluations are flawed. While I do think there is great value in the work done by the authors through this impressive field experiment, and in general in their work toward the development of entomological optical sensors, I believe the data analysis and communication of the results do not support the conclusions drawn.

Reviewer #2 (Public Review):

Summary:

The manuscript by Rydhmer et al. proposes a new technology to survey insects. They deployed optical sensors in agricultural landscapes and contrast their results to those in classical malaise and sweep nets survey methodologies. They found the results of optical sensors to be comparable with classical survey methodologies. The authors discuss the pros and cons of their near-infrared sensor.

Strengths:
Contrasting the results of optical sensors with those obtained with classical malaise and sweep nets was a clever idea.

Weaknesses:
Maybe the first most important shortcoming is the lack of a larger question the new technology can help to answer. If the authors could frame their aims not only as a new tool to sample insects but maybe along the lines of a hypothesis to test in their (agricultural) field of research, this could be a more meaningful article.

The second more important shortcoming is the lack of more complex analyses. The authors seem to be so fixed on counts of abundance and species that they miss the opportunity to look for more complex patterns in their data. The addition of a simple analysis like an NMDS (to test composition changes) could improve the manuscript significantly.

The ecosystem process (granivory) assay is currently poorly contextualized and explained across the text; I was surprised to find this part in M&M without previous warning. It seems to me that adding this part could be a nice addition to the manuscript (see my comment above). But this needs to be explained better in all sections of the manuscript.

As I think that addressing my previous points will reshape the manuscript in important ways, I refrain from giving more specific details at this point. But there are some! Maybe only to mention that Figures 4 and 6 would benefit from individual regressions by crop and Figure 5 from adding results from optical sensors.