A deep learning pipeline for time-lapse camera monitoring of insects and their floral environments

Arthropods, including insects, represent the most diverse group and contribute significantly to animal biomass. Automatic monitoring of insects and other arthropods enables quick and efficient observation and management of ecologically and economically important targets such as pollinators, natural enemies, disease vectors, and agricultural pests. The integration of cameras and computer vision facilitates innovative monitoring approaches for agriculture, ecology, entomology, evolution, and biodiversity. However, studying insects and their interactions with flowers and vegetation in natural environments remains challenging, even with automated camera monitoring. This paper presents a comprehensive methodology to monitor abundance and diversity of arthropods in the wild and to quantify floral cover as a key resource. We apply the methods across more than 10 million images recorded over two years using 48 insect camera traps placed in three main habitat types. The cameras monitor arthropods, including insect visits, on a specific mix of Sedum plant species with white, yellow and red/pink colored of flowers. The proposed deep-learning pipeline estimates flower cover and detects and classifies arthropod taxa from time-lapse recordings. However, the flower cover serves only as an estimate to correlate insect activity with the flowering plants.Color and semantic segmentation with DeepLabv3 are combined to estimate the percent cover of flowers of different colors. Arthropod detection incorporates motion-informed enhanced images and object detection with You-Only-Look-Once (YOLO), followed by filtering stationary objects to minimize double counting of non-moving animals and erroneous background detections. This filtering approach has been demonstrated to significantly decrease the incidence of false positives, since arthropods, occur in less than 3% of the captured images. The final step involves grouping arthropods into 19 taxonomic classes. Seven state-of-the-art models were trained and validated, achieving F1-scores ...