In the work presented here, a new technique based upon stereo vision is proposed to acquire three-dimensional time-resolved trajectories with labeling support for multi-species insects under unconstrained flying conditions. A low-cost, off-the-shelf depth camera is used for stereo vision which is equipped with two wide-angle global-shutter IR imagers to compute depth map and a separate narrow-angle color imager to have color and texture information. Two novel strategies are employed to tackle the challenges imposed by the small size of insects, fast movements, and natural daylight, etc.; (i) introduction of a simple but robust technique for detection and tracking of insects using only the valid depth map data and subsequently to acquire 3-D trajectories (ii) object localization scheme for the insects in acquired trajectories in allied color frames based upon the analysis of insects' flying kinematics and their 3-D location coordinates. The object localization of flying insects in allied color frames provides support to label the multi-spp. trajectories with respective types of the species. The proposed technique provides completely generalized solution to acquire 3-D trajectories of the multi-spp. insects in natural outdoor conditions and in the presented work it is applied to acquire the trajectories of honey bees (Apis mellifera) and invasive hornets (Vespa spp.) near beehives. These hornets are serious honey bee stressors and cause severe damage to the hive's foraging force. The trajectory patterns of honey bees together with Vespa spp. can be analyzed for early detection of Vespa spp. near beehives as well as to estimate stress levels on the honey bees. Acquired 3-D trajectory data are validated for trajectory measurements as well as for the labeling support information. A mean error upper bound of 5 mm is estimated in trajectory measurements whereas 100% inter-spp. and up to 94% intra-spp. labeling support accuracies are recorded.

在这里介绍的工作中，提出了一种基于立体视觉的新技术，用于在无约束飞行条件下为多物种昆虫获取具有标记支持的三维时间分辨轨迹。低成本、现成的深度相机用于立体视觉，配备两个广角全局快门红外成像器来计算深度图和一个单独的窄角彩色成像器来获取颜色和纹理信息。采用两种新颖的策略来应对昆虫体型小、移动速度快和自然光等带来的挑战；(i) 引入一种简单而强大的技术，用于仅使用有效的深度图数据来检测和跟踪昆虫，然后获取 3-D 轨迹 (ii) 基于相关颜色帧中获取的轨迹中昆虫的目标定位方案分析昆虫的飞行运动学及其 3-D 位置坐标。相关颜色框架中飞行昆虫的对象定位为标记多物种提供了支持。不同种类物种的轨迹。所提出的技术提供了完全通用的解决方案来获取多 spp 的 3-D 轨迹。自然户外条件下的昆虫，在目前的工作中，它被应用于获取蜜蜂的轨迹 相关颜色框架中飞行昆虫的对象定位为标记多物种提供了支持。不同种类物种的轨迹。所提出的技术提供了完全通用的解决方案来获取多 spp 的 3-D 轨迹。自然户外条件下的昆虫，在目前的工作中，它被应用于获取蜜蜂的轨迹 相关颜色框架中飞行昆虫的对象定位为标记多物种提供了支持。不同种类物种的轨迹。所提出的技术提供了完全通用的解决方案来获取多 spp 的 3-D 轨迹。自然户外条件下的昆虫，在目前的工作中，它被应用于获取蜜蜂的轨迹(Apis mellifera)和蜂巢附近的入侵大黄蜂 ( Vespa spp.)。这些大黄蜂是蜜蜂的严重压力源，会对蜂巢的觅食力造成严重损害。蜜蜂与胡蜂属的轨迹模式。可以分析早期检测Vespa spp。蜂箱附近以及估计蜜蜂的压力水平。获取的 3-D 轨迹数据针对轨迹测量以及标记支持信息进行验证。轨迹测量中估计的平均误差上限为 5 mm，而 100% inter-spp。高达 94% 的物种内。记录标签支持准确度。