当前位置: X-MOL 学术Inform. Fusion › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
SaccadeFork: A lightweight multi-sensor fusion-based target detector
Information Fusion ( IF 18.6 ) Pub Date : 2021-08-02 , DOI: 10.1016/j.inffus.2021.07.004
Zhenchao Ouyang 1, 2, 3, 4 , Jiahe Cui 1, 3, 4 , Xiaoyun Dong 1, 2, 3 , Yanqi Li 1 , Jianwei Niu 1, 3, 4
Affiliation  

Commercialization of self-driving applications requires precision and reliability of the perception system due to the highly dynamic and complex road environment. Early perception systems either rely on the camera or on LiDAR for moving obstacle detection. With the development of vehicular sensors and deep learning technologies, the multi-view and sensor fusion based convolutional neural network (CNN) model for detection tasks has become a popular research area. In this paper, we present a novel multi-sensor fusion-based CNN model–SaccadeFork–that integrates the image and upsampled LiDAR point clouds as the input. SaccadeFork includes two modules: (1) a lightweight backbone that consists of hourglass convolution feature extraction module and a parallel dilation convolution module for adaptation of the system to different target sizes; (2) an anchor-based detection head. The model also considers deployment of resource-limited edge devices in the vehicle. Two refinement strategies, i.e., Mixup and Swish activation function are also adopted to improve the model. Comparison with a series of latest models on public dataset of KITTI shows that SaccadeFork can achieve the optimal detection accuracy on vehicles and pedestrians under different scenarios. The final model is also deployed and tested on a local dataset collected based on edge devices and low-cost sensor solutions, and the results show that the model can achieve real-time efficiency and high detection accuracy.



中文翻译:

SaccadeFork:基于多传感器融合的轻量级目标检测器

由于高度动态和复杂的道路环境,自动驾驶应用的商业化需要感知系统的精度和可靠性。早期的感知系统要么依靠摄像头,要么依靠 LiDAR 进行移动障碍物检测。随着车载传感器和深度学习技术的发展,用于检测任务的基于多视图和传感器融合的卷积神经网络(CNN)模型已成为一个热门的研究领域。在本文中,我们提出了一种新颖的基于多传感器融合的 CNN 模型——SaccadeFork——它集成了图像和上采样的 LiDAR 点云作为输入。SaccadeFork 包括两个模块:(1)一个轻量级的主干,由沙漏卷积特征提取模块和并行扩张卷积模块组成,用于系统适应不同的目标尺寸;(2)基于anchor的检测头。该模型还考虑了在车辆中部署资源有限的边缘设备。还采用了两种细化策略,即 Mixup 和 Swish 激活函数来改进模型。与 KITTI 公共数据集上的一系列最新模型的比较表明,SaccadeFork 可以在不同场景下实现对车辆和行人的最佳检测精度。最终模型还在基于边缘设备和低成本传感器解决方案收集的本地数据集上进行部署和测试,结果表明该模型可以实现实时效率和高检测精度。与 KITTI 公共数据集上的一系列最新模型的比较表明,SaccadeFork 可以在不同场景下实现对车辆和行人的最佳检测精度。最终模型还在基于边缘设备和低成本传感器解决方案收集的本地数据集上进行部署和测试,结果表明该模型可以实现实时效率和高检测精度。与 KITTI 公共数据集上的一系列最新模型的比较表明,SaccadeFork 可以在不同场景下实现对车辆和行人的最佳检测精度。最终模型还在基于边缘设备和低成本传感器解决方案收集的本地数据集上进行部署和测试,结果表明该模型可以实现实时效率和高检测精度。

更新日期:2021-08-12
down
wechat
bug