Frustum-based 3D detection methods suffer from the ignorance of a 2D detector for that the object will never be detected in point cloud if it is omitted by a 2D image proposal. In this work, we propose a novel method named semantic frustum-based sparsely embedded convolutional detection (SFB-SECOND) for 3D object detection, which is devoted to solving the limitation of frustum-based methods, i.e., heavily relying on the accurate 2D detector. Specifically, for the image and LIDAR describing the same scene, we initially use developed methods of semantic segmentation and object detection to generate the object mask, selecting all potential targets within two confidence-related regions. Through this object mask, we quickly locate the objects of interest in LIDAR and dig them up as semantic frustum. This selected frustum not only rules out more background and irrelevant objects in LIDAR but also maximizes the use of rich 3D information. Then, to accurate the orientation estimation, we introduce a refined form of region-aware loss regression to cooperate with the region-aware frustum. Besides, a new data augmentation strategy is proposed to further make haste the convergence speed and improve detection performance. In addition, the proposed SFB-SECOND achieves state-of-the-art performances on the 3D object detection benchmark KITTI with real-time speed, showing superiority over previous methods.

中文翻译：

---

基于语义平截头体的稀疏嵌入卷积检测

基于 Frustum 的 3D 检测方法受到 2D 检测器的无知的影响，因为如果 2D 图像建议省略了对象，则永远不会在点云中检测到对象。在这项工作中，我们提出了一种名为基于语义视锥体的稀疏嵌入卷积检测 (SFB-SECOND) 的 3D 对象检测新方法，该方法致力于解决基于视锥体的方法的局限性，即严重依赖于准确的 2D 检测器. 具体来说，对于描述同一场景的图像和激光雷达，我们最初使用语义分割和对象检测的开发方法来生成对象掩码，选择两个与置信度相关的区域内的所有潜在目标。通过这个对象掩码，我们可以快速定位 LIDAR 中感兴趣的对象，并将它们挖掘为语义平截头体。这种选择的平截头体不仅排除了激光雷达中更多的背景和不相关的物体，而且最大限度地利用了丰富的 3D 信息。然后，为了准确估计方向，我们引入了一种改进的区域感知损失回归形式，以与区域感知平截头体配合。此外，提出了一种新的数据增强策略，以进一步加快收敛速度​​并提高检测性能。此外，所提出的 SFB-SECOND 以实时速度在 3D 对象检测基准 KITTI 上实现了最先进的性能，显示出优于以前的方法。提出了一种新的数据增强策略，以进一步加快收敛速度​​并提高检测性能。此外，所提出的 SFB-SECOND 以实时速度在 3D 对象检测基准 KITTI 上实现了最先进的性能，显示出优于以前的方法。提出了一种新的数据增强策略，以进一步加快收敛速度​​并提高检测性能。此外，所提出的 SFB-SECOND 以实时速度在 3D 对象检测基准 KITTI 上实现了最先进的性能，显示出优于以前的方法。