3D pedestrian detection is a challenging task in automated driving because pedestrians are relatively small, frequently occluded and easily confused with narrow vertical objects. LiDAR and camera are two commonly used sensor modalities for this task, which should provide complementary information. Unexpectedly, LiDAR-only detection methods tend to outperform multisensor fusion methods in public benchmarks. Recently, PointPainting has been presented to eliminate this performance drop by effectively fusing the output of a semantic segmentation network instead of the raw image information. In this paper, we propose a generalization of PointPainting to be able to apply fusion at different levels. After the semantic augmentation of the point cloud, we encode raw point data in pillars to get geometric features and semantic point data in voxels to get semantic features and fuse them in an effective way. Experimental results on the KITTI test set show that SemanticVoxels achieves state-of-the-art performance in both 3D and bird's eye view pedestrian detection benchmarks. In particular, our approach demonstrates its strength in detecting challenging pedestrian cases and outperforms current state-of-the-art approaches.

中文翻译：

---

SemanticVoxels：使用 LiDAR 点云和语义分割进行 3D 行人检测的顺序融合

3D 行人检测在自动驾驶中是一项具有挑战性的任务，因为行人相对较小，经常被遮挡并且容易与狭窄的垂直物体混淆。LiDAR 和相机是此任务的两种常用传感器模式，应提供补充信息。出乎意料的是，在公共基准测试中，仅使用 LiDAR 的检测方法往往优于多传感器融合方法。最近，PointPainting 被提出通过有效地融合语义分割网络的输出而不是原始图像信息来消除这种性能下降。在本文中，我们提出了 PointPainting 的泛化，以便能够在不同级别应用融合。点云语义增强后，我们在柱子中编码原始点数据以获得几何特征和体素中的语义点数据以获得语义特征并以有效的方式融合它们。在 KITTI 测试集上的实验结果表明，SemanticVoxels 在 3D 和鸟瞰行人检测基准测试中都达到了最先进的性能。特别是，我们的方法展示了其在检测具有挑战性的行人案例方面的优势，并且优于当前最先进的方法。