In this work we address the problem of 3D object detection from point clouds in data-limited environments. Training with simulated data is a common approach in such scenarios; however a sim-to-real gap exists between clean and crisp simulated clouds and noisy real clouds. Previous sim-to-real approaches for processing point cloud scenes have compressed clouds into 2D and used 2D transfer techniques. However, this may compress useful 3D information and does not effectively reason about the unstructured nature of point cloud data. We thus propose a 3D adversarial training architecture that leverages an adaptive sampling module to reason about the unstructured nature of point cloud data. Our approach encourages the 3D feature encoder to extract features that are invariant across simulated and real scenes. We validate our approach in the context of the DARPA Subterranean Challenge and demonstrate that our 3D adversarial training architecture improves 3D object detection performance by up to 15% depending on the data representation.

中文翻译：

---

用于模拟真实 3D 物体检测的点云对抗训练


在这项工作中，我们解决了数据有限环境中点云的 3D 对象检测问题。在这种情况下，使用模拟数据进行训练是一种常见的方法；然而，干净、清晰的模拟云与嘈杂的真实云之间存在模拟与真实的差距。以前用于处理点云场景的模拟到真实方法已将云压缩为 2D 并使用 2D 传输技术。然而，这可能会压缩有用的 3D 信息，并且不能有效地推理点云数据的非结构化性质。因此，我们提出了一种 3D 对抗训练架构，利用自适应采样模块来推理点云数据的非结构化性质。我们的方法鼓励 3D 特征编码器提取在模拟和真实场景中不变的特征。我们在 DARPA 地下挑战赛中验证了我们的方法，并证明我们的 3D 对抗训练架构可将 3D 对象检测性能提高高达 15%，具体取决于数据表示。