Deep convolutional neural networks have shown outstanding performance in the task of semantically segmenting images. Applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data. Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input. A PointNet describes the local geometry which we embed into a sparse permutohedral lattice. The lattice allows for fast convolutions while keeping a low memory footprint. Further, we introduce DeformSlice, a novel learned data-dependent interpolation for projecting lattice features back onto the point cloud. We present results of 3D segmentation on multiple datasets where our method achieves state-of-the-art performance. We also extend and evaluate our network for instance and dynamic object segmentation.

深度卷积神经网络在语义分割图像的任务中表现出色。由于大量内存需求和缺乏结构化数据，在 3D 数据上应用相同的方法仍然面临挑战。在这里，我们提出了 LatticeNet，这是一种新的 3D 语义分割方法，它以原始点云作为输入。PointNet 描述了我们将其嵌入到稀疏的 permutohedral 格子中的局部几何。格子允许快速卷积，同时保持低内存占用。此外，我们引入了 DeformSlice，这是一种新颖的学习数据相关插值，用于将点阵特征投影回点云。我们在多个数据集上展示 3D 分割结果，我们的方法实现了最先进的性能。