3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human–machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model.

在众多的视觉，机器人技术和人机交互应用中，3D对象检测是一个重要而又具有挑战性的问题。给定RGB-D图像，任务是推断图像中对象的类标签和3D边界框。尽管在过去的十年中，先前的研究取得了显着进展，但是如何有效利用神经网络的特征融合来提高3D对象检测性能仍然是一个悬而未决的问题。本文提出了一种用于检测RGB-D图像中3D对象的多级融合网络（MFN）模型。MFN模型包含两个神经网络流，分别使用级联卷积模块提取RGB和深度特征。为了有效利用3D对象的信息，采用多级融合机制将卷积RGB和深度特征融合到多个级别。为了训练网络，我们通过在3D边界框回归上编码几何属性的差异，提出了一个新的加权损失函数。由于原始深度数据充满了噪点，因此我们还开发了一种自适应滤波算法来还原和校正深度图像。我们在具有挑战性的RGB-D数据集上测试了提出的模型。数据集上的实验结果证明了该模型的优势和优势。我们在具有挑战性的RGB-D数据集上测试了提出的模型。数据集上的实验结果证明了该模型的优势和优势。我们在具有挑战性的RGB-D数据集上测试了提出的模型。数据集上的实验结果证明了该模型的优势和优势。