Object manipulation in object-stacking scenes is a significant but challenging skill for intelligent robots. In most cases, the relationships among objects should be considered before manipulation to prevent chaos and damages. However, the analysis of object relationships in object-stacking scenes, especially for robotic manipulation, remains to be unsolved. To this end, this paper presents a new convolutional neural network (CNN) architecture, called Visual Manipulation Relationship Network (VMRN), to recognize the visual manipulation relationships (VMR) between objects in real-time. By considering the manipulation relationships in object-stacking scenes, it ensures that the robot can complete manipulation tasks safely and reliably. The core of our model is the Object Pairing Pooling Layer (OP²L), which makes it possible to recognize objects and all possible VMRs in one forward process. Moreover, to train VMRN, we contribute a dataset named Visual Manipulation Relationship Dataset (VMRD) consisting of 4683 images with more than 16,000 object instances and the VMRs between each object pair. The experimental results show that the proposed network architecture can detect objects and predict VMRs.

对于智能机器人，在对象堆叠场景中进行对象操纵是一项重要但具有挑战性的技能。在大多数情况下，在操作之前应考虑对象之间的关系，以防止混乱和损坏。然而，对象堆叠场景中的对象关系分析，尤其是对于机器人操纵，尚待解决。为此，本文提出了一种新的卷积神经网络（CNN）架构，称为视觉操纵关系网络（VMRN），以实时识别对象之间的视觉操纵关系（VMR）。通过考虑对象堆叠场景中的操纵关系，可以确保机器人可以安全可靠地完成操纵任务。我们模型的核心是对象配对池层（OP ²L），这样就可以在一个正向过程中识别对象和所有可能的VMR。此外，为了训练VMRN，我们贡献了一个名为“视觉操纵关系数据集”（VMRD）的数据集，该数据集包含4683张图像，其中包含16,000多个对象实例以及每个对象对之间的VMR。实验结果表明，所提出的网络体系结构可以检测物体并预测VMR。