Instance segmentation of unknown objects from images is regarded as relevant for several robot skills including grasping, tracking and object sorting. Recent results in computer vision have shown that large hand-labeled datasets enable high segmentation performance. To overcome the time-consuming process of manually labeling data for new environments, we present a transfer learning approach for robots that learn to segment objects by interacting with their environment in a self-supervised manner. Our robot pushes unknown objects on a table and uses information from optical flow to create training labels in the form of object masks. To achieve this, we fine-tune an existing DeepMask network for instance segmentation on the self-labeled training data acquired by the robot. We evaluate our trained network (SelfDeepMask) on a set of real images showing challenging and cluttered scenes with novel objects. Here, SelfDeepMask outperforms the DeepMask network trained on the COCO dataset by 9.5% in average precision. Furthermore, we combine our approach with recent approaches for training with noisy labels in order to better cope with induced label noise.

中文翻译：

---

通过物理交互进行实例分割的自监督迁移学习

图像中未知物体的实例分割被认为与多种机器人技能相关，包括抓取、跟踪和物体分类。计算机视觉的最新结果表明，大型手工标记数据集可实现高分割性能。为了克服为新环境手动标记数据的耗时过程，我们为机器人提供了一种迁移学习方法，该方法通过以自我监督的方式与环境交互来学习分割对象。我们的机器人将未知物体推到桌子上，并使用来自光流的信息以物体掩码的形式创建训练标签。为了实现这一点，我们对现有的 DeepMask 网络进行了微调，例如对机器人获取的自标记训练数据进行分割。我们在一组真实图像上评估我们训练的网络 (SelfDeepMask)，这些图像显示具有新物体的具有挑战性和混乱的场景。在这里，SelfDeepMask 的平均精度比在 COCO 数据集上训练的 DeepMask 网络高 9.5%。此外，我们将我们的方法与最近使用噪声标签进行训练的方法相结合，以更好地应对诱导的标签噪声。