We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into the instance segmentation model. This joint learning fuses the instance- and image-level reasoning of the mask prediction on the different segmentation tasks, which was missing in the previous work of learning instance segmentation only (instance-only). In the experiments, we evaluated the proposed joint learning comparing the instance-only learning on the test dataset. We also applied the joint learning model to 2 different types of robotic pick-and-place tasks (random and target picking) and evaluated its effectiveness to achieve real-world robotic tasks.

中文翻译：

---

杂波中具有重度遮挡的机器人拾取和放置的实例和语义分割的联合学习

我们提出了可见和遮挡区域掩码的实例和语义分割的联合学习。与实例遮挡分割共享特征提取器，我们将语义遮挡分割引入到实例分割模型中。这种联合学习在不同的分割任务上融合了掩码预测的实例级和图像级推理，这在之前仅学习实例分割（instance-only）的工作中是缺失的。在实验中，我们评估了所提出的联合学习，比较了测试数据集上的仅实例学习。我们还将联合学习模型应用于 2 种不同类型的机器人拾取和放置任务（随机和目标拾取），并评估其实现现实世界机器人任务的有效性。