Unsupervised Domain Adaptation (DA) exploits the supervision of a label-rich source dataset to make predictions on an unlabeled target dataset by aligning the two data distributions. In robotics, DA is used to take advantage of automatically generated synthetic data, that come with “free” annotation, to make effective predictions on real data. However, existing DA methods are not designed to cope with the multi-modal nature of RGB-D data, which are widely used in robotic vision. We propose a novel RGB-D DA method that reduces the synthetic-to-real domain shift by exploiting the inter-modal relation between the RGB and depth image. Our method consists of training a convolutional neural network to solve, in addition to the main recognition task, the pretext task of predicting the relative rotation between the RGB and depth image. To evaluate our method and encourage further research in this area, we define two benchmark datasets for object categorization and instance recognition. With extensive experiments, we show the benefits of leveraging the inter-modal relations for RGB-D DA. The code is available at: “https://github.com/MRLoghmani/relative-rotation”.

中文翻译：

---

RGB-D 对象识别的模态间旋转的无监督域适应

无监督域适应 (DA) 利用对标签丰富的源数据集的监督，通过对齐两个数据分布来对未标记的目标数据集进行预测。在机器人技术中，DA 用于利用带有“免费”注释的自动生成的合成数据，对真实数据进行有效预测。然而，现有的 DA 方法并不是为了应对 RGB-D 数据的多模态特性而设计的，RGB-D 数据广泛用于机器人视觉。我们提出了一种新颖的 RGB-D DA 方法，该方法通过利用 RGB 和深度图像之间的模态间关系来减少合成域到真实域的偏移。我们的方法包括训练卷积神经网络以解决除主要识别任务之外的预测 RGB 和深度图像之间的相对旋转的借口任务。为了评估我们的方法并鼓励在该领域的进一步研究，我们为对象分类和实例识别定义了两个基准数据集。通过大量实验，我们展示了利用 RGB-D DA 的模态间关系的好处。该代码位于：“https://github.com/MRLoghmani/relative-rotation”。