Recent problems in robotics can sometimes only be tackled using machine learning technologies, particularly those that utilize deep learning (DL) with transfer learning. Transfer learning takes advantage of pretrained models, which are later fine-tuned using smaller task-specific datasets. The fine-tuned models must be robust against changes in environmental factors such as illumination since, often, there is no guarantee for them to be constant. Although synthetic data for pretraining has been shown to enhance DL model generalization, there is limited research on its application for fine-tuning. One limiting factor is that the generation and annotation of synthetic datasets can be cumbersome and impractical for the purpose of fine-tuning. To address this issue, we propose two methods for automatically generating annotated image datasets for object segmentation, one for real-world and another for synthetic images. We also introduce a novel domain adaptation approach called filling the reality gap (FTRG), which can blend elements from real-world and synthetic scenes in a single image to achieve domain adaptation. We demonstrate through experimentation on a representative robot application that FTRG outperforms other domain adaptation techniques, such as domain randomization or photorealistic synthetic images, in creating robust models. Furthermore, we evaluate the benefits of using synthetic data for fine-tuning in transfer learning and continual learning with experience replay using our proposed methods and FTRG. Our findings indicate that fine-tuning with synthetic data can produce superior results compared to solely using real-world data.

中文翻译：

---

提高用于对象分割的深度学习模型的鲁棒性：用于混合自动注释的真实数据和合成数据的框架。

机器人技术中的最新问题有时只能使用机器学习技术来解决，尤其是那些利用深度学习 (DL) 和迁移学习的技术。迁移学习利用预训练模型，这些模型稍后会使用较小的特定于任务的数据集进行微调。经过微调的模型必须能够抵抗光照等环境因素的变化，因为通常无法保证它们保持不变。尽管用于预训练的合成数据已被证明可以增强 DL 模型的泛化能力，但对其在微调中的应用研究有限。一个限制因素是合成数据集的生成和注释对于微调目的来说可能很麻烦且不切实际。为了解决这个问题，我们提出了两种自动生成用于对象分割的带注释图像数据集的方法，一种用于真实世界，另一种用于合成图像。我们还介绍了一种称为填充现实差距 (FTRG) 的新型域适应方法，它可以将来自现实世界和合成场景的元素混合在单个图像中以实现域适应。我们通过对代表性机器人应用程序的实验证明，在创建稳健模型方面，FTRG 优于其他域适应技术，例如域随机化或逼真合成图像。此外，我们使用我们提出的方法和 FTRG 评估了使用合成数据微调迁移学习和持续学习以及经验回放的好处。