Predictive models have been at the core of many robotic systems, from quadrotors to walking robots. However, it has been challenging to develop and apply such models to practical robotic manipulation due to high-dimensional sensory observations such as images. Previous approaches to learning models in the context of robotic manipulation have either learned whole image dynamics or used autoencoders to learn dynamics in a low-dimensional latent state. In this work, we introduce model-based prediction with self-supervised visual correspondence learning, and show that not only is this indeed possible, but demonstrate that these types of predictive models show compelling performance improvements over alternative methods for vision-based RL with autoencoder-type vision training. Through simulation experiments, we demonstrate that our models provide better generalization precision, particularly in 3D scenes, scenes involving occlusion, and in category-generalization. Additionally, we validate that our method effectively transfers to the real world through hardware experiments. Videos and supplementary materials available at https://sites.google.com/view/keypointsintothefuture

中文翻译：

---

未来的关键点：基于模型的强化学习中的自监督通信

预测模型一直是许多机器人系统的核心，从四旋翼机器人到步行机器人。然而，由于图像等高维感官观察，开发此类模型并将其应用于实际机器人操作一直具有挑战性。以前在机器人操作背景下学习模型的方法要么学习了整个图像动态，要么使用自动编码器来学习低维潜在状态下的动态。在这项工作中，我们引入了基于模型的预测和自监督的视觉对应学习，并表明这不仅确实是可能的，而且证明这些类型的预测模型显示出与使用自动编码器的基于视觉的 RL 的替代方法相比具有引人注目的性能改进型视力训练。通过模拟实验，我们证明我们的模型提供了更好的泛化精度，特别是在 3D 场景、涉及遮挡的场景和类别泛化中。此外，我们通过硬件实验验证了我们的方法有效地转移到现实世界。https://sites.google.com/view/keypointsintothefuture 提供的视频和补充材料