Internal computational models of physical bodies are fundamental to the ability of robots and animals alike to plan and control their actions. These “self-models” allow robots to consider outcomes of multiple possible future actions without trying them out in physical reality. Recent progress in fully data-driven self-modeling has enabled machines to learn their own forward kinematics directly from task-agnostic interaction data. However, forward kinematic models can only predict limited aspects of the morphology, such as the position of end effectors or velocity of joints and masses. A key challenge is to model the entire morphology and kinematics without prior knowledge of what aspects of the morphology will be relevant to future tasks. Here, we propose that instead of directly modeling forward kinematics, a more useful form of self-modeling is one that could answer space occupancy queries, conditioned on the robot’s state. Such query-driven self-models are continuous in the spatial domain, memory efficient, fully differentiable, and kinematic aware and can be used across a broader range of tasks. In physical experiments, we demonstrate how a visual self-model is accurate to about 1% of the workspace, enabling the robot to perform various motion planning and control tasks. Visual self-modeling can also allow the robot to detect, localize, and recover from real-world damage, leading to improved machine resiliency.

中文翻译：

---

机器人形态的全身视觉自建模

物理身体的内部计算模型是机器人和动物等计划和控制其行为的能力的基础。这些“自我模型”允许机器人考虑多种可能的未来行动的结果，而无需在物理现实中进行尝试。完全数据驱动的自我建模的最新进展使机器能够直接从与任务无关的交互数据中学习自己的正向运动学。然而，正向运动学模型只能预测形态的有限方面，例如末端执行器的位置或关节和质量的速度。一个关键挑战是在不事先了解形态的哪些方面与未来任务相关的情况下对整个形态和运动学进行建模。在这里，我们建议不要直接建模正向运动学，一种更有用的自我建模形式是可以根据机器人的状态回答空间占用查询。这种查询驱动的自我模型在空间域中是连续的、内存高效的、完全可微分的和运动学感知的，并且可以用于更广泛的任务。在物理实验中，我们展示了视觉自模型如何精确到大约 1% 的工作空间，使机器人能够执行各种运动规划和控制任务。视觉自建模还可以让机器人检测、定位并从现实世界的损坏中恢复，从而提高机器的弹性。和运动学感知，可用于更广泛的任务。在物理实验中，我们展示了视觉自模型如何精确到大约 1% 的工作空间，使机器人能够执行各种运动规划和控制任务。视觉自建模还可以让机器人检测、定位并从现实世界的损坏中恢复，从而提高机器的弹性。和运动学感知，可用于更广泛的任务。在物理实验中，我们展示了视觉自模型如何精确到大约 1% 的工作空间，使机器人能够执行各种运动规划和控制任务。视觉自建模还可以让机器人检测、定位并从现实世界的损坏中恢复，从而提高机器的弹性。