We present a strategy for simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation–reality gap, we propose a method for increasing the sample efficiency and robustness of existing simulation-to-real approaches which exploits hierarchy and online adaptation. Instead of learning a unique policy for each desired robotic task, we learn a diverse set of skills and their variations, and embed those skill variations in a continuously parameterized space. We then interpolate, search, and plan in this space to find a transferable policy which solves more complex, high-level tasks by combining low-level skills and their variations. In this work, we first characterize the behavior of this learned skill space, by experimenting with several techniques for composing pre-learned latent skills. We then discuss an algorithm which allows our method to perform long-horizon tasks never seen in simulation, by intelligently sequencing short-horizon latent skills. Our algorithm adapts to unseen tasks online by repeatedly choosing new skills from the latent space, using live sensor data and simulation to predict which latent skill will perform best next in the real world. Importantly, our method learns to control a real robot in joint-space to achieve these high-level tasks with little or no on-robot time, despite the fact that the low-level policies may not be perfectly transferable from simulation to real, and that the low-level skills were not trained on any examples of high-level tasks. In addition to our results indicating a lower sample complexity for families of tasks, we believe that our method provides a promising template for combining learning-based methods with proven classical robotics algorithms such as model-predictive control.

中文翻译：

---

通过学习机器人技能的潜在空间来扩展模拟到真实的转移

我们提出了一种模拟到真实转移的策略，它建立在机器人技能分解的最新进展之上。我们不是专注于最小化模拟与现实的差距，而是提出了一种方法来提高现有模拟到现实方法的样本效率和鲁棒性，该方法利用层次结构和在线适应。我们不是为每个所需的机器人任务学习独特的策略，而是学习一组不同的技能及其变化，并将这些技能变化嵌入到连续参数化的空间中。然后，我们在这个空间中进行插值、搜索和规划，以找到一种可转移的策略，该策略通过结合低级技能及其变体来解决更复杂的高级任务。在这项工作中，我们首先描述了这个学习技能空间的行为，通过尝试多种技术来组合预先学习的潜在技能。然后，我们讨论了一种算法，该算法允许我们的方法通过智能排序短期潜在技能来执行模拟中从未见过的长期任务。我们的算法通过从潜在空间中反复选择新技能，使用实时传感器数据和模拟来预测哪个潜在技能将在现实世界中表现最好，从而适应在线看不见的任务。重要的是，我们的方法学习控制关节空间中的真实机器人，以在很少或没有机器人时间的情况下完成这些高级任务，尽管低级策略可能无法完美地从模拟转移到真实，并且低级技能没有接受任何高级任务示例的培训。