Training agents to autonomously learn how to use anthropomorphic robotic hands has the potential to lead to systems capable of performing a multitude of complex manipulation tasks in unstructured and uncertain environments. In this work, we first introduce a suite of challenging simulated manipulation tasks that current reinforcement learning and trajectory optimisation techniques find difficult. These include environments where two simulated hands have to pass or throw objects between each other, as well as an environment where the agent must learn to spin a long pen between its fingers. We then introduce a simple trajectory optimisation that performs significantly better than existing methods on these environments. Finally, on the challenging PenSpin task we combine sub-optimal demonstrations generated through trajectory optimisation with off-policy reinforcement learning, obtaining performance that far exceeds either of these approaches individually, effectively solving the environment.

中文翻译：

---

使用轨迹优化和强化学习解决具有挑战性的灵巧操作任务

训练代理自主学习如何使用拟人机械手有可能导致系统能够在非结构化和不确定的环境中执行多种复杂的操作任务。在这项工作中，我们首先介绍了一套具有挑战性的模拟操作任务，当前的强化学习和轨迹优化技术发现这些任务很困难。其中包括两只模拟手必须在彼此之间传递或投掷物体的环境，以及代理必须学会在手指之间旋转长笔的环境。然后，我们引入了一个简单的轨迹优化，它在这些环境中的性能明显优于现有方法。最后，