Programming a robot to deal with open-ended tasks remains a challenge, in particular if the robot has to manipulate objects. Launching, grasping, pushing or any other object interaction can be simulated but the corresponding models are not reversible and the robot behavior thus cannot be directly deduced. These behaviors are hard to learn without a demonstration as the search space is large and the reward sparse. We propose a method to autonomously generate a diverse repertoire of simple object interaction behaviors in simulation. Our goal is to bootstrap a robot learning and development process with limited informations about what the robot has to achieve and how. This repertoire can be exploited to solve different tasks in reality thanks to a proposed adaptation method or could be used as a training set for data-hungry algorithms. The proposed approach relies on the definition of a goal space and generates a repertoire of trajectories to reach attainable goals, thus allowing the robot to control this goal space. The repertoire is built with an off-the-shelf simulation thanks to a quality diversity algorithm. The result is a set of solutions tested in simulation only. It may result in two different problems: (1) as the repertoire is discrete and finite, it may not contain the trajectory to deal with a given situation or (2) some trajectories may lead to a behavior in reality that differs from simulation because of a reality gap. We propose an approach to deal with both issues by using a local linearization between the motion parameters and the observed effects. Furthermore, we present an approach to update the existing solution repertoire with the tests done on the real robot. The approach has been validated on two different experiments on the Baxter robot: a ball launching and a joystick manipulation tasks.

中文翻译：

---

从探索到控制：通过新奇搜索和局部适应学习对象操作技能

对机器人进行编程以处理开放式任务仍然是一个挑战，尤其是在机器人必须操纵物体的情况下。可以模拟发射、抓取、推动或任何其他物体交互，但相应的模型不可逆，因此无法直接推导出机器人的行为。这些行为在没有演示的情况下很难学习，因为搜索空间很大且奖励稀疏。我们提出了一种在模拟中自主生成各种简单对象交互行为的方法。我们的目标是利用有限的关于机器人必须实现什么以及如何实现的信息来引导机器人学习和开发过程。由于提出的自适应方法，可以利用该曲目来解决现实中的不同任务，或者可以用作数据饥渴算法的训练集。所提出的方法依赖于目标空间的定义，并生成一系列轨迹以达到可实现的目标，从而允许机器人控制该目标空间。由于质量多样性算法，曲目是用现成的模拟构建的。结果是一组仅在模拟中测试的解决方案。这可能会导致两个不同的问题：（1）由于曲目是离散的和有限的，它可能不包含处理给定情况的轨迹或（2）某些轨迹可能导致现实中的行为与模拟不同，因为现实差距。我们提出了一种通过在运动参数和观察到的效果之间使用局部线性化来处理这两个问题的方法。此外，我们提出了一种通过在真实机器人上进行的测试来更新现有解决方案库的方法。该方法已在 Baxter 机器人的两个不同实验中得到验证：球发射和操纵杆操作任务。