In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the agent's design that is better suited for its task, jointly with the policy. We propose an alteration to the popular OpenAI Gym framework, where we parameterize parts of an environment, and allow an agent to jointly learn to modify these environment parameters along with its policy. We demonstrate that an agent can learn a better structure of its body that is not only better suited for the task, but also facilitates policy learning. Joint learning of policy and structure may even uncover design principles that are useful for assisted-design applications.

中文翻译：

---

用于改进代理设计的强化学习

在许多强化学习任务中，目标是学习一种策略来操纵代理，其设计是固定的，以最大化累积奖励的一些概念。代理物理结构的设计很少针对手头的任务进行优化。在这项工作中，我们探索了与策略一起学习更适合其任务的代理设计版本的可能性。我们建议对流行的 OpenAI Gym 框架进行更改，在该框架中我们参数化环境的一部分，并允许代理共同学习修改这些环境参数及其策略。我们证明了代理可以学习更好的身体结构，这不仅更适合任务，而且有助于策略学习。