Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form. So, how should one go about finding a morphology fit for solving tasks in a given environment? Current approaches that co-adapt morphology and behavior use a specific task's reward as a signal for morphology optimization. However, this often requires expensive policy optimization and results in task-dependent morphologies that are not built to generalize. In this work, we propose a new approach, Task-Agnostic Morphology Evolution (TAME), to alleviate both of these issues. Without any task or reward specification, TAME evolves morphologies by only applying randomly sampled action primitives on a population of agents. This is accomplished using an information-theoretic objective that efficiently ranks agents by their ability to reach diverse states in the environment and the causality of their actions. Finally, we empirically demonstrate that across 2D, 3D, and manipulation environments TAME can evolve morphologies that match the multi-task performance of those learned with task supervised algorithms. Our code and videos can be found at https://sites.google.com/view/task-agnostic-evolution.

中文翻译：

---

与任务无关的形态演变

深度强化学习主要集中于学习行为，通常忽略了主体功能很大程度上由形式决定这一事实。因此，应该如何找到一种适合在给定环境中解决任务的形态？当前，形态和行为共同适应的方法使用特定任务的奖励作为形态优化的信号。但是，这通常需要进行昂贵的策略优化，并且会导致无法建立依赖于任务的形态。在这项工作中，我们提出了一种新方法，即与任务无关的形态演化（TAME），以缓解这两个问题。在没有任何任务或奖励说明的情况下，TAME通过仅将随机采样的动作原语应用到代理群体上来演化形态。这是通过使用信息理论的目标来实现的，该目标通过根据代理达到环境中各种状态的能力以及其行为的因果关系来对代理进行有效地排名。最后，我们凭经验证明，在2D，3D和操纵环境中，TAME可以演变出与通过任务监督算法学习的多任务性能相匹配的形态。我们的代码和视频可在https://sites.google.com/view/task-agnostic-evolution上找到。