We propose a novel method for exploring the dynamics of physically based animated characters, and learning a task-agnostic action space that makes movement optimization easier. Like several previous papers, we parameterize actions as target states, and learn a short-horizon goal-conditioned low-level control policy that drives the agent's state towards the targets. Our novel contribution is that with our exploration data, we are able to learn the low-level policy in a generic manner and without any reference movement data. Trained once for each agent or simulation environment, the policy improves the efficiency of optimizing both trajectories and high-level policies across multiple tasks and optimization algorithms. We also contribute novel visualizations that show how using target states as actions makes optimized trajectories more robust to disturbances; this manifests as wider optima that are easy to find. Due to its simplicity and generality, our proposed approach should provide a building block that can improve a large variety of movement optimization methods and applications.

中文翻译：

---

学习用于运动优化的与任务无关的动作空间

我们提出了一种新方法来探索基于物理的动画角色的动态，并学习与任务无关的动作空间，使运动优化更容易。与之前的几篇论文一样，我们将动作参数化为目标状态，并学习了一个短视域目标条件下的低级控制策略，该策略将代理的状态推向目标。我们的新贡献是，利用我们的探索数据，我们能够以通用方式学习低级策略，而无需任何参考运动数据。该策略为每个代理或模拟环境训练一次，提高了优化跨多个任务和优化算法的轨迹和高级策略的效率。我们还提供了新颖的可视化，展示了如何使用目标状态作为动作使优化的轨迹对干扰更加鲁棒；这表现为更容易找到的更广泛的最优解。由于其简单性和通用性，我们提出的方法应该提供一个可以改进各种运动优化方法和应用的构建块。