Humans can master a new task within a few trials by drawing upon skills acquired through prior experience. To mimic this capability, hierarchical models combining primitive policies learned from prior tasks have been proposed. However, these methods fall short comparing to the human's range of transferability. We propose a method, which leverages the hierarchical structure to train the combination function and adapt the set of diverse primitive polices alternatively, to efficiently produce a range of complex behaviors on challenging new tasks. We also design two regularization terms to improve the diversity and utilization rate of the primitives in the pre-training phase. We demonstrate that our method outperforms other recent policy transfer methods by combining and adapting these reusable primitives in tasks with continuous action space. The experiment results further show that our approach provides a broader transferring range. The ablation study also shows the regularization terms are critical for long range policy transfer. Finally, we show that our method consistently outperforms other methods when the quality of the primitives varies.

中文翻译：

---

迈向稳健的远程政策转移

通过利用以前的经验获得的技能，人类可以在几次试验中完成一项新任务。为了模仿这种能力，已经提出了结合从先前任务中学到的原始策略的分层模型。但是，与人类的可转移性范围相比，这些方法不足。我们提出了一种方法，该方法利用层次结构来训练组合功能并交替使用一组不同的原始策略，以有效地产生具有挑战性的新任务的一系列复杂行为。我们还设计了两个正则项，以提高预训练阶段中基元的多样性和利用率。我们证明，通过在连续动作空间的任务中组合和改编这些可重用原语，我们的方法优于其他最近的策略转移方法。实验结果进一步表明，我们的方法提供了更广的传输范围。消融研究还显示，正规化条款对于长期政策转移至关重要。最后，我们证明了当基元的质量发生变化时，我们的方法始终优于其他方法。