Some researchers have introduced transfer learning mechanisms to multiagent reinforcement learning (MARL). However, the existing works devoted to cross-task transfer for multiagent systems were designed just for homogeneous agents or similar domains. This work proposes an all-purpose cross-transfer method, called multiagent lateral transfer (MALT), assisting MARL with alleviating the training burden. We discuss several challenges in developing an all-purpose multiagent cross-task transfer learning method and provide a feasible way of reusing knowledge for MARL. In the developed method, we take features as the transfer object rather than policies or experiences, inspired by the progressive network. To achieve more efficient transfer, we assign pretrained policy networks for agents based on clustering, while an attention module is introduced to enhance the transfer framework. The proposed method has no strict requirements for the source task and target task. Compared with the existing works, our method can transfer knowledge among heterogeneous agents and also avoid negative transfer in the case of fully different tasks. As far as we know, this article is the first work denoted to all-purpose cross-task transfer for MARL. Several experiments in various scenarios have been conducted to compare the performance of the proposed method with baselines. The results demonstrate that the method is sufficiently flexible for most settings, including cooperative, competitive, homogeneous, and heterogeneous configurations.

中文翻译：

---

多智能体强化学习的横向迁移学习


一些研究人员将迁移学习机制引入多智能体强化学习（MARL）中。然而，现有的致力于多智能体系统跨任务传输的工作只是针对同构智能体或类似领域而设计的。这项工作提出了一种通用的交叉转移方法，称为多智能体横向转移（MALT），帮助 MARL 减轻训练负担。我们讨论了开发通用多智能体跨任务迁移学习方法的几个挑战，并为 MARL 提供了重用知识的可行方法。在开发的方法中，受渐进网络的启发，我们将特征而不是策略或经验作为传递对象。为了实现更有效的传输，我们为基于聚类的代理分配预训练的策略网络，同时引入注意模块来增强传输框架。该方法对源任务和目标任务没有严格的要求。与现有的工作相比，我们的方法可以在异构智能体之间传递知识，并且还可以避免在完全不同的任务情况下的负迁移。据我们所知，本文是第一篇针对 MARL 的通用跨任务传输的工作。已经在各种场景中进行了多次实验，以将所提出的方法与基线的性能进行比较。结果表明，该方法对于大多数设置都足够灵活，包括合作、竞争、同质和异构配置。