With the incremental deployment of software defined networking, the routing algorithms have gained more power on observability and controllability. Deep reinforcement learning, as an experience-driven approach, shows considerable potential in routing problem with the help of the centralized controller. It is an adaptive, lightweight, and model-free approach to coping with dynamic runtime status, large-scale traffic, and heterogeneous objective of SDN routing. However, it is still not suitable for the variable and complex emerging networks, because the huge training cost prevents fast convergence in a varying or discrepant environment. In this paper, we propose a transfer reinforcement learning algorithm to improve the training efficiency, and handle the variation in network status and topology. Specifically, we leverage the generative adversarial network to learn domain-invariant features that is suitable for deep reinforcement learning-based routing in different network environments. This mechanism utilizes the previous model and accelerates the training process. We implement our routing algorithm in the production level software switches and controller, while evaluating it comprehensively with many topologies and network status distributions. The experimental results show that our work not only outperforms the state-of-the-art deep reinforcement learning-based routing frameworks, but also has more training efficiency than the naive transfer learning algorithm both on different topologies and network status distributions.

中文翻译：

---

基于生成对抗网络的转移强化学习，用于具有先验知识的路由

随着软件定义网络的增量部署，路由算法在可观察性和可控性方面获得了更多的权力。深度强化学习作为一种经验驱动的方法，在集中控制器的帮助下在路由问题中显示出相当大的潜力。它是一种自适应、轻量级和无模型的方法，用于应对 SDN 路由的动态运行时状态、大规模流量和异构目标。然而，它仍然不适合可变复杂的新兴网络，因为巨大的训练成本阻碍了在变化或差异环境中的快速收敛。在本文中，我们提出了一种迁移强化学习算法来提高训练效率，并处理网络状态和拓扑结构的变化。具体来说，我们利用生成对抗网络来学习适用于不同网络环境中基于深度强化学习的路由的域不变特征。这种机制利用了以前的模型并加速了训练过程。我们在生产级软件交换机和控制器中实施我们的路由算法，同时通过许多拓扑和网络状态分布对其进行全面评估。实验结果表明，我们的工作不仅优于最先进的基于深度强化学习的路由框架，而且在不同的拓扑和网络状态分布上都比朴素的迁移学习算法具有更高的训练效率。