With the incremental deployment of software defined networking, the routing algorithms have gained more power on observability and controllability. Deep reinforcement learning, as an experience-driven approach, shows considerable potential in routing problem with the help of the centralized controller. It is an adaptive, lightweight, and model-free approach to coping with dynamic runtime status, large-scale traffic, and heterogeneous objective of SDN routing. However, it is still not suitable for the variable and complex emerging networks, because the huge training cost prevents fast convergence in a varying or discrepant environment. In this paper, we propose a transfer reinforcement learning algorithm to improve the training efficiency, and handle the variation in network status and topology. Specifically, we leverage the generative adversarial network to learn domain-invariant features that is suitable for deep reinforcement learning-based routing in different network environments. This mechanism utilizes the previous model and accelerates the training process. We implement our routing algorithm in the production level software switches and controller, while evaluating it comprehensively with many topologies and network status distributions. The experimental results show that our work not only outperforms the state-of-the-art deep reinforcement learning-based routing frameworks, but also has more training efficiency than the naive transfer learning algorithm both on different topologies and network status distributions.

中文翻译：

---

基于生成对抗网络的传输强化学习，用于利用先验知识进行路由


随着软件定义网络的增量部署，路由算法在可观测性和可控性方面获得了更多的能力。深度强化学习作为一种经验驱动的方法，在集中控制器的帮助下在路由问题上显示出巨大的潜力。它是一种自适应、轻量级、无模型的方法，用于应对 SDN 路由的动态运行状态、大规模流量和异构目标。然而，它仍然不适合可变和复杂的新兴网络，因为巨大的训练成本阻碍了在变化或差异环境中的快速收敛。在本文中，我们提出了一种迁移强化学习算法来提高训练效率，并处理网络状态和拓扑的变化。具体来说，我们利用生成对抗网络来学习适用于不同网络环境中基于深度强化学习的路由的域不变特征。该机制利用了之前的模型并加速了训练过程。我们在生产级软件交换机和控制器中实施我们的路由算法，同时使用许多拓扑和网络状态分布对其进行全面评估。实验结果表明，我们的工作不仅优于最先进的基于深度强化学习的路由框架，而且在不同的拓扑和网络状态分布上都比朴素的迁移学习算法具有更高的训练效率。