The emergence of Software Defined Network (SDN) provides a centralized and flexible approach to route network flows. Due to the technical and economic challenges in upgrading to a fully SDN-enabled network, hybrid SDN, with a partial deployment of SDN switches in a traditional network, has been a prevailing network architecture. Meanwhile, Traffic Engineering (TE) in the hydbrid SDN has attracted wide attentions from academia and industry. Previous studies on TE in the hybrid SDN are either traffic-oblivious or time-consuming, which causes routing schemes failed in responding to the dynamically-changing traffic rapidly and intelligently. Therefore, in this paper, we propose a Reinforcement Learning (RL) based method, which learns a traffic-splitting agent to address the dynamically-changing traffic and achieve the link load balancing in the hybrid SDN. Specifically, to rapidly and intelligently determine a routing scheme to the new traffic demands, a traffic-splitting agent is designed and learnt offline by exploiting the RL algorithm to establish the direct relationship between traffic demands and traffic-splitting policies. Once the traffic-splitting agent is learnt, the effective traffic-splitting policies, which are used to determine the traffic-splitting ratios on SDN switches, can be generated rapidly. Additionally, to meet the interactive requirements for learning a traffic-splitting agent, a reasonable simulation environment is proposed to be constructed to avoid routing loops when traffic-splitting policies are taken. Extensive evaluations on different topologies and real traffic demands demonstrate that the proposed method achieves the comparable network performance and performs superiorities in rapidly generating the satisfying routing schemes.

软件定义网络 (SDN) 的出现为路由网络流提供了一种集中且灵活的方法。由于升级到完全支持 SDN 的网络面临技术和经济挑战，在传统网络中部分部署 SDN 交换机的混合 SDN 一直是一种流行的网络架构。同时，混合SDN中的流量工程（TE）也引起了学术界和工业界的广泛关注。以往对混合SDN中TE的研究要么忽略流量，要么耗时，导致路由方案无法快速智能地响应动态变化的流量。因此，在本文中，我们提出了一种基于强化学习（RL）的方法，它学习了一个流量分流代理来处理动态变化的流量并实现混合 SDN 中的链路负载平衡。具体来说，为了快速、智能地确定新的流量需求的路由方案，设计流量分流代理并离线学习，利用RL算法建立流量需求与流量分流策略之间的直接关系。一旦学习了分流代理，就可以快速生成有效的分流策略，用于确定SDN交换机上的分流比。此外，为了满足学习分流代理的交互需求，建议构建合理的仿真环境，以避免在采取分流策略时出现路由循环。