Data-center networks (DCN) possess multiple new features: coexistence of elephant flow/mice flow/coflow, and coexistence of multiple network resources (bandwidth, cache and computing). The cache should be a factor of effecting routing decision because it can eliminate redundant traffic in DCN. However, the conventional routing schemes cannot learn from their previous experiences regarding network abnormalities (such as, congestion), and their metric are still the single link state (such as, hop, distance, and cost) which does not include the effect of cache. Thus, they cannot enough efficiently allocate these resources to well meet the performance requirements for various flow types. Therefore, this paper proposes deep reinforcement learning-based routing (DRL-R). Firstly, we propose a method that recombines multiple network resources with different metrics, where we recombine cache and bandwidth by quantifying their contribution score in reducing the delay. Secondly, we propose a routing scheme with resource-recombined state. By optimally allocating network resources for traffic, a DRL agent deployed on a software-defined networking (SDN) controller continually interacts with the network to adaptively perform reasonable routing according to the network state. We employ deep Q-network (DQN) and deep deterministic policy gradient (DDPG) to build the DRL-R. Finally, we demonstrate the effectiveness of DRL-R through extensive simulations. Benefitting from continuous learning with a global view, DRL-R has lower flow completion time, higher throughput and better load balance as well as better robustness, compared to OSPF. In addition, because it efficiently utilizes the network resources, DRL-R can also outperform another DRL-based routing scheme (namely TIDE). Compared to OSPF and TIDE, respectively, DRL-R can improve throughput by up to 40% and 18.5%; DRL-R can reduce flow completion time by up to 47% and 39%; DRL-R can improve the link load balance by up to 18.8% and 9.3%. Additionally, we observed that DDPG has better performance than DQN.

数据中心网络（DCN）具有多个新功能：大象流/小鼠流/并流的共存，以及多种网络资源（带宽，缓存和计算）的共存。缓存应该是影响路由决策的因素，因为它可以消除DCN中的冗余流量。但是，常规路由方案无法从其先前的有关网络异常（例如拥塞）的经验中学习，并且它们的度量标准仍然是单个链路状态（例如跃点，距离和成本），其中不包括缓存的影响。因此，他们不能足够有效地分配这些资源来很好地满足各种流类型的性能要求。因此，本文提出了基于深度强化学习的路由（DRL-R）。首先，我们提出了一种将多个网络资源与不同指标重新组合的方法，其中我们通过量化缓存和带宽的贡献分数来减少延迟，从而重新组合了缓存和带宽。其次，我们提出了一种具有资源重组状态的路由方案。通过为流量最佳分配网络资源，部署在软件定义网络（SDN）控制器上的DRL代理会与网络连续交互，以根据网络状态自适应地执行合理的路由。我们采用深度Q网络（DQN）和深度确定性策略梯度（DDPG）来构建DRL-R。最后，我们通过广泛的仿真演示了DRL-R的有效性。受益于全局视野的持续学习，DRL-R具有更短的流程完成时间，更高的吞吐量，更好的负载平衡以及更强的鲁棒性，与OSPF相比。另外，由于DRL-R有效地利用了网络资源，因此其性能也可以胜过其他基于DRL的路由方案（即TIDE）。与OSPF和TIDE相比，DRL-R分别可以将吞吐量提高40％和18.5％。DRL-R可以将流完成时间分别减少47％和39％；DRL-R可以将链路负载平衡分别提高18.8％和9.3％。此外，我们观察到DDPG的性能优于DQN。