Content caching is an effective technique to alleviate the burden on backhaul links and reduce the traffic in cellular networks. In converged networks, broadcasting networks can push non-real-time popular services to different types of terminals during off-peak hours, while cellular networks are deployed to meet personalized needs. However, the storage capacity of terminals are usually limited. In this context, we study on the converged networks to push and cache the popular services in the router nodes close to the terminators. In this scheme, the most popular services are transmitted by the broadcasting base station and cached in the router nodes in a distributed cache network. Then, users can access the cached services in a more energy efficient manner. Due to the limited storage capacity of the router node, we assume that the user can access the cached services within two hops. Then, we formulate the service scheduling problem as a Markov Decision Process, aiming to maximize equivalent throughput (ET). Due to the large state space involved in the distributed cache network, it is quite challenging to obtain a tractable solution by the classical optimization algorithms. To handle this problem, a deep reinforcement learning based framework is proposed to tackle this problem. The simulation results show that the proposed algorithms are very effective; and they outperform the conventional one in term of the ET, especially when the users in the network subject to Poisson point process distribution.

中文翻译：

---

融合网络中的分布式缓存：一种深度强化学习方法

内容缓存是减轻回程链路负担和减少蜂窝网络流量的有效技术。在融合网络中，广播网络可以在非高峰时段向不同类型的终端推送非实时的热门业务，而蜂窝网络则是为了满足个性化需求。然而，终端的存储容量通常是有限的。在这种情况下，我们研究了融合网络，以在靠近终结器的路由器节点中推送和缓存流行的服务。在该方案中，最流行的服务由广播基站传输并缓存在分布式缓存网络中的路由器节点中。然后，用户可以以更节能的方式访问缓存的服务。由于路由器节点的存储容量有限，我们假设用户可以在两跳内访问缓存的服务。然后，我们将服务调度问题表述为马尔可夫决策过程，旨在最大化等效吞吐量 (ET)。由于分布式缓存网络中涉及的状态空间很大，因此通过经典优化算法获得易于处理的解决方案非常具有挑战性。为了解决这个问题，提出了一个基于深度强化学习的框架来解决这个问题。仿真结果表明，所提算法非常有效；并且它们在ET方面优于传统的，特别是当网络中的用户受泊松点过程分布时。由于分布式缓存网络中涉及的状态空间很大，因此通过经典优化算法获得易于处理的解决方案非常具有挑战性。为了解决这个问题，提出了一个基于深度强化学习的框架来解决这个问题。仿真结果表明，所提算法非常有效；并且它们在ET方面优于传统的，特别是当网络中的用户受泊松点过程分布时。由于分布式缓存网络中涉及的状态空间很大，因此通过经典优化算法获得易于处理的解决方案非常具有挑战性。为了解决这个问题，提出了一个基于深度强化学习的框架来解决这个问题。仿真结果表明，所提算法非常有效；并且它们在ET方面优于传统的，特别是当网络中的用户受泊松点过程分布时。