This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%–64% improved in terms of the cache hit ratio and 56%–59% decreased in terms of the average latency.

中文翻译：

---

基于深度强化学习的智能缓存内容更新策略

本文提出了一种在启用缓存的网络中基于DRL的缓存内容更新策略，以提高缓存命中率并减少平均延迟。与现有策略相反，在这项工作中考虑了更实际的缓存方案，其中内容请求随时间和位置而变化。考虑到有限的缓存容量的限制，将动态内容更新问题建模为马尔可夫决策过程（MDP）。除此之外，利用深度Q学习网络（DQN）算法来解决MDP问题。具体来说，神经网络经过优化可以近似从经验重播存储器中选择训练数据的值。DQN代理会为缓存决策得出最佳策略。与现有策略相比，仿真结果表明，我们提出的策略在缓存命中率方面提高了56％–64％，在平均延迟方面降低了56％–59％。