360$^o$ video is an essential component of VR/AR/MR systems that provides immersive experience to the users. However, 360$^o$ video is associated with high bandwidth requirements. The required bandwidth can be reduced by exploiting the fact that users are interested in viewing only a part of the video scene and that users request viewports that overlap with each other. Motivated by the findings of recent works where the benefits of caching video tiles at edge servers instead of caching entire 360$^o$ videos were shown, in this paper, we introduce the concept of virtual viewports that have the same number of tiles with the original viewports. The tiles forming these viewports are the most popular ones for each video and are determined by the users' requests. Then, we propose a proactive caching scheme that assumes unknown videos' and viewports' popularity. Our scheme determines which videos to cache as well as which is the optimal virtual viewport per video. Virtual viewports permit to lower the dimensionality of the cache optimization problem. To solve the problem, we first formulate the content placement of 360$^o$ videos in edge cache networks as a Markov Decision Process (MDP), and then we determine the optimal caching placement using the Deep Q-Network (DQN) algorithm. The proposed solution aims at maximizing the overall quality of the 360$^o$ videos delivered to the end-users by caching the most popular 360$^o$ videos at base quality along with a virtual viewport in high quality. We extensively evaluate the performance of the proposed system and compare it with that of known systems such as LFU, LRU, FIFO, over both synthetic and real 360$^o$ video traces. The results reveal the large benefits coming from proactive caching of virtual viewports instead of the original ones in terms of the overall quality of the rendered viewports, the cache hit ratio, and the servicing cost.

中文翻译：

---

用于 360$^o$ 视频缓存的视口感知深度强化学习方法

360$^o$ 视频是 VR/AR/MR 系统的重要组成部分，可为用户提供身临其境的体验。然而，360$^o$ 视频与高带宽要求相关。通过利用用户只对观看视频场景的一部分感兴趣并且用户请求彼此重叠的视口这一事实，可以减少所需的带宽。最近工作的发现显示了在边缘服务器缓存视频块而不是缓存整个 360$^o$ 视频的好处，在本文中，我们引入了虚拟视口的概念，该概念具有与原始视口。形成这些视口的图块是每个视频中最受欢迎的图块，由用户的请求决定。然后，我们提出了一种假设未知视频的主动缓存方案 和视口的受欢迎程度。我们的方案确定要缓存哪些视频以及每个视频的最佳虚拟视口。虚拟视口允许降低缓存优化问题的维度。为了解决这个问题，我们首先将边缘缓存网络中 360$^o$ 视频的内​​容放置制定为马尔可夫决策过程 (MDP)，然后我们使用深度 Q 网络 (DQN) 算法确定最佳缓存放置。所提议的解决方案旨在通过以基本质量缓存最受欢迎的 360$^o$ 视频以及高质量的虚拟视口，最大限度地提高交付给最终用户的 360$^o$ 视频的整体质量。我们广泛评估了所提出系统的性能，并将其与已知系统（如 LFU、LRU、FIFO）在合成和真实 360$^o$ 视频轨迹上的性能进行比较。