Instead of relying on remote clouds, today’s Augmented Reality (AR) applications usually send videos to nearby edge servers for analysis (such as objection detection) so as to optimize the user’s quality of experience (QoE), which is often determined by not only detection latency but also detection accuracy, playback fluency, etc. Therefore, many studies have been conducted to help adaptively choose best video configuration, e.g., resolution and frame per second (fps), based on network bandwidth to further improve QoE. However, we notice that the video content itself has significant impacts on the configuration selection, e.g., the videos with high-speed objects must be encoded with a high fps to meet the user’s fluency requirement. In this article, we aim to adaptively select configurations that match the time-varying network condition as well as the video content. We design Cuttlefish, a system that generates video configuration decisions using reinforcement learning (RL). Cuttlefish trains a neural network model that picks a configuration for the next encoding slot based on observations collected by AR devices. Cuttlefish does not rely on any pre-programmed models or specific assumptions on the environments. Instead, it learns to make configuration decisions solely through observations of the resulting performance of historical decisions. Cuttlefish automatically learns the adaptive configuration policy for diverse AR video streams and obtains a gratifying QoE. We compared Cuttlefish to several state-of-the-art bandwidth-based and velocity-based methods using trace-driven and real world experiments. The results show that Cuttlefish achieves a 18.4-25.8 percent higher QoE than the others.

中文翻译：

---

墨鱼：实时增强现实中视频分析的神经配置适应

当今的增强现实 (AR) 应用程序不再依赖远程云，而是将视频发送到附近的边缘服务器进行分析（例如物体检测），以优化用户体验质量 (QoE)，而这通常不仅取决于检测延迟以及检测精度、播放流畅度等。因此，已经进行了许多研究来帮助自适应地选择最佳视频配置，例如基于网络带宽的分辨率和每秒帧数 (fps)，以进一步提高 QoE。但是，我们注意到视频内容本身对配置选择有重大影响，例如，具有高速对象的视频必须以高 fps 进行编码才能满足用户的流畅性要求。在本文中，我们的目标是自适应地选择与时变网络条件以及视频内容相匹配的配置。我们设计了 Cuttlefish，这是一个使用强化学习 (RL) 生成视频配置决策的系统。Cuttlefish 训练一个神经网络模型，该模型根据 AR 设备收集的观察结果为下一个编码槽选择配置。Cuttlefish 不依赖于任何预先编程的模型或对环境的特定假设。相反，它仅通过观察历史决策的结果性能来学习做出配置决策。Cuttlefish 针对不同的 AR 视频流自动学习自适应配置策略，获得令人满意的 QoE。我们使用跟踪驱动和真实世界的实验将 Cuttlefish 与几种最先进的基于带宽和基于速度的方法进行了比较。结果表明，Cuttlefish 的 QoE 比其他产品高 18.4-25.8%。