Both mobile edge cloud (MEC) and software-defined networking (SDN) are technologies for next generation mobile networks. In this paper, we propose to simultaneously optimize energy consumption and quality of experience (QoE) metrics in video streaming over software-defined mobile networks (SDMN) combined with MEC. Specifically, we propose a novel mechanism to jointly consider buffer dynamics, video quality adaption, edge caching, video transcoding and transmission. First, we assume that the time-varying channel is a discrete-time Markov chain (DTMC). Then, based on this assumption, we formulate two optimization problems which can be depicted as a constrained Markov decision process (CMDP) and a Markov decision process (MDP). Then, we transform the CMDP problem into regular MDP by deploying Lyapunov technique. We utilize asynchronous advantage actor-critic (A3C) algorithm, one of the model-free deep reinforcement learning (DRL) methods, to solve the corresponding MDP issues. Simulation results are presented to show that the proposed scheme can achieve the goal of energy saving and QoE enhancement with the corresponding constraints satisfied.

中文翻译：

---

软件定义移动网络上具有边缘缓存和视频转码的自适应视频流：一种深度强化学习方法

移动边缘云 (MEC) 和软件定义网络 (SDN) 都是下一代移动网络的技术。在本文中，我们建议在结合 MEC 的软件定义移动网络 (SDMN) 上同时优化视频流中的能耗和体验质量 (QoE) 指标。具体来说，我们提出了一种新机制来联合考虑缓冲区动态、视频质量自适应、边缘缓存、视频转码和传输。首先，我们假设时变信道是离散时间马尔可夫链（DTMC）。然后，基于这个假设，我们制定了两个优化问题，可以将其描述为约束马尔可夫决策过程（CMDP）和马尔可夫决策过程（MDP）。然后，我们通过部署 Lyapunov 技术将 CMDP 问题转换为常规 MDP。我们利用异步优势actor-critic（A3C）算法，一种无模型深度强化学习（DRL）方法，来解决相应的MDP问题。仿真结果表明，所提出的方案可以在满足相应约束的情况下实现节能和QoE增强的目标。