We consider a multi-user multi-server mobile edge computing (MEC) network with time-varying fading channels and formulate an offloading decision and resource allocation problem. To solve this mixed-integer non-convex problem, we propose two hybrid approaches that learn offloading strategy with DQN (opt-DQN) or Q-table (opt-QL) at each user equipment (UE). The communication resources are allocated with an optimization algorithm at each computational access point (CAP). We also propose a pure DQN method that learns both the offloading strategy and resource allocation via Q-learning (QL). We analyze the convergence behavior of the QL-based algorithms from a game-theoretical perspective and demonstrate the performance of the proposed hybrid approaches for different network sizes. The simulation results show that the hybrid approaches reach lower costs than other baseline algorithms and the pure-DQN approach. Moreover, the performance of the pure-DQN approach degrades severely as the network size increases, while opt-DQN still performs the best, followed by opt-QL. These observations demonstrate that the hybrid approach that combines the advantages of both QL and convex optimization is a promising design for a multi-user MEC network, wherein complicated offloading and resource allocation strategies need to be determined in a timely and accurate fashion.

中文翻译：

---

MEC 网络中策略和资源分配的混合 DQN 和优化方法

我们考虑具有时变衰落信道的多用户多服务器移动边缘计算 (MEC) 网络，并制定卸载决策和资源分配问题。为了解决这个混合整数非凸问题，我们提出了两种混合方法，它们在每个用户设备 (UE) 上使用 DQN (opt-DQN) 或 Q-table (opt-QL) 学习卸载策略。通信资源在每个计算接入点 (CAP) 处分配有优化算法。我们还提出了一种纯 DQN 方法，该方法通过 Q-learning (QL) 学习卸载策略和资源分配。我们从博弈论的角度分析了基于 QL 的算法的收敛行为，并展示了针对不同网络规模所提出的混合方法的性能。仿真结果表明，混合方法的成本低于其他基线算法和纯 DQN 方法。此外，随着网络规模的增加，纯 DQN 方法的性能严重下降，而 opt-DQN 仍然表现最好，其次是 opt-QL。这些观察表明，结合 QL 和凸优化优点的混合方法是多用户 MEC 网络的一种有前途的设计，其中需要及时准确地确定复杂的卸载和资源分配策略。