Computation offloading decisions play a crucial role in implementing mobile-edge computing (MEC) technology in the Internet of Things (IoT) services. Mobile network operators (MNOs) can employ computation offloading techniques to reduce task completion delay and improve the Quality of Service (QoS) for users by optimizing the system’s processing delay and energy consumption. However, different IoT applications (e.g., entertainment and autonomous driving) generate different delay tolerances and benefits for computational tasks from the MNO perspective. Therefore, simply minimizing the delay of all tasks does not satisfy the QoS of each user. The system architecture design should consider the significance of users and the heterogeneity of tasks. Unfortunately, rare work has been done to discuss this practical issue. In this article, from the perspective of MNO, we investigate the computation offloading optimization problem of multiuser delay-sensitive tasks. First, we propose a new optimization model, which designs different optimization objectives for the cost and revenue of tasks. Then, we transform the problem into a Markov decision processes problem, which leads to designing a multiagent iterative optimization framework. For the strategic optimization of each agent, we further propose a cooperative multiagent deep reinforcement learning (CMDRL) algorithm to optimize two different objectives at the same time. Two agents are integrated into the CMDRL framework to enable agents to collaborate and converge to the global optimum in a distributed manner. At the same time, the priority experience replay method is introduced to improve the utilization rate of effective samples and the learning efficiency of the algorithm. The experimental results show that our proposed method can effectively achieve a significantly higher profit than the alternative state-of-the-art method and exhibit a more favorable computational performance than benchmark deep reinforcement learning methods.

中文翻译：

---

用于计算卸载的协作多智能体深度强化学习：移动网络运营商的视角


计算卸载决策在物联网 (IoT) 服务中实施移动边缘计算 (MEC) 技术方面发挥着至关重要的作用。移动网络运营商（MNO）可以采用计算卸载技术，通过优化系统的处理延迟和能耗来减少任务完成延迟并提高用户的服务质量（QoS）。然而，从 MNO 的角度来看，不同的物联网应用（例如娱乐和自动驾驶）会产生不同的延迟容限和计算任务的优势。因此，简单地最小化所有任务的延迟并不能满足每个用户的QoS。系统架构设计应考虑用户的重要性和任务的异构性。不幸的是，很少有工作来讨论这个实际问题。在本文中，我们从MNO的角度研究多用户延迟敏感任务的计算卸载优化问题。首先，我们提出了一种新的优化模型，针对任务的成本和收入设计了不同的优化目标。然后，我们将问题转化为马尔可夫决策过程问题，从而设计出多智能体迭代优化框架。对于每个智能体的策略优化，我们进一步提出了一种协作多智能体深度强化学习（CMDRL）算法来同时优化两个不同的目标。 CMDRL 框架中集成了两个智能体，使智能体能够以分布式方式协作并收敛到全局最优。同时引入优先经验回放方法，提高有效样本的利用率和算法的学习效率。 实验结果表明，我们提出的方法可以有效地实现比替代最先进方法显着更高的利润，并且比基准深度强化学习方法表现出更有利的计算性能。