Currently, huge amounts of data are produced by edge device. Considering the heavy burden of network bandwidth and the service delay requirements of delay-sensitive applications, processing the data at network edge is a great choice. However, edge devices such as smart wearables, connected and autonomous vehicles usually have several limitations on computational capacity and energy which will influence the quality of service. As an effective and efficient strategy, offloading is widely used to address this issue. But when facing device heterogeneity problem and task complexity increase, service quality degradation and resource utility decrease often occur due to unreasonable task distribution. Since conventional simplex offloading strategies show limited performance in complex environment, we are motivated to design a dynamic regional resource scheduling framework which is able to work effectively taking different indexes into consideration. Thus, in this article we first propose a double offloading framework to simulate the offloading process in real edge scenario which consists of different edge servers and devices. Then we formulate the offloading as a Markov Decision Process (MDP) and utilize a deep reinforcement learning (DRL) algorithm named asynchronous advantage actor-critic (A3C) as the offloading decision making strategy to balance the workload of edge servers and finally reduce the overhead in terms of energy and time. Comparison experiments for local computing and wide-used DRL algorithm DQN are conducted in a comprehensive benchmark and the results show that our work performs much better on self-adjusting and overhead reduction.

中文翻译：

---

A3C-DO：边缘场景下基于深度强化学习的区域资源调度框架

目前，大量的数据是由边缘设备产生的。考虑到网络带宽的沉重负担和对延迟敏感的应用的服务延迟要求，在网络边缘处理数据是一个很好的选择。然而，智能可穿戴设备、联网和自动驾驶汽车等边缘设备通常在计算能力和能源方面存在一些限制，这会影响服务质量。作为一种有效且高效的策略，卸载被广泛用于解决这个问题。但是当面临设备异构问题和任务复杂度增加时，由于任务分配不合理，往往会出现服务质量下降和资源效用下降。由于传统的单纯形卸载策略在复杂环境中表现出有限的性能，我们有动力设计一个动态的区域资源调度框架，该框架能够有效地考虑不同的指标。因此，在本文中，我们首先提出了一个双重卸载框架来模拟由不同边缘服务器和设备组成的真实边缘场景中的卸载过程。然后我们将卸载制定为马尔可夫决策过程（MDP），并利用名为异步优势演员-评论家（A3C）的深度强化学习（DRL）算法作为卸载决策策略来平衡边缘服务器的工作负载并最终降低开销在能量和时间方面。