User equipment produces a series of tasks that are processed locally or remotely, falling into three categories: (i) local computing only, (ii) a fraction of the task is computed locally and the remaining task unprocessed is offloaded for remote computation, and (iii) the entire task is offloaded. Each case has attracted substantial attention in recent studies, where a delay-constrained non-linear optimization problem is often formulated. The solutions employed are either based on Lagrange duality, heuristic search, or dynamic programming. To our knowledge, there is no unifying task-processing orchestrator that is an online tailored solver for learning the model-free problems, encapsulating the three cases above. We fill this gap and present the first attempt on an innovative actor-critic reinforcement learning approach in consideration of the energy-efficiency, to compute the asymptotically optimal solutions via decomposing the comprehensive optimization into sub-problems. Rigorous theoretical analyses and experience-driven simulations demonstrate significant advantages over the benchmark approaches, in terms of task-processing delay, power efficiency, and convergence time.

用户设备产生一系列在本地或远程处理的任务，分为三类：（i）仅本地计算，（ii）一部分任务在本地计算，剩下的未处理任务被卸载以进行远程计算，并且（ iii）卸载了整个任务。在最近的研究中，每种情况都引起了人们的极大关注，其中经常提出延迟受限的非线性优化问题。所采用的解决方案基于拉格朗日对偶，启发式搜索或动态编程。据我们所知，没有统一的任务处理协调器是用于学习无模型问题的在线量身定制的求解器，它封装了上述三种情况。我们填补了这一空白，并提出了一种基于能源效率的，创新的行为者-批判强化学习方法的首次尝试，以通过将综合优化分解为子问题来计算渐近最优解。严格的理论分析和经验驱动的仿真证明，在任务处理延迟，电源效率和收敛时间方面，它们比基准方法具有明显优势。