Unmanned aerial vehicle (UAV)-assisted computation offloading allows mobile devices (MDs) to process computation-intensive and latency-sensitive tasks with limited or no-available infrastructures. To achieve long-term performance under changing environment, deep reinforcement-based methods have been applied to solve the UAV-assisted computation offloading problem. However, the deployment of multiple UAVs for computation offloading in mobile edge computing (MEC) network still faces the challenge of lacking flexible learning scheme to efficiently adjust computation offloading policy according to dynamic UAV mobility pattern and UAV failure. To this end, a distributed deep reinforcement learning (DRL)-based method with the cooperative exploring and prioritized experience replay (PER) is proposed in this paper. Our distributed exploring process achieves flexible learning scheme under UAV failure by allowing MDs to learning cost-efficient offloading policy cooperatively. Furthermore, PER allows MDs can explore the transitions with high TD-error, which can improve the performance under dynamic UAV mobility patterns. The efficiency of the proposed method is demonstrated by comparing with the existing computation offloading methods, and results show that the proposed method outperforms the compared methods in terms of convergence rate, energy-task efficiency and average processing time.

无人机 (UAV) 辅助计算卸载允许移动设备 (MD) 在有限或不可用的基础设施下处理计算密集型和延迟敏感的任务。为了在不断变化的环境下实现长期性能，基于深度强化的方法已被应用于解决无人机辅助计算卸载问题。然而，在移动边缘计算（MEC）网络中部署多个无人机进行计算卸载仍然面临着缺乏灵活的学习方案来根据动态无人机移动模式和无人机故障有效调整计算卸载策略的挑战。为此，本文提出了一种基于分布式深度强化学习（DRL）的协同探索和优先体验重放（PER）的方法。我们的分布式探索过程通过允许 MD 合作学习具有成本效益的卸载策略，在无人机故障下实现了灵活的学习方案。此外，PER 允许 MD 可以探索具有高 TD 误差的转换，这可以提高动态无人机移动模式下的性能。通过与现有计算卸载方法的比较证明了所提出方法的效率，结果表明，所提出的方法在收敛速度、能量任务效率和平均处理时间方面优于比较方法。