As one of the most promising networks, the air-ocean integrated networks (AOINs) composed of unmanned aerial vehicles (UAVs) and unmanned surface vehicles (USVs) meet a variety of requests from different maritime missions with the characteristics of seamless, high-rate, and reliable transmission. However, due to the limited storage capacities of UAVs and uncertain navigation paths of USVs, it is challenging to accomplish the ocean observation mission. In this paper, a UAV and USV cooperative content offloading scheme in AOINs with Q-learning and game theory is proposed. Specifically, the state evaluation mechanism for both UAV and USV is first designed to make cooperation strategies. Afterward, the interaction between UAV and USV is modeled as the bargaining game, where the Nash equilibrium as the optimal transaction price is obtained by the backward induction method. To realize the maximization of revenue, we devise a Q-learning based algorithm to make path planning for each USV to offload contents as many as possible under the limited energy. Finally, the effectiveness and efficiency of the proposed scheme is conducted by extensive simulations.

作为最有前景的网络之一，由无人机（UAV）和无人水面舰艇（USV）组成的海空一体化网络（AOIN）以无缝、高速率的特点满足不同海上任务的多种需求。 ，并且传输可靠。然而，由于无人机的存储容量有限，且无人艇的导航路径不确定，完成海洋观测任务具有挑战性。在本文中，提出了一种基于 Q 学习和博弈论的 AOIN 中的无人机和 USV 协同内容卸载方案。具体来说，无人机和USV的状态评估机制首先是为了制定合作策略。之后，无人机和 USV 之间的交互被建模为讨价还价博弈，其中，作为最优交易价格的纳什均衡是通过逆向归纳法得到的。为了实现收益最大化，我们设计了一种基于 Q-learning 的算法，为每个 USV 进行路径规划，以在有限的能量下尽可能多地卸载内容。最后，通过广泛的模拟来验证所提出方案的有效性和效率。