Due to the advantages of flexible deployment and extensive coverage, unmanned aerial vehicles (UAVs) have significant potential for sensing applications in the next generation of cellular networks, which will give rise to a cellular Internet of UAVs. In this article, we consider a cellular Internet of UAVs, where the UAVs execute sensing tasks through cooperative sensing and transmission to minimize the age of information (AoI). However, the cooperative sensing and transmission is tightly coupled with the UAVs’ trajectories, which makes the trajectory design challenging. To tackle this challenge, we propose a distributed sense-and-send protocol, where the UAVs determine the trajectories by selecting from a discrete set of tasks and a continuous set of locations for sensing and transmission. Based on this protocol, we formulate the trajectory design problem for AoI minimization and propose a compound-action actor-critic (CA2C) algorithm to solve it based on deep reinforcement learning. The CA2C algorithm can learn the optimal policies for actions involving both continuous and discrete variables and is suited for the trajectory design. Our simulation results show that the CA2C algorithm outperforms four baseline algorithms. Also, we show that by dividing the tasks, cooperative UAVs can achieve a lower AoI compared to non-cooperative UAVs.

中文翻译：

---

1.4无人机协同互联网：多智能体深度强化学习的分布式轨迹设计

由于具有灵活部署和广泛覆盖的优势，无人机在下一代蜂窝网络中具有巨大的传感应用潜力，这将催生无人机的蜂窝互联网。在本文中，我们考虑了无人机的蜂窝互联网，其中无人机通过协作感知和传输执行感知任务，以最大限度地减少信息年龄 (AoI)。然而，协同传感和传输与无人机的轨迹紧密耦合，这使得轨迹设计具有挑战性。为了应对这一挑战，我们提出了一种分布式传感和发送协议，其中无人机通过从一组离散的任务和一组连续的传感和传输位置中进行选择来确定轨迹。基于这个协议，我们为 AoI 最小化制定了轨迹设计问题，并提出了一种基于深度强化学习的复合动作演员评论 (CA2C) 算法来解决该问题。CA2C 算法可以学习涉及连续和离散变量的动作的最佳策略，适用于轨迹设计。我们的仿真结果表明，CA2C 算法优于四种基线算法。此外，我们表明，通过划分任务，与非合作无人机相比，合作无人机可以实现更低的 AoI。我们的仿真结果表明，CA2C 算法优于四种基线算法。此外，我们表明，通过划分任务，与非合作无人机相比，合作无人机可以实现更低的 AoI。我们的仿真结果表明，CA2C 算法优于四种基线算法。此外，我们表明，通过划分任务，与非合作无人机相比，合作无人机可以实现更低的 AoI。