Unmanned aerial vehicles (UAVs) are envisioned to play a key role in intelligent transportation systems to complement the communication infrastructure in future smart cities. UAV-assisted vehicular networking research typically adopts throughput and latency as the main performance metrics. These conventional metrics, however, are not adequate to reflect the freshness of the information, an attribute that has been recently identified as a critical requirement to enable services such as autonomous driving and accident prevention. In this paper, we consider a UAV-assisted single-hop vehicular network, wherein sensors (e.g., LiDARs and cameras) on vehicles generate time sensitive data streams, and UAVs are used to collect and process this data while maintaining a minimum age of information (AoI). We aim to jointly optimize the trajectories of UAVs and find scheduling policies to keep the information fresh under minimum throughput constraints. The formulated optimization problem is shown to be mixed integer non-linear program (MINLP) and generally hard to be solved. Motivated by the success of machine learning (ML) techniques particularly deep learning in solving complex problems with low complexity, we reformulate the trajectories and scheduling policies problem as a Markov decision process (MDP) where the system state space considers the vehicular network dynamics. Then, we develop deep reinforcement learning (DRL) to learn the vehicular environment and its dynamics in order to handle UAVs’ trajectory and scheduling policy. In particular, we leverage Deep Deterministic Policy Gradient (DDPG) for learning the trajectories of the deployed UAVs to efficiently minimize the Expected Weighted Sum AoI (EWSA). Simulations results demonstrate the effectiveness of the proposed design and show the deployed UAVs adapt their velocities during the data collection mission in order to minimize the AoI.

中文翻译：

---

智能交通系统中无人机​​的信息感知轨迹规划时代：一种深度学习方法

无人机 (UAV) 被设想在智能交通系统中发挥关键作用，以补充未来智能城市的通信基础设施。无人机辅助车载网络研究通常采用吞吐量和延迟作为主要性能指标。然而，这些传统指标不足以反映信息的新鲜度，这一属性最近被确定为实现自动驾驶和事故预防等服务的关键要求。在本文中，我们考虑了无人机辅助的单跳车载网络，其中车辆上的传感器（例如，激光雷达和摄像头）生成时间敏感的数据流，无人机用于收集和处理这些数据，同时保持信息的最小年龄(AOI)。我们的目标是联合优化无人机的轨迹并找到调度策略，以在最小吞吐量限制下保持信息新鲜。公式化的优化问题被证明是混合整数非线性规划 (MINLP)，通常很难解决。受机器学习 (ML) 技术，特别是深度学习在解决低复杂度的复杂问题方面的成功启发，我们将轨迹和调度策略问题重新表述为马尔可夫决策过程 (MDP)，其中系统状态空间考虑了车辆网络动态。然后，我们开发了深度强化学习 (DRL) 来学习车辆环境及其动力学，以便处理无人机的轨迹和调度策略。特别是，我们利用深度确定性策略梯度 (DDPG) 来学习部署的无人机的轨迹，以有效地最小化预期加权总和 AoI (EWSA)。模拟结果证明了所提议设计的有效性，并表明部署的无人机在数据收集任务期间会调整其速度，以最大限度地减少 AoI。