Applications of Unmanned Aerial Vehicles (UAVs) for data collection are a promising means to extend Internet-ofThings (IoT) networks into remote and hostile areas, and areas with no access to power supplies. Adequate design of velocity control and communication decisions of UAVs is critical to minimize the data packet losses of ground IoT nodes resulting from overflowing buffers and transmission failure. However, online velocity control and communication decision-making is challenging in UAV-enabled IoT networks, due to the lack of the up-to-date knowledge on the state of the IoT nodes, e.g., battery energy, buffer length and channel conditions, at the UAV. Current methodology using reinforcement learning complements real-time solutions to small-scale decision problems in static IoT networks. However, reinforcement learning is impractical for the online velocity control and communication decision in the UAV-enabled IoT network, due to its rapidly growing complexity (also known as the curse-of-dimensionality). This article discusses the design of onboard deep Q-network to deliver the online velocity control and communication decision of UAVs. The onboard deep Q-network can jointly determine the optimal patrol velocity of the UAV and decide the IoT node to be interrogated for data collection, thereby minimizing asymptotically the data packet loss of the IoT networks.

中文翻译：

---

物联网无人机的在线速度控制和数据捕获：一种机载深度强化学习方法

无人机 (UAV) 用于数据收集的应用是将物联网 (IoT) 网络扩展到偏远和敌对地区以及无法获得电源的地区的一种很有前景的方法。无人机的速度控制和通信决策的适当设计对于最大限度地减少由于缓冲区溢出和传输失败导致的地面物联网节点的数据包丢失至关重要。然而，在线速度控制和通信决策在支持无人机的物联网网络中具有挑战性，因为缺乏物联网节点状态的最新知识，例如电池能量、缓冲区长度和信道条件，在无人机上。当前使用强化学习的方法补充了静态物联网网络中小规模决策问题的实时解决方案。然而，由于其快速增长的复杂性（也称为维度诅咒），强化学习对于支持无人机的物联网网络中的在线速度控制和通信决策是不切实际的。本文讨论了机载深度 Q 网络的设计，以实现无人机的在线速度控制和通信决策。机载深度Q网络可以共同确定无人机的最佳巡逻速度，并决定要询问的物联网节点进行数据采集，从而渐近地最小化物联网网络的数据包丢失。本文讨论了机载深度 Q 网络的设计，以实现无人机的在线速度控制和通信决策。机载深度Q网络可以共同确定无人机的最佳巡逻速度，并决定要询问的物联网节点进行数据采集，从而渐近地最小化物联网网络的数据包丢失。本文讨论了机载深度 Q 网络的设计，以实现无人机的在线速度控制和通信决策。机载深度Q网络可以共同确定无人机的最佳巡逻速度，并决定要询问的物联网节点进行数据采集，从而渐近地最小化物联网网络的数据包丢失。