In this letter, we focus on the design of a relay selection scheme in large-scale energy-harvesting wireless sensor networks. Considering the dynamic nature required for practical networks, multiple features, including queuing state, energy level, channel quality, and location, are involved in modeling. By introducing reinforcement learning techniques, a novel relay selection scheme based on an actor-critic algorithm with linear function approximation is proposed to improve network reliability while also taking into account the transmission delay and energy efficiency. The proposed scheme can be implemented independently in each source to maximize the data delivery ratio. Such a distributed scheme is more stable and scalable than centralized structures. The Lagrangian formula is applied to satisfy the constraints on hops and energy efficiency. Compared with traditional timer-based and Q-learning-based schemes, our simulation results show that our proposed scheme achieves good performance in terms of network reliability and obtains higher energy efficiency and lower transmission delay.

中文翻译：

---

一种用于大规模能量收集 WSN 的多特征 Actor-Critic 中继选择方案

在这封信中，我们专注于大规模能量收集无线传感器网络中中继选择方案的设计。考虑到实际网络所需的动态特性，建模涉及多个特征，包括排队状态、能量水平、信道质量和位置。通过引入强化学习技术，提出了一种基于具有线性函数逼近的actor-critic算法的新型中继选择方案，以提高网络可靠性，同时兼顾传输延迟和能量效率。所提出的方案可以在每个源中独立实施，以最大限度地提高数据传输率。这种分布式方案比集中式结构更稳定和可扩展。应用拉格朗日公式来满足对啤酒花和能量效率的约束。