Industrial wireless networks (IWNs) have attracted significant attention for providing time-critical delivery services, which can benefit from device-to-device (D2D) communication for low transmission delay. In this article, a distributed scheduling problem is investigated for D2D-enabled IWNs, where D2D links have various age-of-information (AoI) constraints for information freshness. This problem is formulated as a constrained optimization problem to optimize D2D packet delivery over limited spectrum resources, which is intractable since D2D users have no prior knowledge of the operating environment. To tackle this problem, in this article, an AoI-aware scheduling scheme is proposed based on primal-dual optimization and actor--critic reinforcement learning. In specific, multiple local actors for D2D devices learn AoI-aware scheduling policies to make on-site decisions with their stochastic AoI constraints addressed in the dual domain. An edge-based critic estimates the performance of all actors’ decision-making policies from a global view, which can effectively address the nonstationary environment caused by concurrent learning of multiple local actors. Theoretical analysis on the convergence of learning is provided and simulation results demonstrate the effectiveness of the proposed scheme.

中文翻译：

---

边缘辅助工业无线网络的信息时代感知调度

工业无线网络（IWN）在提供时间紧迫的交付服务方面引起了极大的关注，该服务可以受益于设备到设备（D2D）通信以实现低传输延迟。在本文中，研究了启用D2D的IWN的分布式调度问题，其中D2D链接具有各种信息年龄（AoI）约束，以保证信息新鲜度。该问题被公式化为约束优化问题，以优化有限频谱资源上的D2D数据包传输，这是棘手的，因为D2D用户不了解操作环境。为了解决这个问题，本文提出了一种基于原始对偶优化和行为者与批判强化学习的AoI感知调度方案。具体来说，D2D设备的多个本地参与者学习AoI感知的调度策略，以其在双域中解决的随机AoI约束做出现场决策。边缘评论家从全局角度评估所有参与者的决策政策的绩效，可以有效解决因同时学习多个本地参与者而导致的非平稳环境。对学习的收敛性进行了理论分析，仿真结果证明了该方案的有效性。