Information freshness is essential to industrial wireless networks (IWNs) and can be quantified by the age-of-information (AoI) metric. This paper addresses an AoI-aware spectrum sharing (AgeS) problem in IWNs, where multiple device-to-device (D2D) links opportunistically access the spectrum to satisfy their AoI constraints while maximizing primal links’ throughput. Particularly, we orchestrate the access of D2D links in a distributed manner. Since distributed scheduling results in incomplete observation, D2D links share the spectrum with uncertainty on the transmission environment. Therefore, we propose a distributed scheduling scheme, called D-age, to deal with the transmission uncertainty in the AgeS problem, where an adaptation of actor-critic method is adopted with AoI constraints tackled in the dual domain. To address the non-stationary environment and multi-agent credit assignment issue, cooperative multi-agent reinforcement learning (MARL) approach is developed, where multiple local actors are designed to guide D2D links to make real-time decisions via distributed scheduling policies, which are evaluated by an edge-assisted global critic with action-aware advantage functions. Integrated with graph attention networks (GATs), the critic selectively learns contextual information by assigning different importances to neighboring links, which enables the evaluation of scheduling policies in a scalable and computation-efficient manner. Theoretical guarantee of the time-averaged AoI constraints is provided and the effectiveness of D-age in terms of both AoI violation ratio and the capacity of primal links is demonstrated by simulation.

中文翻译：

---

用于新鲜度感知工业无线网络的边缘辅助频谱共享：基于学习的方法


信息新鲜度对于工业无线网络 (IWN) 至关重要，可以通过信息时代 (AoI) 指标进行量化。本文解决了 IWN 中的 AoI 感知频谱共享 (AgeS) 问题，其中多个设备到设备 (D2D) 链路机会性地访问频谱以满足其 AoI 约束，同时最大化原始链路的吞吐量。特别是，我们以分布式方式编排D2D链路的访问。由于分布式调度导致观测不完整，D2D链路共享频谱，传输环境具有不确定性。因此，我们提出了一种分布式调度方案，称为 D-age，来处理 AgeS 问题中的传输不确定性，其中采用了 Actor-Critic 方法的改进，并在双域中解决了 AoI 约束。为了解决非平稳环境和多智能体信用分配问题，开发了协作多智能体强化学习（MARL）方法，其中多个本地参与者被设计来引导D2D链路通过分布式调度策略做出实时决策，这由具有行动感知优势功能的边缘辅助全局评论家进行评估。与图注意网络（GAT）集成，批评者通过为相邻链接分配不同的重要性来选择性地学习上下文信息，从而能够以可扩展和计算高效的方式评估调度策略。提供了时间平均 AoI 约束的理论保证，并通过仿真证明了 D-age 在 AoI 违规率和原始链路容量方面的有效性。