In this paper, we investigate the problem of age of information (AoI)-aware radio resource management for expected long-term performance optimization in a Manhattan grid vehicle-to-vehicle network. With the observation of global network state at each scheduling slot, the roadside unit (RSU) allocates the frequency bands and schedules packet transmissions for all vehicle user equipment-pairs (VUE-pairs). We model the stochastic decision-making procedure as a discrete-time single-agent Markov decision process (MDP). The technical challenges in solving the optimal control policy originate from high spatial mobility and temporally varying traffic information arrivals of the VUE-pairs. To make the problem solving tractable, we first decompose the original MDP into a series of per-VUE-pair MDPs. Then we propose a proactive algorithm based on long short-term memory and deep reinforcement learning techniques to address the partial observability and the curse of high dimensionality in local network state space faced by each VUE-pair. With the proposed algorithm, the RSU makes the optimal frequency band allocation and packet scheduling decision at each scheduling slot in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical experiments validate the theoretical analysis and demonstrate the significant performance improvements from the proposed algorithm.

中文翻译：

---

车载网络中的信息感知无线电资源管理时代：一个主动的深度强化学习视角

在本文中，我们研究了信息时代 (AoI) 感知无线电资源管理问题，以实现曼哈顿网格车辆到车辆网络中预期的长期性能优化。通过观察每个调度时隙的全局网络状态，路侧单元 (RSU) 为所有车辆用户设备对 (VUE-pairs) 分配频段并调度数据包传输。我们将随机决策过程建模为离散时间单智能体马尔可夫决策过程 (MDP)。解决最优控制策略的技术挑战源于 VUE 对的高空间移动性和随时间变化的交通信息到达。为了使问题解决变得容易，我们首先将原始 MDP 分解为一系列 per-VUE-pair MDP。然后，我们提出了一种基于长短期记忆和深度强化学习技术的主动算法，以解决每个 VUE 对所面临的局部网络状态空间中的部分可观察性和高维诅咒。使用所提出的算法，RSU根据对VUE-pairs的全局网络状态的局部观察，以分散的方式在每个调度时隙做出最优频带分配和分组调度决策。数值实验验证了理论分析并证明了所提出算法的显着性能改进。RSU根据对VUE-pairs的全局网络状态的局部观察，以分散的方式在每个调度时隙做出最优频带分配和分组调度决策。数值实验验证了理论分析并证明了所提出算法的显着性能改进。RSU根据对VUE-pairs的全局网络状态的局部观察，以分散的方式在每个调度时隙做出最优频带分配和分组调度决策。数值实验验证了理论分析并证明了所提出算法的显着性能改进。