This paper studies the multi-agent resource allocation problem in vehicular networks using non-orthogonal multiple access (NOMA) and network slicing. Vehicles want to broadcast multiple packets with heterogeneous quality-of-service (QoS) requirements, such as safety-related packets (e.g., accident reports) that require very low latency communication, while raw sensor data sharing (e.g., high-definition map sharing) requires high-speed communication. To ensure heterogeneous service requirements for different packets, we propose a network slicing architecture. We focus on a non-cellular network scenario where vehicles communicate by the broadcast approach via the direct device-to-device interface (i.e., sidelink communication). In such a vehicular network, resource allocation among vehicles is very difficult, mainly due to (i) the rapid variation of wireless channels among highly mobile vehicles and (ii) the lack of a central coordination point. Thus, the possibility of acquiring instantaneous channel state information to perform centralized resource allocation is precluded. The resource allocation problem considered is therefore very complex. It includes not only the usual spectrum and power allocation, but also coverage selection (which target vehicles to broadcast to) and packet selection (which network slice to use). This problem must be solved jointly since selected packets can be overlaid using NOMA and therefore spectrum and power must be carefully allocated for better vehicle coverage. To do so, we first provide a mathematical programming formulation and a thorough NP-hardness analysis of the problem. Then, we model it as a multi-agent Markov decision process. Finally, to solve it efficiently, we use a deep reinforcement learning (DRL) approach and specifically propose a deep Q learning (DQL) algorithm. The proposed DQL algorithm is practical because it can be implemented in an online and distributed manner. It is based on a cooperative learning strategy in which all agents perceive a common reward and thus learn cooperatively and distributively to improve the resource allocation solution through offline training. We show that our approach is robust and efficient when faced with different variations of the network parameters and compared to centralized benchmarks.

本文研究了使用非正交多址 (NOMA) 和网络切片的车载网络中的多代理资源分配问题。车辆希望广播多个具有异构服务质量 (QoS) 要求的数据包，例如需要极低延迟通信的安全相关数据包（例如，事故报告），同时原始传感器数据共享（例如，高清地图共享） ) 需要高速通信。为了确保不同数据包的异构服务需求，我们提出了一种网络切片架构。我们专注于非蜂窝网络场景，其中车辆通过直接设备到设备接口（即侧链通信）通过广播方法进行通信。在这样的车联网中，车辆之间的资源分配非常困难，主要是由于 (i) 高度移动车辆之间无线信道的快速变化以及 (ii) 缺乏中央协调点。因此，排除了获取瞬时信道状态信息以进行集中资源分配的可能性。因此所考虑的资源分配问题非常复杂。它不仅包括通常的频谱和功率分配，还包括覆盖选择（目标车辆广播）和数据包选择（使用哪个网络切片）。这个问题必须联合解决，因为可以使用 NOMA 覆盖选定的数据包，因此必须仔细分配频谱和功率以实现更好的车辆覆盖。为此，我们首先提供数学规划公式和对问题的彻底 NP 硬度分析。然后，我们将其建模为多智能体马尔可夫决策过程。最后，为了有效地解决它，我们使用了深度强化学习 (DRL) 方法，并专门提出了深度 Q 学习 (DQL) 算法。所提出的 DQL 算法是实用的，因为它可以以在线和分布式方式实现。它基于一种合作学习策略，其中所有智能体都感知到一个共同的奖励，从而通过离线训练进行协作和分布式学习，以改进资源分配解决方案。我们表明，当面对网络参数的不同变化并与集中式基准比较时，我们的方法是稳健和高效的。所提出的 DQL 算法是实用的，因为它可以以在线和分布式方式实现。它基于一种合作学习策略，其中所有智能体都感知到一个共同的奖励，从而通过离线训练进行协作和分布式学习，以改进资源分配解决方案。我们表明，当面对网络参数的不同变化并与集中式基准比较时，我们的方法是稳健和高效的。所提出的 DQL 算法是实用的，因为它可以以在线和分布式方式实现。它基于一种合作学习策略，其中所有智能体都感知到一个共同的奖励，从而通过离线训练进行协作和分布式学习，以改进资源分配解决方案。我们表明，当面对网络参数的不同变化并与集中式基准比较时，我们的方法是稳健和高效的。