Fifth generation and beyond networks are envisioned to support multi industrial Internet of Things (IIoT) applications with a diverse quality-of-service (QoS) requirements. Network slicing is recognized as a flagship technology that enables IIoT networks with multiservices and resource requirements by allowing the network-as-infrastructure transition to the network-as-service. Motivated by the increasing IIoT computational capacity, and taking into consideration the QoS satisfaction and private data sharing challenges, federated reinforcement learning (RL) has become a promising approach that distributes data acquisition and computation tasks over distributed network agents, exploiting local computation capacities and agent's self-learning experiences. This article proposes a novel deep RL scheme to provide a federated and dynamic network management and resource allocation for differentiated QoS services in future IIoT networks. This involves IIoT slices resource allocation in terms of transmission power (TP) and spreading factor (SF) according to the slices QoS requirements. Toward this goal, the proposed deep federated Q-learning (DFQL) is reached into two main steps. First, we propose a multiagent deep Q-learning-based dynamic slices TP and SF adjustment process that aims at maximizing self-QoS requirements in term of throughput and delay. Second, the deep federated learning is proposed to learn multiagent self-model and enable them to find an optimal action decision on the TP and the SF that satisfy IIoT virtual network slice QoS reward, exploiting the shared experiences between agents. Simulation results show that the proposed DFQL framework achieves efficient performance compared to the traditional approaches.

中文翻译：

---

适用于工业物联网的基于 Q-Learning 的深度联合网络切片


第五代及以后的网络预计将支持具有不同服务质量 (QoS) 要求的多种工业物联网 (IIoT) 应用。网络切片被认为是一项旗舰技术，通过允许网络即基础设施过渡到网络即服务，使工业物联网网络能够满足多服务和资源需求。受工业物联网计算能力不断增强的推动，并考虑到服务质量满意度和私有数据共享挑战，联合强化学习（RL）已成为一种有前途的方法，它通过分布式网络代理分配数据采集和计算任务，利用本地计算能力和代理的自学经历。本文提出了一种新颖的深度强化学习方案，为未来 IIoT 网络中的差异化 QoS 服务提供联合且动态的网络管理和资源分配。这涉及根据切片 QoS 要求在传输功率 (TP) 和扩频因子 (SF) 方面进行 IIoT 切片资源分配。为了实现这一目标，所提出的深度联合 Q 学习（DFQL）分为两个主要步骤。首先，我们提出了一种基于多智能体深度 Q 学习的动态切片 TP 和 SF 调整过程，旨在最大化吞吐量和延迟方面的自我 QoS 要求。其次，提出了深度联邦学习来学习多智能体自模型，并使它们能够利用智能体之间的共享经验，找到满足 IIoT 虚拟网络切片 QoS 奖励的 TP 和 SF 上的最佳动作决策。仿真结果表明，与传统方法相比，所提出的 DFQL 框架实现了高效的性能。