In this paper, we aim to make the best joint decision of device selection and computing and spectrum resource allocation for optimizing federated learning (FL) performance in distributed industrial Internet of Things (IIoT) networks. To implement efficient FL over geographically dispersed data, we introduce a three-layer collaborative FL architecture to support deep neural network (DNN) training. Specifically, using the data dispersed in IIoT devices, the industrial gateways locally train the DNN model and the local models can be aggregated by their associated edge servers every FL epoch or by a cloud server every a few FL epochs for obtaining the global model. To optimally select participating devices and allocate computing and spectrum resources for training and transmitting the model parameters, we formulate a stochastic optimization problem with the objective of minimizing FL evaluating loss while satisfying delay and long-term energy consumption requirements. Since the objective function of the FL evaluating loss is implicit and the energy consumption is temporally correlated, it is difficult to solve the problem via traditional optimization methods. Thus, we propose a “ Reinforcement on Federated ” (RoF) scheme, based on deep multi-agent reinforcement learning, to solve the problem. Specifically, the RoF scheme is executed decentralizedly at edge servers, which can cooperatively make the optimal device selection and resource allocation decisions. Moreover, a device refinement subroutine is embedded into the RoF scheme to accelerate convergence while effectively saving the on-device energy. Simulation results demonstrate that the RoF scheme can facilitate efficient FL and achieve better performance compared with state-of-the-art benchmarks.

中文翻译：

---

优化分布式工业物联网中的联邦学习：一种多代理方法

在本文中，我们旨在做出设备选择和计算以及频谱资源分配的最佳联合决策，以优化分布式工业物联网 (IIoT) 网络中的联邦学习 (FL) 性能。为了在地理上分散的数据上实现高效的 FL，我们引入了一个三层协作 FL 架构来支持深度神经网络 (DNN) 训练。具体来说，使用分散在 IIoT 设备中的数据，工业网关在本地训练 DNN 模型，本地模型可以在每个 FL epoch 时由其关联的边缘服务器聚合，或者每隔几个 FL epoch 由云服务器聚合以获得全局模型。为了优化选择参与设备并分配用于训练和传输模型参数的计算和频谱资源，我们制定了一个随机优化问题，其目标是在满足延迟和长期能耗要求的同时最小化 FL 评估损失。由于FL评估损失的目标函数是隐式的，能量消耗是时间相关的，传统的优化方法很难解决这个问题。因此，我们提出一个“ Reinforcement on Federated” (RoF) 方案，基于深度多智能体强化学习，解决问题。具体而言，RoF 方案在边缘服务器上分散执行，可以协同做出最佳设备选择和资源分配决策。此外，在 RoF 方案中嵌入了一个设备优化子程序，以加速收敛，同时有效地节省设备上的能量。仿真结果表明，与最先进的基准测试相比，RoF 方案可以促进高效的 FL 并实现更好的性能。