Federated learning (FL) involves training machine learning models over distributed edge nodes (i.e., workers) while facing three critical challenges, edge heterogeneity, Non-IID data and communication resource constraint. In the synchronous FL, the parameter server has to wait for the slowest workers, leading to significant waiting time due to edge heterogeneity. Though asynchronous FL can well tackle the edge heterogeneity, it requires frequent model transfers, resulting in massive communication resource consumption. Moreover, the different relative frequency of workers participating in asynchronous updating may seriously hurt training accuracy, especially on Non-IID data. In this paper, we propose a semi-asynchronous federated learning mechanism (FedSA), where the parameter server aggregates a certain number of local models by their arrival order in each round. We theoretically analyze the quantitative relationship between the convergence bound of FedSA and different factors, e.g., the number of participating workers in each round, the degree of data Non-IID and edge heterogeneity. Based on the convergence bound, we present an efficient algorithm to determine the number of participating workers to minimize the training completion time. To further improve the training accuracy on Non-IID data, FedSA deploys adaptive learning rates for workers by their relative participation frequency. We extend our proposed mechanism to the dynamic and multiple learning tasks scenarios. Experimental results on the testbed show that our proposed mechanism and algorithms address the three challenges more effectively than the state-of-the-art solutions.

中文翻译：

---

FedSA：异构边缘计算中的半异步联邦学习机制


联邦学习（FL）涉及在分布式边缘节点（即工作人员）上训练机器学习模型，同时面临三个关键挑战：边缘异构性、非独立同分布数据和通信资源约束。在同步FL中，参数服务器必须等待最慢的worker，由于边缘异构性导致等待时间很长。虽然异步FL可以很好地解决边缘异构性，但它需要频繁的模型传输，导致大量的通信资源消耗。此外，参与异步更新的工作人员的相对频率不同可能会严重损害训练准确性，尤其是在非独立同分布数据上。在本文中，我们提出了一种半异步联邦学习机制（FedSA），其中参数服务器在每轮中按到达顺序聚合一定数量的本地模型。我们从理论上分析了FedSA的收敛界限与不同因素之间的定量关系，例如每轮参与工人的数量、数据非独立同分布的程度以及边缘异质性。基于收敛界限，我们提出了一种有效的算法来确定参与工人的数量，以最小化训练完成时间。为了进一步提高非独立同分布数据的训练准确性，FedSA 根据员工的相对参与频率为员工部署自适应学习率。我们将我们提出的机制扩展到动态和多个学习任务场景。测试台上的实验结果表明，我们提出的机制和算法比最先进的解决方案更有效地解决了这三个挑战。