This paper presents new asynchronous methods to the Federated Learning (FL), one of the next-generation paradigms for Artificial Intelligence (AI) systems. We consider the two-fold challenges lay ahead. First, non-IID (non-Independent and Identically Distributed) data across devices cause unstable performance. Second, unreliable and slow environments not only slow the convergence but also cause staleness issues. To address these challenges, this study uses a bottom-up approach for analysis and algorithm design. We first reformulate FL by unifying both synchronous and asynchronous updating schemes with an asynchrony-related parameter. We theoretically analyze this new form and find practical strategies for optimization. The key findings include: 1) a two-stage training strategy to accelerate training and reduce communication overhead; 2) strategies of choosing key hyperparameters optimally for these stages to maintain efficiency and robustness. With these theoretical guarantees, we propose FedSA (Federated Staleness-Aware), a novel asynchronous federated learning algorithm. We validate FedSA on different tasks with non-IID/IID and staleness settings. Our results indicate that, given a large proportion of stale devices, the proposed algorithm presents state-of-the-art performance by outperforming existing methods on both non-IID and IID cases.

本文为联合学习提出了新的异步方法（FL），这是人工智能（AI）系统的下一代范例之一。我们认为面临两个挑战。首先，跨设备的非IID（非独立且完全相同）数据会导致性能不稳定。其次，不可靠和缓慢的环境不仅减慢了收敛速度，而且还会导致陈旧性问题。为了应对这些挑战，本研究使用了一种自下而上的方法进行分析和算法设计。我们首先通过使用与异步相关的参数统一同步和异步更新方案来重新构造FL。我们从理论上分析了这种新形式，并找到了优化的实用策略。主要研究结果包括：1）采用两阶段的培训策略来加快培训速度并减少沟通开销；2）为这些阶段优化选择关键超参数的策略，以保持效率和鲁棒性。有了这些理论上的保证，我们建议FedSA（美联储erated小号taleness-甲洁具），一种新型异步联合的学习算法。我们使用非IID / IID和陈旧性设置来验证FedSA在不同任务上的作用。我们的结果表明，在相当大的陈旧设备中，该算法通过在非IID和IID情况下均优于现有方法，从而表现出了最先进的性能。