当前位置:
X-MOL 学术
›
IEEE Trans. Signal Process.
›
论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
FedPD: A Federated Learning Framework With Adaptivity to Non-IID Data
IEEE Transactions on Signal Processing ( IF 5.4 ) Pub Date : 2021-10-01 , DOI: 10.1109/tsp.2021.3115952 Xinwei Zhang , Mingyi Hong , Sairaj Dhople , Wotao Yin , Yang Liu
IEEE Transactions on Signal Processing ( IF 5.4 ) Pub Date : 2021-10-01 , DOI: 10.1109/tsp.2021.3115952 Xinwei Zhang , Mingyi Hong , Sairaj Dhople , Wotao Yin , Yang Liu
Federated Learning (FL) is popular for communication-efficient learning from distributed data. To utilize data at different clients without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a computation then aggregation model, in which multiple local updates are performed using local data before aggregation. These algorithms fail to work when faced with practical challenges, e.g., the local data being non-identically independently distributed. In this paper, we first characterize the behavior of the FedAvg algorithm, and show that without strong and unrealistic assumptions on the problem structure, it can behave erratically. Aiming at designing FL algorithms that are provably fast and require as few assumptions as possible, we propose a new algorithm design strategy from the primal-dual optimization perspective. Our strategy yields algorithms that can deal with non-convex objective functions, achieves the best possible optimization and communication complexity (in a well-defined sense), and accommodates full-batch and mini-batch local computation models. Importantly, the proposed algorithms are communication efficient
, in that the communication effort can be reduced when the level of heterogeneity among the local data also reduces. In the extreme case where the local data becomes homogeneous, only $\mathcal {O}(1)$ communication is required among the agents. To the best of our knowledge, this is the first algorithmic framework for FL that achieves all the above properties.
中文翻译:
FedPD:一种对非 IID 数据具有适应性的联邦学习框架
联邦学习 (FL) 因从分布式数据中进行高效通信学习而广受欢迎。为了在不将数据移动到云端的情况下利用不同客户端的数据,诸如联合平均 (FedAvg) 之类的算法采用了计算然后聚合的模型,其中在聚合之前使用本地数据执行多个本地更新。这些算法在面临实际挑战时无法工作,例如,本地数据不完全相同地独立分布。在本文中,我们首先描述了 FedAvg 算法的行为,并表明如果没有对问题结构的强烈和不切实际的假设,它会表现得不规律。旨在设计可证明快速且需要尽可能少假设的 FL 算法,我们从原始对偶优化的角度提出了一种新的算法设计策略。我们的策略产生的算法可以处理非凸目标函数,实现最佳的优化和通信复杂度(在明确定义的意义上),并适应全批次和小批量本地计算模型。重要的是,所提出的算法是通信效率高,因为当本地数据之间的异质性水平也降低时,可以减少通信工作量。在本地数据变得同质的极端情况下,只有$\mathcal {O}(1)$ 代理之间需要通信。据我们所知,这是第一个实现上述所有属性的 FL 算法框架。
更新日期:2021-11-16
中文翻译:
FedPD:一种对非 IID 数据具有适应性的联邦学习框架
联邦学习 (FL) 因从分布式数据中进行高效通信学习而广受欢迎。为了在不将数据移动到云端的情况下利用不同客户端的数据,诸如联合平均 (FedAvg) 之类的算法采用了计算然后聚合的模型,其中在聚合之前使用本地数据执行多个本地更新。这些算法在面临实际挑战时无法工作,例如,本地数据不完全相同地独立分布。在本文中,我们首先描述了 FedAvg 算法的行为,并表明如果没有对问题结构的强烈和不切实际的假设,它会表现得不规律。旨在设计可证明快速且需要尽可能少假设的 FL 算法,我们从原始对偶优化的角度提出了一种新的算法设计策略。我们的策略产生的算法可以处理非凸目标函数,实现最佳的优化和通信复杂度(在明确定义的意义上),并适应全批次和小批量本地计算模型。重要的是,所提出的算法是