The primal-dual method of multipliers (PDMM) was originally designed for solving a decomposable optimisation problem over a general network. In this paper, we revisit PDMM for optimisation over a centralised network. We first note that the recently proposed method FedSplit [1] implements PDMM for a centralised network. In [1], Inexact FedSplit (i.e., gradient based FedSplit) was also studied both empirically and theoretically. We identify the cause for the poor reported performance of Inexact FedSplit, which is due to the suboptimal initialisation in the gradient operations at the client side. To fix the issue of Inexact FedSplit, we propose two versions of Inexact PDMM, which are referred to as gradient-based PDMM (GPDMM) and accelerated GPDMM (AGPDMM), respectively. AGPDMM accelerates GPDMM at the cost of transmitting two times the number of parameters from the server to each client per iteration compared to GPDMM. We provide a new convergence bound for GPDMM for a class of convex optimisation problems. Our new bounds are tighter than those derived for Inexact FedSplit. We also investigate the update expressions of AGPDMM and SCAFFOLD to find their similarities. It is found that when the number K of gradient steps at the client side per iteration is K=1, both AGPDMM and SCAFFOLD reduce to vanilla gradient descent with proper parameter setup. Experimental results indicate that AGPDMM converges faster than SCAFFOLD when K>1 while GPDMM converges slightly slower than SCAFFOLD.

中文翻译：

---

重新审视集中式网络优化的乘法器原对偶方法


原对偶乘子法 (PDMM) 最初是为解决一般网络上的可分解优化问题而设计的。在本文中，我们重新审视 PDMM 以通过集中式网络进行优化。我们首先注意到最近提出的方法 FedSplit [1] 为集中式网络实现了 PDMM。在[1]中，还对Inexact FedSplit（即基于梯度的FedSplit）进行了实证和理论研究。我们确定了 Inexact FedSplit 报告性能不佳的原因，这是由于客户端梯度操作的初始化不理想所致。为了解决 Inexact FedSplit 的问题，我们提出了 Inexact PDMM 的两个版本，分别称为基于梯度的 PDMM (GPDMM) 和加速的 GPDMM (AGPDMM)。 AGPDMM 加速了 GPDMM，但每次迭代从服务器到每个客户端传输的参数数量是 GPDMM 的两倍。我们为一类凸优化问题提供了 GPDMM 的新收敛界。我们的新界限比 Inexact FedSplit 得出的界限更严格。我们还研究了 AGPDMM 和 SCAFFOLD 的更新表达式以发现它们的相似之处。研究发现，当每次迭代客户端的梯度步数 K 为 K=1 时，AGPDMM 和 SCAFFOLD 都可以通过适当的参数设置简化为普通梯度下降。实验结果表明，当K>1时，AGPDMM比SCAFFOLD收敛得更快，而GPDMM比SCAFFOLD收敛得稍慢。