There has been significant interest in distributed optimization algorithms, motivated by applications in Big Data analytics, smart grid, vehicle networks, etc. While there have been extensive theory and theoretical advances, a proportionally small body of scientific literature focuses on numerical evaluation of the proposed methods in actual practical, parallel programming environments. This paper considers a general algorithmic framework of first and second order methods with sparsified communications and computations across worker nodes. The considered framework subsumes several existing methods. In addition, a novel method that utilizes unidirectional sparsified communications is introduced and theoretical convergence analysis is also provided. Namely, we prove R-linear convergence in the expected norm. A thorough empirical evaluation of the methods using Message Passing Interface (MPI) on a High Performance Computing (HPC) cluster is carried out and several useful insights and guidelines on the performance of algorithms and inherent communication-computational trade-offs in a realistic setting are derived.

在大数据分析、智能电网、车辆网络等应用的推动下，人们对分布式优化算法产生了浓厚的兴趣。实际的并行编程环境中的方法。本文考虑了具有跨工作节点的稀疏通信和计算的一阶和二阶方法的通用算法框架。所考虑的框架包含了几种现有的方法。此外，还介绍了一种利用单向稀疏通信的新方法，并提供了理论收敛性分析。也就是说，我们证明了在预期范数中的 R 线性收敛。