There has been an increasing necessity for scalable optimization methods, especially due to the explosion in the size of data sets and model complexity in modern machine learning applications. Scalable solvers often distribute the computation over a network of processing units. For simple algorithms, such as gradient descent, the dependence of the convergence time with the topology of this network is well known. However, for more involved algorithms, such as the alternating direction method of multipliers (ADMM), much less is known. At the heart of many distributed optimization algorithms, there exists a gossip subroutine which averages local information over the network, whose efficiency is crucial for the overall performance of the method. In this article, we review recent research in this area, and with the goal of isolating such a communication exchange behavior, we compare different algorithms when applied to a canonical distributed averaging consensus problem. We also show interesting connections between ADMM and the lifted Markov chains besides providing an explicit characterization of its convergence and optimal parameter tuning in terms of spectral properties of the network. Finally, we empirically study the connection between network topology and convergence rates for different algorithms on a real-world problem of sensor localization.

中文翻译：

---

分布式优化、通过 ADMM 求平均和网络拓扑

对可扩展优化方法的需求越来越大，尤其是由于现代机器学习应用程序中数据集规模和模型复杂性的爆炸式增长。可扩展求解器通常将计算分布在处理单元网络上。对于简单的算法，例如梯度下降，收敛时间与该网络拓扑的相关性是众所周知的。然而，对于更复杂的算法，例如乘法器的交替方向法（ADMM），知之甚少。在许多分布式优化算法的核心，存在一个 gossip 子程序，它在网络上平均本地信息，其效率对于该方法的整体性能至关重要。在本文中，我们回顾了该领域的最新研究，为了隔离这种通信交换行为，我们比较了应用于规范分布式平均共识问题时的不同算法。我们还展示了 ADMM 和提升的马尔可夫链之间有趣的联系，除了根据网络的频谱特性提供其收敛性和最佳参数调整的明确表征之外。最后，我们针对传感器定位的实际问题，凭经验研究了不同算法的网络拓扑与收敛率之间的联系。我们还展示了 ADMM 和提升的马尔可夫链之间有趣的联系，除了根据网络的频谱特性提供其收敛性和最佳参数调整的明确表征之外。最后，我们针对传感器定位的实际问题，凭经验研究了不同算法的网络拓扑与收敛率之间的联系。我们还展示了 ADMM 和提升的马尔可夫链之间有趣的联系，除了根据网络的频谱特性提供其收敛性和最佳参数调整的明确表征之外。最后，我们针对传感器定位的实际问题，凭经验研究了不同算法的网络拓扑与收敛率之间的联系。