While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms.

中文翻译：

---

非凸非光滑问题的具有动量的分布式随机共识优化

虽然已经提出了许多分布式优化算法来解决网络上的平滑或凸问题，但很少有人能够处理非凸和非平滑问题。基于近端原始对偶方法，本文提出了一种新的（随机）分布式算法，该算法具有 Nesterov 动量，用于加速优化非凸和非光滑问题。从理论上讲，我们表明所提出的算法可以实现$\epsilon$ - 恒定步长下的平稳解 $\mathcal {O}(1/\epsilon ^2)$ 计算复杂度和 $\mathcal {O}(1/\epsilon)$当非平滑项的题词是多面体集时，通信复杂度。与现有的基于梯度跟踪的方法相比，该算法的计算复杂度相同，但通信复杂度较低。据我们所知，所呈现的结果是第一个随机算法$\mathcal {O}(1/\epsilon)$非凸和非光滑问题的通信复杂度。提出了分布式非凸回归问题和基于深度神经网络的分类问题的数值实验，以说明所提出算法的有效性。