This paper investigates decentralized composite optimization problems involving a common non-smooth regularization term over an undirected and connected network. In the same situation, there exist lots of gradient-based proximal distributed methods, but most of them are only sublinearly convergent. The proof of linear convergence for this series of algorithms is extremely difficult. To set up the problem, we presume all networked agents use the same non-smooth regularization term, which is the circumstance for most machine learning to implement based on centralized optimization. For this scenario, most existing proximal-gradient algorithms trend to ignore the cost of gradient evaluations, which results in degraded performance. To tackle this problem, we further set the local cost function to the average of a moderate amount of local cost subfunctions and develop an edge-based stochastic proximal gradient algorithm (SPG-Edge) by employing local unbiased stochastic averaging gradient method. When the non-smooth term does not exist, the proposed algorithm could be extended to some notable primal-dual domain algorithms, such as EXTRA and DIGing. Finally, we provide a simplified proof of linear convergence and conduct numerical experiments to illustrate the validity of theoretical results.

本文研究了涉及无向和连接网络上常见非平滑正则化项的分散复合优化问题。在同样的情况下，存在很多基于梯度的近端分布式方法，但大多数都只是次线性收敛。这一系列算法的线性收敛证明是极其困难的。为了设置问题，我们假设所有网络代理都使用相同的非平滑正则化项，这是大多数机器学习基于集中优化实现的情况。对于这种情况，大多数现有的近端梯度算法倾向于忽略梯度评估的成本，从而导致性能下降。为了解决这个问题，我们进一步将局部成本函数设置为中等数量的局部成本子函数的平均值，并通过采用局部无偏随机平均梯度方法开发基于边缘的随机近端梯度算法（SPG-Edge）。当不存在非光滑项时，所提出的算法可以扩展到一些著名的原始对偶域算法，如 EXTRA 和 DIGing。最后，我们提供了线性收敛的简化证明，并进行了数值实验来说明理论结果的有效性。