当前位置: X-MOL 学术SIAM J. Optim. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Revisiting EXTRA for Smooth Distributed Optimization
SIAM Journal on Optimization ( IF 3.1 ) Pub Date : 2020-07-01 , DOI: 10.1137/18m122902x
Huan Li , Zhouchen Lin

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1795-1821, January 2020.
EXTRA is a popular method for dencentralized distributed optimization and has broad applications. This paper revisits EXTRA. First, we give a sharp complexity analysis for EXTRA with the improved $O\big(\big(\frac{L}{\mu}+\frac{1}{1-\sigma_2({W})}\big)\log\frac{1}{\epsilon(1-\sigma_2({W}))}\big)$ communication and computation complexities for $\mu$-strongly convex and $L$-smooth problems, where $\sigma_2({W})$ is the second largest singular value of the weight matrix ${W}$. When the strong convexity is absent, we prove the $O\big(\big(\frac{L}{\epsilon}+\frac{1}{1-\sigma_2({W})}\big)\log\frac{1}{1-\sigma_2({W})}\big)$ complexities. Then, we use the Catalyst framework to accelerate EXTRA and obtain the $O\big(\sqrt{\frac{L}{\mu(1-\sigma_2({W}))}}\log\frac{ L}{\mu(1-\sigma_2({W}))}\log\frac{1}{\epsilon}\big)$ communication and computation complexities for strongly convex and smooth problems and the $O\big(\sqrt{\frac{L}{\epsilon(1-\sigma_2({W}))}}\log\frac{1}{\epsilon(1-\sigma_2(\mathbf{W}))}\big)$ complexities for nonstrongly convex ones. Our communication complexities of the accelerated EXTRA are only worse by the factors of $\big(\log\frac{L}{\mu(1-\sigma_2(\mathbf{W}))}\big)$ and $\big(\log\frac{1}{\epsilon(1-\sigma_2({W}))}\big)$ from the lower complexity bounds for strongly convex and nonstrongly convex problems, respectively.


中文翻译:

回顾EXTRA以实现平滑的分布式优化

SIAM优化杂志,第30卷,第3期,第1795-1821页,2020年1月。
EXTRA是一种用于分散式分布式优化的流行方法,具有广泛的应用。本文回顾了EXTRA。首先,我们使用改进的$ O \ big(\ big(\ frac {L} {\ mu} + \ frac {1} {1- \ sigma_2({W})} \ big)进行EXTRA的复杂性分析。 \ log \ frac {1} {\ epsilon(1- \ sigma_2({W}))} \ big)$通信和计算复杂度,用于$ \ mu $-强凸和$ L $-平滑问题,其中$ \ sigma_2 ({W})$是权重矩阵$ {W} $的第二大奇异值。当不存在强凸度时,我们证明$ O \ big(\ big(\ frac {L} {\ epsilon} + \ frac {1} {1- \ sigma_2({W})} \ big)\ log \ frac {1} {1- \ sigma_2({W})} \ big)$复杂度。然后,我们使用Catalyst框架加速EXTRA并获得$ O \ big(\ sqrt {\ frac {L} {\ mu(1- \ sigma_2({W}))}} \\ log \ frac {L} {\ mu (1- \ sigma_2({W}))} \ log \ frac {1} {\ epsilon} \ big)$通信和计算复杂度,用于强凸和平滑问题以及$ O \ big(\ sqrt {\ frac { L} {\ epsilon(1- \ sigma_2({W}))}} \ log \ frac {1} {\ epsilon(1- \ sigma_2(\ mathbf {W}))} \ big)$非强凸的复杂度那些。仅由于$ \ big(\ log \ frac {L} {\ mu(1- \ sigma_2(\ mathbf {W}))} \ big)$和$ \ big的因素,我们加速EXTRA的通讯复杂性才会变差(\ log \ frac {1} {\ epsilon(1- \ sigma_2({W}))} \ big)$分别从低复杂度边界中分别针对强凸和非强凸问题。
更新日期:2020-07-23
down
wechat
bug