SIAM Journal on Optimization, Volume 30, Issue 4, Page 2866-2896, January 2020.
In this paper, we propose two novel non-stationary first-order primal-dual algorithms to solve non-smooth composite convex optimization problems. Unlike existing primal-dual schemes where the parameters are often fixed, our methods use predefined and dynamic sequences for parameters. We prove that our first algorithm can achieve an $\mathcal{O}\left(1/k\right)$ convergence rate on the primal-dual gap, and primal and dual objective residuals, where $k$ is the iteration counter. Our rate is on the non-ergodic (i.e., the last iterate) sequence of the primal problem and on the ergodic (i.e., the averaging) sequence of the dual problem, which we call the semi-ergodic rate. By modifying the step-size update rule, this rate can be boosted even faster on the primal objective residual. When the problem is strongly convex, we develop a second primal-dual algorithm that exhibits an $\mathcal{O}\left(1/k^2\right)$ convergence rate on the same three types of guarantees. Again by modifying the step-size update rule, this rate becomes faster on the primal objective residual. Our primal-dual algorithms are the first ones to achieve such fast convergence rate guarantees under mild assumptions compared to existing works, to the best of our knowledge. As byproducts, we apply our algorithms to solve constrained convex optimization problems and prove the same convergence rates on both the objective residuals and the feasibility violation. We still obtain at least $\mathcal{O}\left(1/k^2\right)$ rates even when the problem is “semi-strongly” convex. We verify our theoretical results via two well-known numerical examples.

中文翻译：

---

收敛速度更快的非平稳一阶本原对偶算法

SIAM优化杂志，第30卷，第4期，第2866-2896页，2020年1月。
在本文中，我们提出了两种新颖的非平稳一阶原对偶算法来解决非光滑复合凸优化问题。与现有的原始对偶方案（参数通常是固定的）不同，我们的方法对参数使用预定义和动态序列。我们证明我们的第一个算法可以在原始对偶间隙以及原始和对偶目标残差上实现$ \ mathcal {O} \ left（1 / k \ right）$收敛速度，其中$ k $是迭代计数器。我们的速率取决于原始问题的非遍历（即最后一个迭代）序列和对偶问题的遍历（即平均）序列，我们称之为半遍历率。通过修改步长更新规则，可以在原始目标残差上更快地提高此速率。当问题很凸时，我们开发了第二种原始对偶算法，该算法在相同的三种类型的担保下表现出$ \ mathcal {O} \ left（1 / k ^ 2 \ right）$收敛速度。同样，通过修改步长更新规则，该速率在原始目标残差上变得更快。据我们所知，我们的原始对偶算法是第一个在温和的假设下与现有作品相比能够实现如此快速收敛速率保证的算法。作为副产品，我们将我们的算法用于解决约束凸优化问题，并证明在目标残差和可行性违规方面的收敛速度相同。即使问题是“半强”凸的，我们仍然至少获得$ \ mathcal {O} \ left（1 / k ^ 2 \ right）$比率。我们通过两个著名的数值示例验证了我们的理论结果。