SIAM Journal on Optimization, Volume 30, Issue 4, Page 3230-3251, January 2020.
We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extragradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method. This is similar to the approach taken in (A. Nemirovski (2004), SIAM J. Optim., 15, pp. 229--251) which analyzes EG as an approximation of the “conceptual mirror prox.” In this paper, we highlight how gradients used in OGDA and EG try to approximate the gradient of the proximal point method. We then exploit this interpretation to show that both algorithms produce iterates that remain within a bounded set. We further show that the primal-dual gap of the averaged iterates generated by both of these algorithms converge with a rate of $\mathcal{O}(1/k)$. Our theoretical analysis is of interest as it provides the first convergence rate estimate for OGDA in the general convex-concave setting. Moreover, it provides a simple convergence analysis for the EG algorithm in terms of function value without using a compactness assumption.

中文翻译：

---

光滑凹凸凹鞍点问题中最优梯度和超梯度方法的$ \ mathcal {O}（1 / k）$的收敛速度

SIAM优化杂志，第30卷，第4期，第3230-3251页，2020年1月。
我们研究了乐观梯度下降-上升（OGDA）方法和超梯度（EG）方法的迭代复杂性，以找到凸凹无约束最小-最大问题的鞍点。为此，我们首先表明OGDA和EG都可以解释为近端点方法的近似变体。这类似于（A. Nemirovski（2004），SIAM J. Optim。，15，pp。229--251）中采用的方法，该方法将EG分析为“概念镜像代理”的近似值。在本文中，我们重点介绍了OGDA和EG中使用的梯度如何尝试近似近端法的梯度。然后，我们利用这种解释来表明，两种算法都产生了保留在有界集合内的迭代。我们进一步表明，由这两种算法生成的平均迭代的原始对偶间隙以$ \ mathcal {O}（1 / k）$的速率收敛。我们的理论分析很有趣，因为它为一般凸凹设置中的OGDA提供了第一个收敛速率估计。而且，它在函数值方面为EG算法提供了一个简单的收敛分析，而无需使用紧凑性假设。