当前位置: X-MOL 学术Found. Comput. Math. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Fast Optimistic Gradient Descent Ascent (OGDA) Method in Continuous and Discrete Time
Foundations of Computational Mathematics ( IF 3 ) Pub Date : 2023-11-29 , DOI: 10.1007/s10208-023-09636-5
Radu Ioan Boţ , Ernö Robert Csetnek , Dang-Khoa Nguyen

In the framework of real Hilbert spaces, we study continuous in time dynamics as well as numerical algorithms for the problem of approaching the set of zeros of a single-valued monotone and continuous operator V. The starting point of our investigations is a second-order dynamical system that combines a vanishing damping term with the time derivative of V along the trajectory, which can be seen as an analogous of the Hessian-driven damping in case the operator is originating from a potential. Our method exhibits fast convergence rates of order \(o \left( \frac{1}{t\beta (t)} \right) \) for \(\Vert V(z(t))\Vert \), where \(z(\cdot )\) denotes the generated trajectory and \(\beta (\cdot )\) is a positive nondecreasing function satisfying a growth condition, and also for the restricted gap function, which is a measure of optimality for variational inequalities. We also prove the weak convergence of the trajectory to a zero of V. Temporal discretizations of the dynamical system generate implicit and explicit numerical algorithms, which can be both seen as accelerated versions of the Optimistic Gradient Descent Ascent (OGDA) method for monotone operators, for which we prove that the generated sequence of iterates \((z_k)_{k \ge 0}\) shares the asymptotic features of the continuous dynamics. In particular we show for the implicit numerical algorithm convergence rates of order \(o \left( \frac{1}{k\beta _k} \right) \) for \(\Vert V(z^k)\Vert \) and the restricted gap function, where \((\beta _k)_{k \ge 0}\) is a positive nondecreasing sequence satisfying a growth condition. For the explicit numerical algorithm, we show by additionally assuming that the operator V is Lipschitz continuous convergence rates of order \(o \left( \frac{1}{k} \right) \) for \(\Vert V(z^k)\Vert \) and the restricted gap function. All convergence rate statements are last iterate convergence results; in addition to these, we prove for both algorithms the convergence of the iterates to a zero of V. To our knowledge, our study exhibits the best-known convergence rate results for monotone equations. Numerical experiments indicate the overwhelming superiority of our explicit numerical algorithm over other methods designed to solve monotone equations governed by monotone and Lipschitz continuous operators.



中文翻译:

连续和离散时间的快速乐观梯度下降上升(OGDA)方法

在实希尔伯特空间的框架中,我们研究时间动态连续性以及逼近单值单调连续算子V的零集问题的数值算法。我们研究的起点是一个二阶动力系统,它将消失阻尼项与V沿轨迹的时间导数相结合,可以将其视为类似于 Hessian 驱动阻尼,以防算子源自潜在的。我们的方法对于\(\Vert V(z(t))\Vert \ )表现出\(o \left( \frac{1}{t\beta (t)} \right) \)阶的快速收敛速度,其中\(z(\cdot )\)表示生成的轨迹,\(\beta (\cdot )\)是满足增长条件的正非递减函数,也适用于受限间隙函数,它是变分最优性的度量不平等。我们还证明了轨迹弱收敛到V的零。动力系统的时间离散化生成隐式和显式数值算法,它们都可以看作单调算子的乐观梯度下降上升(OGDA)方法的加速版本,为此我们证明生成的迭代序列 \(( z_k) _{k \ge 0}\)具有连续动力学的渐近特征。特别是,我们显示了\(\Vert V(z^k)\Vert \ )的隐式数值算法收敛速度为\(o \left( \frac{1}{k\beta _k} \right) \)和受限间隙函数,其中\((\beta _k)_{k \ge 0}\)是满足增长条件的正非递减序列。对于显式数值算法,我们通过另外假设算子V为 Lipschitz 连续收敛速度来表示对于\(\Vert V(z^ k)\Vert \)和受限间隙函数。所有收敛速度表述均为最后一次迭代收敛结果;除此之外,我们还证明了这两种算法的迭代收敛到V的零。据我们所知,我们的研究展示了单调方程最著名的收敛率结果。数值实验表明,与其他旨在求解由单调和 Lipschitz 连续算子控制的单调方程的方法相比,我们的显式数值算法具有压倒性的优势。

更新日期:2023-12-02
down
wechat
bug