SIAM Journal on Optimization, Volume 31, Issue 2, Page 1299-1329, January 2021.
In this paper, we propose a primal-dual algorithm with a novel momentum term using the partial gradients of the coupling function that can be viewed as a generalization of the method proposed by Chambolle and Pock in [Math. Program., 159 (2016), pp. 253--287] for solving saddle point problems defined by a convex-concave function $\mathcal{L}(x,y)=f(x)+\Phi(x,y)-h(y)$ with a general coupling term $\Phi(x,y)$ that is not assumed to be bilinear. Assuming $\nabla_x\Phi(\cdot,y)$ is Lipschitz for any fixed $y$, and $\nabla_y\Phi(\cdot,\cdot)$ is Lipschitz, we show that the iterate sequence converges to a saddle point, and for any $(x,y)$, we derive error bounds in terms of $\mathcal{L}(\bar{x}_k,y)-\mathcal{L}(x,\bar{y}_k)$ for the ergodic sequence $\{\bar{x}_k,\bar{y}_k\}$. In particular, we show $\mathcal{O}(1/k)$ rate when the problem is merely convex in $x$. Furthermore, assuming $\Phi(x,\cdot)$ is linear for each fixed $x$ and $f$ is strongly convex, we obtain the ergodic convergence rate of $\mathcal{O}(1/k^2)$---we are not aware of another single-loop method in the related literature achieving the same rate when $\Phi$ is not bilinear. Finally, we propose a backtracking technique which does not require knowledge of Lipschitz constants yet ensures the same convergence results. We also consider convex optimization problems with nonlinear functional constraints, and we show that by using the backtracking scheme, the optimal convergence rate can be achieved even when the dual domain is unbounded. We tested our method against other state-of-the-art first-order algorithms for solving quadratically constrained quadratic programming (QCQP): in the first set of experiments, we considered QCQPs with synthetic data, and in the second set, we focused on QCQPs with real data originating from a variant of the linear regression problem with fairness constraints arising in machine learning.

中文翻译：

---

线性搜索的凹凸对偶鞍点问题的原始-对偶算法

SIAM优化杂志，第31卷，第2期，第1299-1329页，2021年1月。
在本文中，我们提出了一种使用耦合函数的局部梯度的具有新颖动量项的原始对偶算法，该算法可以看作是Chambolle和Pock在[数学。计划，159（2016），第253--287页]，用于解决由凸凹函数$ \ mathcal {L}（x，y）= f（x）+ \ Phi（x，y）定义的鞍点问题）-h（y）$，其一般耦合项$ \ Phi（x，y）$不被认为是双线性的。假设$ \ nabla_x \ Phi（\ cdot，y）$是任何固定$ y $的Lipschitz，而$ \ nabla_y \ Phi（\ cdot，\ cdot）$是Lipschitz，则表明迭代序列收敛到一个鞍点，对于任何$（x，y）$，我们根据$ \ mathcal {L}（\ bar {x} _k，y）-\ mathcal {L}（x，\ bar {y} _k ）$代表遍历序列$ \ {\ bar {x} _k，\ bar {y} _k \} $。特别是，当问题仅在$ x $中凸出时，我们显示$ \ mathcal {O}（1 / k）$比率。此外，假设对于每个固定的$ x $，$ \ Phi（x，\ cdot）$是线性的，而$ f $是强凸的，则我们得出$ \ mathcal {O}（1 / k ^ 2）$的遍历收敛率。 ---当$ \ Phi $不是双线性时，我们还不知道相关文献中的另一种单循环方法可以达到相同的速率。最后，我们提出了一种回溯技术，该技术不需要Lipschitz常数的知识，但可以确保相同的收敛结果。我们还考虑了具有非线性功能约束的凸优化问题，并且我们表明，通过使用回溯方案，即使在双域无界时也可以实现最佳收敛速度。