当前位置: X-MOL 学术Math. Program. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
On lower iteration complexity bounds for the convex concave saddle point problems
Mathematical Programming ( IF 2.7 ) Pub Date : 2021-06-07 , DOI: 10.1007/s10107-021-01660-z
Junyu Zhang , Mingyi Hong , Shuzhong Zhang

In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: \(\min _x\max _yF(x,y)\). We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proximal mappings. For problems with gradient Lipschitz constants (\(L_x, L_y\) and \(L_{xy}\)) and strong convexity/concavity constants (\(\mu _x\) and \(\mu _y\)), the class of pure first-order algorithms with the linear span assumption is shown to have a lower iteration complexity bound of \(\Omega \,\left( \sqrt{\frac{L_x}{\mu _x}+\frac{L_{xy}^2}{\mu _x\mu _y}+\frac{L_y}{\mu _y}}\cdot \ln \left( \frac{1}{\epsilon }\right) \right) \), where the term \(\frac{L_{xy}^2}{\mu _x\mu _y}\) explains how the coupling influences the iteration complexity. Under several special parameter regimes, this lower bound has been achieved by corresponding optimal algorithms. However, whether or not the bound under the general parameter regime is optimal remains open. Additionally, for the special case of bilinear coupling problems, given the availability of certain proximal operators, a lower bound of \(\Omega \left( \sqrt{\frac{L_{xy}^2}{\mu _x\mu _y}}\cdot \ln (\frac{1}{\epsilon })\right) \) is established under the linear span assumption, and optimal algorithms have already been developed in the literature. By exploiting the orthogonal invariance technique, we extend both lower bounds to the general pure first-order algorithm class and the proximal algorithm class without the linear span assumption. As an application, we apply proper scaling to the worst-case instances, and we derive the lower bounds for the general convex concave problems with \(\mu _x = \mu _y = 0\). Several existing results in this case can be deduced from our results as special cases.



中文翻译:

关于凸凹鞍点问题的下迭代复杂度边界

在本文中,我们研究了寻找强凸和强凹鞍点问题的鞍点的迭代复杂度下界:\(\min _x\max _yF(x,y)\)。我们将研究中的算法类别限制为纯一阶方法或使用近端映射的方法。对于梯度 Lipschitz 常数(\(L_x, L_y\)\(L_{xy}\))和强凸/凹常数(\(\mu _x\)\(\mu _y\))的问题,类具有线性跨度假设的纯一阶算法的迭代复杂度为\(\Omega \,\left( \sqrt{\frac{L_x}{\mu _x}+\frac{L_{xy}^2}{\mu _x\mu _y}+\frac{L_y}{\mu _y}}\cdot \ln \left( \frac{1}{\epsilon }\right) \right) \),其中术语\(\frac{L_{xy}^2}{\mu _x\mu _y }\)解释了耦合如何影响迭代复杂度。在几个特殊的参数制度下,这个下界已经通过相应的优化算法实现。然而,一般参数制度下的界限是否是最优的仍然是开放的。此外,对于双线性耦合问题的特殊情况,考虑到某些近端算子的可用性,下界\(\Omega \left( \sqrt{\frac{L_{xy}^2}{\mu _x\mu _y }}\cdot \ln (\frac{1}{\epsilon })\right) \)是在线性跨度假设下建立的,并且在文献中已经开发了最优算法。通过利用正交不变性技术,我们将下界扩展到一般纯一阶算法类和没有线性跨度假设的近端算法类。作为一个应用程序,我们对最坏情况的实例应用适当的缩放,并用\(\mu _x = \mu _y = 0\)推导出一般凸凹问题的下界。这种情况下的几个现有结果可以作为特殊情况从我们的结果中推导出来。

更新日期:2021-06-07
down
wechat
bug