Mathematical Programming ( IF 2.7 ) Pub Date : 2021-05-24 , DOI: 10.1007/s10107-021-01665-8 Oran Gannot
We study robustness properties of some iterative gradient-based methods for strongly convex functions, as well as for the larger class of functions with sector-bounded gradients, under a relative error model. Proofs of the corresponding convergence rates are based on frequency-domain criteria for the stability of nonlinear systems. Applications are given to inexact versions of gradient descent and the Triple Momentum Method. To further emphasize the usefulness of frequency-domain methods, we derive improved analytic bounds for the convergence rate of Nesterov’s accelerated method (in the exact setting) on strongly convex functions.
中文翻译:
不精确梯度法的频域分析
我们研究了在相对误差模型下,针对强凸函数以及扇形界梯度的较大类函数的一些基于迭代梯度的方法的鲁棒性。相应收敛速度的证明基于非线性系统稳定性的频域准则。给出了不精确的梯度下降和三重动量法的应用。为了进一步强调频域方法的有用性,我们得出了Nesterov加速方法(在精确设置下)在强凸函数上的收敛速度的改进解析界。