当前位置: X-MOL 学术Optim. Lett. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Zeroth-order methods for noisy Hölder-gradient functions
Optimization Letters ( IF 1.6 ) Pub Date : 2021-04-29 , DOI: 10.1007/s11590-021-01742-z
Innokentiy Shibaev , Pavel Dvurechensky , Alexander Gasnikov

In this paper, we prove new complexity bounds for zeroth-order methods in non-convex optimization with inexact observations of the objective function values. We use the Gaussian smoothing approach of Nesterov and Spokoiny(Found Comput Math 17(2): 527–566, 2015. https://doi.org/10.1007/s10208-015-9296-2) and extend their results, obtained for optimization methods for smooth zeroth-order non-convex problems, to the setting of minimization of functions with Hölder-continuous gradient with noisy zeroth-order oracle, obtaining noise upper-bounds as well. We consider finite-difference gradient approximation based on normally distributed random Gaussian vectors and prove that gradient descent scheme based on this approximation converges to the stationary point of the smoothed function. We also consider convergence to the stationary point of the original (not smoothed) function and obtain bounds on the number of steps of the algorithm for making the norm of its gradient small. Additionally we provide bounds for the level of noise in the zeroth-order oracle for which it is still possible to guarantee that the above bounds hold. We also consider separately the case of \(\nu = 1\) and show that in this case the dependence of the obtained bounds on the dimension can be improved.



中文翻译:

嘈杂的Hölder梯度函数的零阶方法

在本文中,我们通过对目标函数值的不精确观测,证明了非凸优化中零阶方法的新复杂度界限。我们使用Nesterov和Spokoiny的高斯平滑方法(Found Comput Math 17(2):527–566,2015. https://doi.org/10.1007/s10208-015-9296-2)并扩展它们的结果,以得到平滑零阶非凸问题的最优化方法,设置带有带噪声的零阶预言值的Hölder连续梯度函数的最小化设置,以及获得噪声上限。我们考虑基于正态分布随机高斯向量的有限差分梯度近似,并证明基于该近似的梯度下降方案收敛到平滑函数的平稳点。我们还考虑收敛到原始(未平滑)函数的固定点,并获得算法步数的界限,以使其梯度范数变小。另外,我们提供了零阶预言中噪声水平的界限,对于该界限,仍然可以保证上述界限成立。我们还单独考虑以下情况\(\ nu = 1 \)并表明在这种情况下,可以改善获得的边界对维的依赖性。

更新日期:2021-04-29
down
wechat
bug