We here adapt an extended version of the adaptive cubic regularization method with dynamic inexact Hessian information for nonconvex optimization in Bellavia et al. [Adaptive cubic regularization methods with dynamic inexact hessian information and applications to finite-sum minimization. IMA Journal of Numerical Analysis. 2021;41(1):764–799] to the stochastic optimization setting. While exact function evaluations are still considered, this novel variant inherits the innovative use of adaptive accuracy requirements for Hessian approximations introduced in the just quoted paper and additionally employs inexact computations of the gradient. Without restrictions on the variance of the errors, we assume that these approximations are available within a sufficiently large, but fixed, probability and we extend, in the spirit of Cartis and Scheinberg [Global convergence rate analysis of unconstrained optimization methods based on probabilistic models. Math Program Ser A. 2018;159(2):337–375], the deterministic analysis of the framework to its stochastic counterpart, showing that the expected number of iterations to reach a first-order stationary point matches the well-known worst-case optimal complexity. This is, in fact, still given by $O\left({ϵ}^{-3/2}\right)$, with respect to the first-order ϵ tolerance. Finally, numerical tests on nonconvex finite-sum minimization confirm that using inexact first- and second-order derivatives can be beneficial in terms of the computational savings.

我们在这里采用了自适应三次正则化方法的扩展版本，该方法具有动态不精确的 Hessian 信息，用于 Bellavia 等人的非凸优化。[具有动态不精确 hessian 信息的自适应三次正则化方法及其在有限和最小化中的应用。IMA 数值分析杂志。2021;41(1):764–799] 到随机优化设置。虽然仍在考虑精确的函数评估，但这种新颖的变体继承了刚刚引用的论文中引入的 Hessian 近似自适应精度要求的创新使用，并且还采用了不精确的梯度计算。在不限制误差方差的情况下，我们假设这些近似值在足够大但固定的概率内可用，我们扩展，本着 Cartis 和 Scheinberg 的精神 [基于概率模型的无约束优化方法的全局收敛速度分析。Math Program Ser A. 2018;159(2):337–375]，框架对其随机对应物的确定性分析，表明达到一阶静止点的预期迭代次数与众所周知的最差-案例最优复杂度。事实上，这仍然是由$○\left({\epsilon }^{-3/2}\right)$, 关于一阶ε容差。最后，非凸有限和最小化的数值测试证实，使用不精确的一阶和二阶导数可以节省计算量。