We consider variants of a recently developed Newton-CG algorithm for nonconvex problems (Royer, C. W. & Wright, S. J. (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477) in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on the inexactness measures, we derive iteration complexity bounds for achieving $\epsilon $-approximate second-order optimality that match best-known lower bounds. Our inexactness condition on the gradient is adaptive, allowing for crude accuracy in regions with large gradients. We describe two variants of our approach, one in which the step size along the computed search direction is chosen adaptively, and another in which the step size is pre-defined. To obtain second-order optimality, our algorithms will make use of a negative curvature direction on some steps. These directions can be obtained, with high probability, using the randomized Lanczos algorithm. In this sense, all of our results hold with high probability over the run of the algorithm. We evaluate the performance of our proposed algorithms empirically on several machine learning models. Our approach is a first attempt to introduce inexact Hessian and/or gradient information into the Newton-CG algorithm of Royer & Wright (2018, Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477).

中文翻译：

---

具有复杂性保证的不精确 Newton-CG 算法

我们考虑了最近开发的用于非凸问题的 Newton-CG 算法的变体（Royer, CW & Wright, SJ (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization。SIAM J. Optim., 28, 1448– 1477），其中梯度的不精确估计和 Hessian 信息用于各个步骤。在不精确度量的某些条件下，我们推导出迭代复杂度界限，以实现与已知下界相匹配的近似二阶最优性。我们对梯度的不精确条件是自适应的，允许在具有大梯度的区域中获得粗略的准确性。我们描述了我们方法的两种变体，一种是自适应地选择沿计算搜索方向的步长，另一种是预定义步长。为了获得二阶最优性，我们的算法将在某些步骤上使用负曲率方向。这些方向可以使用随机 Lanczos 算法以高概率获得。从这个意义上说，我们所有的结果在算法运行过程中都具有很高的概率。我们在几个机器学习模型上根据经验评估我们提出的算法的性能。我们的方法是首次尝试将不精确的 Hessian 和/或梯度信息引入 Royer & Wright 的 Newton-CG 算法（2018，Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization。SIAM J. Optim., 28 , 1448–1477)。使用随机 Lanczos 算法。从这个意义上说，我们所有的结果在算法运行过程中都具有很高的概率。我们在几个机器学习模型上根据经验评估我们提出的算法的性能。我们的方法是首次尝试将不精确的 Hessian 和/或梯度信息引入 Royer & Wright 的 Newton-CG 算法（2018，Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization。SIAM J. Optim., 28 , 1448–1477)。使用随机 Lanczos 算法。从这个意义上说，我们所有的结果在算法运行过程中都具有很高的概率。我们在几个机器学习模型上根据经验评估我们提出的算法的性能。我们的方法是首次尝试将不精确的 Hessian 和/或梯度信息引入 Royer & Wright 的 Newton-CG 算法（2018，Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization。SIAM J. Optim., 28 , 1448–1477)。用于平滑非凸优化的二阶线搜索算法的复杂性分析。SIAM J. Optim., 28, 1448–1477)。用于平滑非凸优化的二阶线搜索算法的复杂性分析。SIAM J. Optim., 28, 1448–1477)。