SIAM Review, Volume 62, Issue 2, Page 395-436, January 2020.
We consider minimization of indefinite quadratics with either trust-region (norm) constraints or cubic regularization. Despite the nonconvexity of these problems we prove that, under mild assumptions, gradient descent converges to their global solutions and give a nonasymptotic rate of convergence for the cubic variant. We also consider Krylov subspace solutions and establish sharp convergence guarantees to the solutions of both trust-region and cubic-regularized problems. Our rates mirror the behavior of these methods on convex quadratics and eigenvector problems, highlighting their scalability. When we use Krylov subspace solutions to approximate the cubic-regularized Newton step, our results recover the strongest known convergence guarantees to approximate second-order stationary points of general smooth nonconvex functions.

中文翻译：

---

非凸二次最小化的一阶方法

SIAM评论，第62卷，第2期，第395-436页，2020年1月。
我们考虑使用信任区域（范数）约束或三次正则化来最小化不确定二次方。尽管这些问题不具有凸性，但我们证明，在温和的假设下，梯度下降收敛至其整体解，并且给出了三次变体的非渐近收敛速率。我们还考虑了Krylov子空间解决方案，并为信任区域和三次正则问题的解决方案建立了清晰的收敛性保证。我们的费率反映了这些方法在凸二次方程和特征向量问题上的行为，突出了它们的可扩展性。当我们使用Krylov子空间解来近似三次正则化的牛顿步时，我们的结果将恢复最强大的已知收敛性保证，以近似于一般光滑非凸函数的二阶固定点。