In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be approximated by stochastic oracles. The proposed method combines general stochastic higher order steps derived from an underlying proximal type fixed-point equation with additional stochastic proximal gradient steps to guarantee convergence. Based on suitable bounds on the step sizes, we establish global convergence to stationary points in expectation and an extension of the approach using variance reduction techniques is discussed. Motivated by large-scale and big data applications, we investigate a stochastic coordinate-type quasi-Newton scheme that allows to generate cheap and tractable stochastic higher order directions. Finally, numerical results on large-scale logistic regression and deep learning problems show that our proposed algorithm compares favorably with other state-of-the-art methods.

为了解决一类非光滑非凸复合优化问题，本文提出了一种新颖的随机额外步拟牛顿法。我们假设目标函数的平滑部分的梯度只能由随机预言近似。所提出的方法结合了从底层近端类型不动点方程派生的一般随机高阶步长与附加随机近端梯度步长，以确保收敛。基于步长的适当界限，我们建立了到期望中的平稳点的全局收敛，并讨论了使用方差减少技术的方法的扩展。受大规模和大数据应用的推动，我们研究了一种随机坐标类型的拟牛顿方案，该方案允许生成便宜且易于处理的随机高阶方向。最后，关于大规模逻辑回归和深度学习问题的数值结果表明，我们提出的算法与其他最新方法相比具有优势。