This paper studies an acceleration technique for incremental aggregated gradient (IAG) method through the use of curvature information for solving strongly convex finite sum optimization problems. These optimization problems of interest arise in large-scale learning applications. Our technique utilizes a curvature-aided gradient tracking step to produce accurate gradient estimates incrementally using Hessian information. We propose and analyze two methods utilizing the new technique, the curvature-aided IAG (CIAG) method and the accelerated CIAG (A-CIAG) method, which are analogous to gradient method and Nesterov’s accelerated gradient method, respectively. Setting \(\kappa\) to be the condition number of the objective function, we prove the R linear convergence rates of \(1 - \frac{4c_0 \kappa }{(\kappa +1)^2}\) for the CIAG method, and \(1 - \sqrt{\frac{c_1}{2\kappa }}\) for the A-CIAG method, where \(c_0,c_1 \le 1\) are constants inversely proportional to the distance between the initial point and the optimal solution. When the initial iterate is close to the optimal solution, the R linear convergence rates match with the gradient and accelerated gradient method, albeit CIAG and A-CIAG operate in an incremental setting with strictly lower computation complexity. Numerical experiments confirm our findings. The source codes used for this paper can be found on http://github.com/hoitowai/ciag/.

中文翻译：

---

利用曲率信息加速增量梯度优化

通过利用曲率信息来解决强凸有限和优化问题，研究了一种增量聚合梯度（IAG）方法的加速技术。这些有趣的优化问题出现在大规模学习应用中。我们的技术利用曲率辅助梯度跟踪步骤，使用Hessian信息逐步生成准确的梯度估计。我们提出并分析了使用新技术的两种方法：曲率辅助IAG（CIAG）方法和加速CIAG（A-CIAG）方法，分别类似于梯度法和Nesterov的加速梯度法。设置\（\ kappa \）作为目标函数的条件数，我们证明了CIAG方法的\（1-\ frac {4c_0 \ kappa} {（\ kappa +1）^ 2} \）的R线性收敛速度，以及\（1 -\ sqrt {\ frac {c_1} {2 \ kappa}} \）用于A-CIAG方法，其中\（c_0，c_1 \ le 1 \）是与初始点和最佳解之间的距离成反比的常数。当初始迭代接近最佳解时，尽管CIAG和A-CIAG，R线性收敛速率仍与梯度和加速梯度方法匹配在增量设置中进行操作，并严格降低了计算复杂度。数值实验证实了我们的发现。可以在http://github.com/hoitowai/ciag/上找到用于本文的源代码。