SIAM Journal on Optimization, Volume 30, Issue 2, Page 1300-1326, January 2020.
In 1988, Barzilai and Borwein published a pioneering paper which opened the way to inexpensively accelerate first-order. In more detail, in the framework of unconstrained optimization, Barzilai and Borwein developed two strategies to select the step length in gradient descent methods with the aim of encoding some second-order information of the problem without computing and/or employing the Hessian matrix of the objective function. Starting from these ideas, several efficient step length techniques have been suggested in the last decades in order to make gradient descent methods more and also more appealing for problems which handle large-scale data and require real-time solutions. Typically, these new step length selection rules have been tuned in the quadratic unconstrained framework for sweeping the spectrum of the Hessian matrix, and then applied also to nonquadratic constrained problems, without any substantial modification, by showing them to be very effective anyway. In this paper, we deeply analyze how, in quadratic and nonquadratic minimization problems, the presence of a feasible region, expressed by a single linear equality constraint together with lower and upper bounds, influences the spectral properties of the original Barzilai--Borwein (BB) rules, generalizing recent results provided for box-constrained quadratic problems. This analysis gives rise to modified BB approaches able not only to capture second-order information but also to exploit the nature of the feasible region. We show the benefits gained by the new step length rules on a set of test problems arising also from machine learning and image processing applications.

中文翻译：

---

上下界约束下单线性约束最优化问题的Barzilai-Borwein规则的谱性质

SIAM优化杂志，第30卷，第2期，第1300-1326页，2020年1月。
1988年，Barzilai和Borwein发表了一篇开创性的论文，这为廉价地加速一阶运算开辟了道路。更详细地讲，在无约束优化的框架中，Barzilai和Borwein开发了两种策略来选择梯度下降方法中的步长，目的是编码问题的某些二阶信息，而无需计算和/或使用Hessian矩阵。目标函数。从这些想法出发，在过去的几十年中，已经提出了几种有效的步长技术，以使梯度下降方法越来越适用于处理大规模数据并需要实时解决方案的问题。通常，这些新的步长选择规则已在二次无约束框架中进行了调整，以扫描Hessian矩阵的频谱，然后通过证明它们非常有效，也无需进行任何实质性修改即可将其应用于非二次约束问题。在本文中，我们深入分析了在二次和非二次最小化问题中，由单个线性等式约束以及上下限表示的可行区域的存在如何影响原始Barzilai-Borwein（BB ）规则，概括了针对框约束二次问题提供的最新结果。这种分析产生了改进的BB方法，该方法不仅可以捕获二阶信息，而且还可以利用可行区域的性质。我们针对机器学习和图像处理应用程序中出现的一系列测试问题，展示了新的步长规则所带来的好处。