SIAM Journal on Optimization, Volume 30, Issue 1, Page 349-376, January 2020.
For deterministic optimization, line search methods augment algorithms by providing stability and improved efficiency. Here we adapt a classical backtracking Armijo line search to the stochastic optimization setting. While traditional line search relies on exact computations of the gradient and values of the objective function, our method assumes that these values are available up to some dynamically adjusted accuracy which holds with some sufficiently large, but fixed, probability. We bound the expected number of iterations to reach a desired first-order accuracy in the nonconvex, convex, and strongly convex cases and show that this bound matches the complexity bound of deterministic gradient descent up to constants.

中文翻译：

---

具有期望复杂度分析的随机线搜索方法

SIAM优化杂志，第30卷，第1期，第349-376页，2020年1月。
对于确定性优化，线搜索方法通过提供稳定性和改进的效率来增强算法。在这里，我们将经典的回溯Armijo线搜索调整为随机优化设置。尽管传统的线搜索依赖于目标函数的梯度和值的精确计算，但我们的方法假定这些值可以以某种动态调整的精度获得，并且具有足够大但固定的概率。我们限制了预期的迭代次数，以在非凸，凸和强凸情况下达到所需的一阶精度，并表明该边界与确定性梯度下降直至常数的复杂度边界相匹配。