SIAM Journal on Scientific Computing, Volume 42, Issue 6, Page A3979-A4016, January 2020.
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods for solving three closely related problems: a convex stochastic quadratic optimization problem, a best approximation problem, and its dual, a concave quadratic maximization problem. Among the methods studied are stochastic gradient descent, stochastic Newton, stochastic proximal point, and stochastic subspace ascent. A common feature of these methods is that in their update rule a certain subproblem needs to be solved exactly. We relax this requirement by allowing for the subproblem to be solved inexactly. We provide iteration complexity results under several assumptions on the inexactness error. Inexact variants of many popular and some more exotic methods, including randomized block Kaczmarz, Gaussian block Kaczmarz, and randomized block coordinate descent, can be cast as special cases. Numerical experiments demonstrate the benefits of allowing inexactness.

中文翻译：

---

不精确随机迭代方法的收敛性分析

SIAM科学计算杂志，第42卷，第6期，第A3979-A4016页，2020年1月。
在本文中，我们提出了几种随机迭代方法的不精确变体的收敛速度分析，用于解决三个紧密相关的问题：凸随机二次优化问题，最佳逼近问题及其对偶，凹二次最大化问题。研究的方法包括随机梯度下降，随机牛顿法，随机近点法和随机子空间上升法。这些方法的一个共同特征是，在它们的更新规则中，需要精确地解决某些子问题。我们通过允许子问题不精确地解决来放宽此要求。我们根据不精确性误差的几种假设提供迭代复杂度结果。许多流行的和一些更奇特的方法的不精确变体，包括随机块Kaczmarz，高斯块Kaczmarz，随机块坐标下降，可以作为特殊情况。数值实验证明了允许不精确的好处。