SIAM Journal on Matrix Analysis and Applications, Volume 42, Issue 2, Page 800-831, January 2021.
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size of the sketching matrix a priori; and there is a nontrivial storage cost of the sketched system. In this work, we make progress towards addressing these issues by implicitly generating the sketched system and solving it simultaneously through an iterative procedure. As a result, we replace the question of the size of the sketching matrix with determining appropriate stopping criteria; we also avoid the costs of explicitly representing the sketched linear system; and our implicit representation also solves the system at the same time, which controls the per-iteration computational costs. Additionally, our approach allows us to generate a connection between random sketching methods and randomized iterative solvers (e.g., the randomized Kaczmarz method and randomized Gauss--Seidel). As a consequence, we exploit this connection to (1) produce a stronger, more precise convergence theory for such randomized iterative solvers under arbitrary sampling schemes (i.i.d., adaptive, permutation, dependent, etc.), and (2) improve the rates of convergence of randomized iterative solvers at the expense of a user-determined increase in per-iteration computational and storage costs. We demonstrate these concepts on numerical examples on 49 distinct linear systems.

中文翻译：

---

随机绘制的线性系统的隐式表示和迭代解

SIAM 矩阵分析与应用杂志，第 42 卷，第 2 期，第 800-831 页，2021 年 1 月。
随机线性系统求解器已变得流行，因为它们有可能降低浮点复杂度，同时仍能达到理想的收敛速度。一类特别有前途的方法，即随机草图求解器，在理论上已经达到了最著名的计算复杂度界限，但由于两个实际考虑而变得迟钝：没有明确的方法可以先验地选择草图矩阵的大小；并且草图系统有一个重要的存储成本。在这项工作中，我们通过隐式生成草图系统并通过迭代过程同时解决它来解决这些问题。因此，我们用确定适当的停止标准代替了草图矩阵的大小问题；我们还避免了明确表示草图线性系统的成本；并且我们的隐式表示同时也解决了系统，它控制了每次迭代的计算成本。此外，我们的方法允许我们生成随机草图方法和随机迭代求解器（例如，随机 Kaczmarz 方法和随机 Gauss--Seidel）之间的联系。因此，我们利用这种联系来 (1) 在任意采样方案（iid、自适应、置换、依赖等）下为此类随机迭代求解器生成更强大、更精确的收敛理论，以及 (2) 提高随机迭代求解器的收敛，代价是用户确定的每次迭代计算和存储成本的增加。