It has been observed that the performances of many high-dimensional estimation problems are universal with respect to underlying sensing (or design) matrices. Specifically, matrices with markedly different constructions seem to achieve identical performance if they share the same spectral distribution and have ``generic'' singular vectors. We prove this universality phenomenon for the case of convex regularized least squares (RLS) estimators under a linear regression model with additive Gaussian noise. Our main contributions are two-fold: (1) We introduce a notion of universality classes for sensing matrices, defined through a set of deterministic conditions that fix the spectrum of the sensing matrix and precisely capture the previously heuristic notion of generic singular vectors; (2) We show that for all sensing matrices that lie in the same universality class, the dynamics of the proximal gradient descent algorithm for solving the regression problem, as well as the performance of RLS estimators themselves (under additional strong convexity conditions) are asymptotically identical. In addition to including i.i.d. Gaussian and rotational invariant matrices as special cases, our universality class also contains highly structured, strongly correlated, or even (nearly) deterministic matrices. Examples of the latter include randomly signed versions of incoherent tight frames and randomly subsampled Hadamard transforms. As a consequence of this universality principle, the asymptotic performance of regularized linear regression on many structured matrices constructed with limited randomness can be characterized by using the rotationally invariant ensemble as an equivalent yet mathematically more tractable surrogate.

中文翻译：

---

具有近确定性传感矩阵的正则化线性回归的谱普遍性

已经观察到，许多高维估计问题的性能对于底层传感（或设计）矩阵是普遍的。具体来说，具有显着不同结构的矩阵如果共享相同的光谱分布并具有“通用”奇异向量，则它们似乎可以实现相同的性能。我们在具有加性高斯噪声的线性回归模型下证明了凸正则化最小二乘 (RLS) 估计量的这种普遍性现象。我们的主要贡献有两个：（1）我们引入了感知矩阵的普遍性类的概念，通过一组确定性条件定义，这些条件固定感知矩阵的频谱并精确地捕获了以前的通用奇异向量的启发式概念；(2) 我们表明，对于属于同一普遍性类的所有感知矩阵，用于解决回归问题的近端梯度下降算法的动力学以及 RLS 估计器本身的性能（在额外的强凸性条件下）是渐近的完全相同的。除了包含 iid 高斯和旋转不变矩阵作为特例之外，我们的普遍性类还包含高度结构化、强相关甚至（几乎）确定性矩阵。后者的例子包括不连贯紧帧的随机签名版本和随机二次采样的 Hadamard 变换。由于这一普遍性原则，