Sparse linear regression is a fundamental problem in high-dimensional statistics, but strikingly little is known about how to efficiently solve it without restrictive conditions on the design matrix. We consider the (correlated) random design setting, where the covariates are independently drawn from a multivariate Gaussian $N(0,\Sigma)$ with $\Sigma : n \times n$, and seek estimators $\hat{w}$ minimizing $(\hat{w}-w^*)^T\Sigma(\hat{w}-w^*)$, where $w^*$ is the $k$-sparse ground truth. Information theoretically, one can achieve strong error bounds with $O(k \log n)$ samples for arbitrary $\Sigma$ and $w^*$; however, no efficient algorithms are known to match these guarantees even with $o(n)$ samples, without further assumptions on $\Sigma$ or $w^*$. As far as hardness, computational lower bounds are only known with worst-case design matrices. Random-design instances are known which are hard for the Lasso, but these instances can generally be solved by Lasso after a simple change-of-basis (i.e. preconditioning). In this work, we give upper and lower bounds clarifying the power of preconditioning in sparse linear regression. First, we show that the preconditioned Lasso can solve a large class of sparse linear regression problems nearly optimally: it succeeds whenever the dependency structure of the covariates, in the sense of the Markov property, has low treewidth -- even if $\Sigma$ is highly ill-conditioned. Second, we construct (for the first time) random-design instances which are provably hard for an optimally preconditioned Lasso. In fact, we complete our treewidth classification by proving that for any treewidth-$t$ graph, there exists a Gaussian Markov Random Field on this graph such that the preconditioned Lasso, with any choice of preconditioner, requires $\Omega(t^{1/20})$ samples to recover $O(\log n)$-sparse signals when covariates are drawn from this model.

中文翻译：

---

稀疏线性回归中预处理的威力

稀疏线性回归是高维统计中的一个基本问题，但对于如何在没有设计矩阵限制条件的情况下有效地解决它，我们知之甚少。我们考虑（相关）随机设计设置，其中协变量独立地从多元高斯 $N(0,\Sigma)$ 和 $\Sigma : n \times n$ 中提取，并寻求估计量 $\hat{w}$最小化 $(\hat{w}-w^*)^T\Sigma(\hat{w}-w^*)$，其中 $w^*$ 是 $k$-sparse ground truth。理论上，对于任意 $\Sigma$ 和 $w^*$，可以使用 $O(k \log n)$ 样本实现强误差界限；然而，在没有对 $\Sigma$ 或 $w^*$ 做进一步假设的情况下，即使使用 $o(n)$ 样本，也没有已知的有效算法可以匹配这些保证。就硬度而言，计算下限仅在最坏情况设计矩阵中已知。已知随机设计实例对 Lasso 来说很难，但这些实例通常可以在简单的改变基（即预处理）后由 Lasso 解决。在这项工作中，我们给出了上限和下限，阐明了稀疏线性回归中预处理的能力。首先，我们证明了预处理的 Lasso 可以几乎最优地解决一大类稀疏线性回归问题：只要协变量的依赖结构（在马尔可夫性质的意义上）具有低树宽——即使 $\Sigma$病情严重。其次，我们构造（第一次）随机设计实例，这对于最佳预处理套索来说是困难的。事实上，我们通过证明对于任何 treewidth-$t$ 图形来完成我们的树宽分类，