This paper concerns the folded concave penalized sparse linear regression (FCPSLR), a class of popular sparse recovery methods. Although FCPSLR yields desirable recovery performance when solved globally, computing a global solution is NP-complete. Despite some existing statistical performance analyses on local minimizers or on specific FCPSLR-based learning algorithms, it still remains open questions whether local solutions that are known to admit fully polynomial-time approximation schemes (FPTAS) may already be sufficient to ensure the statistical performance, and whether that statistical performance can be non-contingent on the specific designs of computing procedures. To address the questions, this paper presents the following threefold results: (1) Any local solution (stationary point) is a sparse estimator, under some conditions on the parameters of the folded concave penalties. (2) Perhaps more importantly, any local solution satisfying a significant subspace second-order necessary condition (S$$^3$$3ONC), which is weaker than the second-order KKT condition, yields a bounded error in approximating the true parameter with high probability. In addition, if the minimal signal strength is sufficient, the S$$^3$$3ONC solution likely recovers the oracle solution. This result also explicates that the goal of improving the statistical performance is consistent with the optimization criteria of minimizing the suboptimality gap in solving the non-convex programming formulation of FCPSLR. (3) We apply (2) to the special case of FCPSLR with minimax concave penalty and show that under the restricted eigenvalue condition, any S$$^3$$3ONC solution with a better objective value than the Lasso solution entails the strong oracle property. In addition, such a solution generates a model error (ME) comparable to the optimal but exponential-time sparse estimator given a sufficient sample size, while the worst-case ME is comparable to the Lasso in general. Furthermore, to guarantee the S$$^3$$3ONC admits FPTAS.

中文翻译：

---

折叠凹面惩罚稀疏线性回归：局部解的稀疏性、统计性能和算法理论

本文涉及折叠凹惩罚稀疏线性回归 (FCPSLR)，这是一类流行的稀疏恢复方法。尽管 FCPSLR 在全局求解时产生了理想的恢复性能，但计算全局解是 NP 完全的。尽管对局部最小化器或基于特定 FCPSLR 的学习算法进行了一些现有的统计性能分析，但已知的允许完全多项式时间近似方案 (FPTAS) 的局部解是否已经足以确保统计性能，这仍然是一个悬而未决的问题，以及该统计性能是否不取决于计算程序的特定设计。为了解决这些问题，本文提出了以下三重结果：（1）任何局部解（静止点）都是稀疏估计量，在某些条件下折叠凹面惩罚的参数。(2) 也许更重要的是，任何满足比二阶 KKT 条件弱的重要子空间二阶必要条件 (S$$^3$$3ONC) 的局部解在逼近真实参数时都会产生有界误差很有可能。此外，如果最小信号强度足够，则 S$$^3$$3ONC 解决方案可能会恢复 oracle 解决方案。这一结果也说明，提高统计性能的目标与求解 FCPSLR 的非凸规划公式时最小化次优差距的优化准则是一致的。(3) 我们将 (2) 应用于具有极小极大凹惩罚的 FCPSLR 的特殊情况，并表明在受限特征值条件下，任何具有比 Lasso 解决方案更好的客观价值的 S$$^3$$3ONC 解决方案都需要强大的预言机属性。此外，在给定足够的样本大小的情况下，这种解决方案会生成与最优但指数时间稀疏估计器相当的模型误差 (ME)，而最坏情况的 ME 与一般的套索相当。此外，为了保证 S$$^3$$3ONC 承认 FPTAS。