Abstract

The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet, the degree of sparsity itself is unknown and needs to be estimated, which causes the honesty problem. To resolve this issue, we develop a novel method to construct honest confidence sets for sparse high-dimensional linear regression. The key idea in our construction is to separate signals into a strong and a weak group, and then construct confidence sets for each group separately. This is achieved by a projection and shrinkage approach, the latter implemented via Stein estimation and the associated Stein unbiased risk estimate. Our confidence set is honest over the full parameter space without any sparsity constraints, while its size adapts to the optimal rate of $n^{-1/4}$ when the true parameter is indeed sparse. Moreover, under some form of a separation assumption between the strong and weak signals, the diameter of our confidence set can achieve a faster rate than existing methods. Through extensive numerical comparisons on both simulated and real data, we demonstrate that our method outperforms other competitors with big margins for finite samples, including oracle methods built upon the true sparsity of the underlying model.

摘要

构建置信集的诚实问题出现在非参数回归中。虽然可以实现非参数估计中的最佳速率并将其用于构建清晰的置信集，但在估计平滑度之后经常会发生置信水平的严重下降。类似地，对于高维回归，可以利用稀疏估计量的 oracle 不等式来构建清晰的置信集。然而，稀疏度本身是未知的，需要估计，这会导致诚实问题。为了解决这个问题，我们开发了一种为稀疏高维线性回归构建诚实置信集的新方法。我们构建的关键思想是将信号分成强弱组，然后分别为每个组构建置信集。这是通过投影和收缩方法实现的，后者通过 Stein 估计和相关的 Stein 无偏风险估计实现。我们的置信集在整个参数空间上是诚实的，没有任何稀疏性约束，同时其大小适应最佳速率 $n^{-\mathrm{1个}/\mathrm{4个}}$ 当真实参数确实稀疏时。此外，在强信号和弱信号之间某种形式的分离假设下，我们的置信集的直径可以达到比现有方法更快的速率。通过对模拟数据和真实数据进行广泛的数值比较，我们证明我们的方法在有限样本方面优于其他竞争对手，包括建立在底层模型真正稀疏性基础上的预言机方法。