Forward selection (FS) is a popular variable selection method for linear regression. But theoretical understanding of FS with a diverging number of covariates is still limited. We derive sufficient conditions for FS to attain model selection consistency. Our conditions are similar to those for orthogonal matching pursuit, but are obtained using a different argument. When the true model size is unknown, we derive sufficient conditions for model selection consistency of FS with a data-driven stopping rule, based on a sequential variant of cross-validation. As a byproduct of our proofs, we also have a sharp (sufficient and almost necessary) condition for model selection consistency of “wrapper” forward search for linear regression. We illustrate intuition and demonstrate performance of our methods using simulation studies and real datasets.

中文翻译：

---

高维回归的前向选择和顺序交叉验证的模型选择特性

前向选择（FS）是一种流行的线性回归变量选择方法。但是对于具有不同数量协变量的 FS 的理论理解仍然有限。我们推导出 FS 获得模型选择一致性的充分条件。我们的条件类似于正交匹配追踪的条件，但使用不同的参数获得。当真实模型大小未知时，我们基于交叉验证的顺序变体，使用数据驱动的停止规则推导出 FS 的模型选择一致性的充分条件。作为我们证明的副产品，对于线性回归的“包装器”前向搜索的模型选择一致性，我们也有一个尖锐的（充分且几乎必要的）条件。我们使用模拟研究和真实数据集来说明直觉并展示我们的方法的性能。