In high-dimensional datasets where the number of covariates far exceeds the number of observations, the most popular prediction methods make strong modeling assumptions. Unfortunately, these methods struggle to scale up in model complexity as the number of observations grows. To this end, we consider using neural networks because they span a wide range of model capacities, from sparse linear models to deep neural networks. Because neural networks are notoriously tedious to tune and train, our aim is to develop a convenient procedure that employs a minimal number of hyperparameters. Our method, Ensemble by Averaging Sparse-Input hiERarchical networks (EASIER-net), employs only two L₁-penalty parameters, one that controls the input sparsity and another for the number of hidden layers and nodes. EASIER-net selects the true support with high probability when there is sufficient evidence; otherwise, it performs variable selection with uncertainty quantification, where strongly correlated covariates are selected at similar rates. On a large collection of gene expression datasets, EASIER-net achieved higher classification accuracy and selected fewer genes than existing methods. We found that EASIER-net adaptively selected the model complexity: it fit deep networks when there was sufficient information to learn nonlinearities and interactions and fit sparse logistic models for smaller datasets with less information.

中文翻译：

---

用于高维数据集的集成稀疏输入层次网络

在协变量数量远远超过观察数量的高维数据集中，最流行的预测方法做出了强有力的建模假设。不幸的是，随着观察次数的增加，这些方法难以扩大模型的复杂性。为此，我们考虑使用神经网络，因为它们涵盖了广泛的模型容量，从稀疏线性模型到深度神经网络。由于神经网络的调优和训练是出了名的乏味，我们的目标是开发一种使用最少超参数的便捷程序。我们的方法，Ensemble by Averaging Sparse-Input hiERarchical networks (EASIER-net)，仅使用两个L ₁- 惩罚参数，一个控制输入稀疏性，另一个控制隐藏层和节点的数量。EASIER-net 在有充分证据的情况下，以高概率选择真正的支持；否则，它使用不确定性量化执行变量选择，其中以相似的速率选择强相关的协变量。在大量基因表达数据集上，与现有方法相比，EASIER-net 实现了更高的分类准确率并选择了更少的基因。我们发现 EASIER-net 自适应地选择了模型复杂度：当有足够的信息来学习非线性和交互时，它适合深度网络，并适合信息较少的较小数据集的稀疏逻辑模型。