Feature selection is an important and effective data preprocessing method, which can remove the noise and redundant features while retaining the relevant and discriminative features in high-dimensional data. In real-world applications, the relationships between data samples and their labels are usually nonlinear. However, most of the existing feature selection models focus on learning a linear transformation matrix, which cannot capture such a nonlinear structure in practice and will degrade the performance of downstream tasks. To address the issue, we propose a novel nonlinear feature selection method to select those most relevant and discriminative features in high-dimensional dataset. Specifically, our method learns the nonlinear structure of high-dimensional data by a neural network with cross entropy loss function, and then using the structured sparsity norm such as ℓ2,p\ell _{2,p} -norm to regularize the weights matrix connecting the input layer and the first hidden layer of the neural network model to learn weight of each feature. Therefore, a structural sparse weights matrix is obtained by conducting nonlinear learning based on a neural network with structured sparsity regularization. Then, we use the gradient descent method to achieve the optimal solution of the proposed model. Evaluating the experimental results on several synthetic datasets and real-world datasets shows the effectiveness and superiority of the proposed nonlinear feature selection model.

中文翻译：

---

通过结构化稀疏正则化的非线性特征选择神经网络


特征选择是一种重要且有效的数据预处理方法，可以去除高维数据中的噪声和冗余特征，同时保留相关性和判别性特征。在实际应用中，数据样本与其标签之间的关系通常是非线性的。然而，现有的特征选择模型大多专注于学习线性变换矩阵，在实践中无法捕获这种非线性结构，并且会降低下游任务的性能。为了解决这个问题，我们提出了一种新颖的非线性特征选择方法来选择高维数据集中那些最相关和最具辨别力的特征。具体来说，我们的方法通过具有交叉熵损失函数的神经网络来学习高维数据的非线性结构，然后使用结构化稀疏范数如 ℓ2,p\ell _{2,p} -norm 来正则化权重矩阵连接神经网络模型的输入层和第一隐藏层以学习每个特征的权重。因此，基于结构稀疏正则化的神经网络进行非线性学习，得到结构稀疏权值矩阵。然后，我们使用梯度下降法来实现所提出模型的最优解。评估几个合成数据集和真实世界数据集的实验结果表明了所提出的非线性特征选择模型的有效性和优越性。