Abstract

In this paper, we propose an approach based on Krylov subspace methods for the solution of ${\mathcal{\ell }}_1$ regularized logistic regression problem. The main idea is to transform the constrained ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_1$ minimization problem obtained by applying the IRLS method to a ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$ one that allow regularization matrices in the usual 2-norm regularization term. The regularization parameter that controls the equilibrium between the minimization of the two terms of the ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$ minimization problem can be then chosen inexpensively by solving some reduced minimization problems related to generalized cross-validation (GCV) methods. These reduced problems can be obtained after a few iterations of Krylov subspace based methods. The goal of our simulation study is directed toward the variable selection and the prediction accuracy performance of the proposed method in solving a ${\mathcal{\ell }}_1$ regularized logistic regression problem in large dimensional data with different correlation structures among predictors. Finally, real data are used to confirm the efficiency of the proposed method in terms of the computational cost.

摘要

在本文中，我们提出了一种基于 Krylov 子空间方法的方法来解决${\mathcal{\ell }}_1$正则化逻辑回归问题。主要思想是改变约束${\mathcal{\ell }}_2$-${\mathcal{\ell }}_1$通过将 IRLS 方法应用于${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$允许使用通常的 2 范数正则化项的正则化矩阵。控制方程两项最小化之间平衡的正则化参数${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$然后可以通过解决一些与广义交叉验证（GCV）方法相关的简化最小化问题来廉价地选择最小化问题。这些减少的问题可以在基于克雷洛夫子空间的方法的几次迭代之后获得。我们模拟研究的目标是针对所提出的方法在求解问题时的变量选择和预测精度性能。${\mathcal{\ell }}_1$预测变量之间具有不同相关结构的大维数据中的正则化逻辑回归问题。最后，使用真实数据来确认所提出方法在计算成本方面的效率。