Abstract

Recent advances in mathematical programming have made mixed integer optimization a competitive alternative to popular regularization methods for selecting features in regression problems. The approach exhibits unquestionable foundational appeal and versatility, but also poses important challenges. Here, we propose MIP-BOOST, a revision of standard mixed integer programming feature selection that reduces the computational burden of tuning the critical sparsity bound parameter and improves performance in the presence of feature collinearity and of signals that vary in nature and strength. The final outcome is a more efficient and effective L₀ feature selection method for applications of realistic size and complexity, grounded on rigorous cross-validation tuning and exact optimization of the associated mixed integer program. Computational viability and improved performance in realistic scenarios is achieved through three independent but synergistic proposals. Supplementary materials including additional results, pseudocode, and computer code are available online.

摘要

数学规划的最新进展使混合整数优化成为流行的正则化方法的竞争替代方案，用于在回归问题中选择特征。该方法展示了无可置疑的基础吸引力和多功能性，但也带来了重大挑战。在这里，我们提出了 MIP-BOOST，这是对标准混合整数规划特征选择的修订版，它减少了调整临界稀疏边界参数的计算负担，并在存在特征共线性以及性质和强度不同的信号的情况下提高了性能。最终结果是更高效和有效的L ₀基于严格的交叉验证调整和相关混合整数程序的精确优化，用于实际大小和复杂性应用程序的特征选择方法。通过三个独立但协同的提案实现了现实场景中的计算可行性和改进的性能。在线提供包括附加结果、伪代码和计算机代码在内的补充材料。