Frontiers of Information Technology & Electronic Engineering ( IF 3 ) Pub Date : 2019-09-05 , DOI: 10.1631/fitee.1800566 Lei Guan , Tao Sun , Lin-bo Qiao , Zhi-hui Yang , Dong-sheng Li , Ke-shi Ge , Xi-cheng Lu
Support vector machines (SVMs) have been recognized as a powerful tool to perform linear classification. When combined with the sparsity-inducing nonconvex penalty, SVMs can perform classification and variable selection simultaneously. However, the nonconvex penalized SVMs in general cannot be solved globally and efficiently due to their nondifferentiability, nonconvexity, and nonsmoothness. Existing solutions to the nonconvex penalized SVMs typically solve this problem in a serial fashion, which are unable to fully use the parallel computing power of modern multi-core machines. On the other hand, the fact that many real-world data are stored in a distributed manner urgently calls for a parallel and distributed solution to the nonconvex penalized SVMs. To circumvent this challenge, we propose an efficient alternating direction method of multipliers (ADMM) based algorithm that solves the nonconvex penalized SVMs in a parallel and distributed way. We design many useful techniques to decrease the computation and synchronization cost of the proposed parallel algorithm. The time complexity analysis demonstrates the low time complexity of the proposed parallel algorithm. Moreover, the convergence of the parallel algorithm is guaranteed. Experimental evaluations on four LIBSVM benchmark datasets demonstrate the efficiency of the proposed parallel algorithm.
中文翻译:
非凸罚线性SVM的高效并行和分布式解决方案
支持向量机(SVM)被认为是执行线性分类的强大工具。当与导致稀疏性的非凸罚分结合使用时,SVM可以同时执行分类和变量选择。但是,由于非凸罚分SVM的不可微性,非凸性和不平滑性,因此一般无法全局有效地解决。对于非凸面惩罚型SVM的现有解决方案通常以串行方式解决此问题,而这种方式无法充分利用现代多核计算机的并行计算能力。另一方面,许多现实世界数据以分布式方式存储的事实迫切要求对非凸罚分SVM进行并行和分布式解决方案。为了规避这一挑战,我们提出了一种基于乘数的有效交替方向算法(ADMM),该算法以并行和分布式方式解决了非凸惩罚SVM。我们设计了许多有用的技术来减少所提出的并行算法的计算和同步成本。时间复杂度分析证明了所提出的并行算法的低时间复杂度。而且,保证了并行算法的收敛性。对四个LIBSVM基准数据集的实验评估证明了所提出的并行算法的效率。时间复杂度分析证明了所提出的并行算法的低时间复杂度。而且,保证了并行算法的收敛性。对四个LIBSVM基准数据集的实验评估证明了所提出的并行算法的效率。时间复杂度分析证明了所提出的并行算法的低时间复杂度。而且,保证了并行算法的收敛性。对四个LIBSVM基准数据集的实验评估证明了所提出的并行算法的效率。