Abstract

Semi-Supervised Learning (SSL) has attracted much attention in the field of machine learning and data mining. As an extension of Support Vector Machine (SVM), the Semi-Supervised Support Vector Machine (S³VM) was proposed for SSL. Recent studies have disclosed that optimising the margin distribution is more crucial than maximising the minimum margin in generating a better classification. However, the existing S³VM models still follow the idea of maximising the minimum margin. Therefore, this paper proposes a novel Laplacian Large margin Distribution Machine (LapLDM) for SSL to enhance the classification performance. This method can optimise the margin distribution by maximising the first-order (margin mean) and minimising the second-order (margin variance) statistics of margins, and exploit the geometry information of marginal distribution embedded in the unlabelled data through the Laplacian regularizer. Then this paper develops a Preconditioned Conjugate Gradient (PCG) algorithm to solve the nonlinear LapLDM model on those regular-scaled data sets and a Stochastic Gradient Descent with Variance Reduction (SVRG) algorithm to solve the linear LapLDM model on those large-scaled data sets. These algorithms can accelerate the implementing efficiencies of proposed models and make them available for those large-scaled problems. Finally, the numerical results on four artificial and fourteen public benchmark data sets demonstrate that the LapLDM is superior to some well-known S³VM models.

摘要

半监督学习（SSL）在机器学习和数据挖掘领域引起了广泛关注。作为支持向量机 (SVM) 的扩展，针对 SSL 提出了半监督支持向量机 (S ³ VM)。最近的研究表明，在生成更好的分类时，优化边距分布比最大化最小边距更为重要。然而，现有的 S ³VM 模型仍然遵循最大化最小边距的想法。因此，本文提出了一种用于 SSL 的新型拉普拉斯大边缘分布机 (LapLDM) 以提高分类性能。该方法可以通过最大化边缘的一阶（边缘均值）和最小化边缘的二阶（边缘方差）统计量来优化边缘分布，并通过拉普拉斯正则化器利用嵌入在未标记数据中的边缘分布的几何信息。然后，本文开发了一种预处理共轭梯度 (PCG) 算法来求解那些常规尺度数据集上的非线性 LapLDM 模型，以及一种具有方差减少的随机梯度下降 (SVRG) 算法来求解那些大规模数据集上的线性 LapLDM 模型. 这些算法可以加快提出的模型的实施效率，并使它们可用于那些大规模的问题。最后，四个人工和十四个公共基准数据集的数值结果表明，LapLDM 优于一些著名的 S³ VM 模型。