Online learning represents an important family of efficient and scalable algorithms for large-scale classification problems. Many of them are linear with fast computational speed, but when faced with complex classification, they more likely have low accuracies. In order to improve accuracies, kernel trick is applied, however, it often brings high computational cost. In fact, discriminative information is vital in classification which is still not fully utilized in these algorithms. In this paper, we proposed a novel online linear method, called Sketch Discriminatively Regularized Online Gradient Descent Classification (SDROGD). In order to exploit inter-class separability and intra-class compactness, SDROGD utilizes a matrix to characterize the discriminative information and embeds it directly into a new regularization term. This matrix can be updated by the sketch technique in an online manner. After applying a simple but effective optimization, we show that SDROGD has a good time complexity bound, which is linear with the feature dimension or the number of samples. Experimental results on both toy and real-world datasets demonstrate that SDROGD has not only faster computational speed but also much better classification accuracies than some related kernelized algorithms.

在线学习代表了针对大规模分类问题的重要的高效可扩展算法系列。它们中的许多都是线性的，具有快速的计算速度，但是当面对复杂的分类时，它们更可能具有较低的精度。为了提高精度，应用了内核技巧，但是它经常带来很高的计算成本。实际上，区分性信息在分类中至关重要，而分类算法中这些信息仍未得到充分利用。在本文中，我们提出了一种新颖的在线线性方法，称为草图判别正则化在线梯度下降分类（SDROGD）。为了利用类间的可分离性和类内的紧凑性，SDROGD利用矩阵来表征区分性信息，并将其直接嵌入新的正则化项中。可以通过草图技术以在线方式更新此矩阵。应用简单但有效的优化后，我们证明SDROGD具有良好的时间复杂度界限，与特征维数或样本数量呈线性关系。在玩具和真实数据集上的实验结果表明，SDROGD不仅比某些相关的核化算法具有更快的计算速度而且具有更好的分类准确性。