Learning to improve AUC performance for imbalanced data is an important machine learning research problem. Most methods of AUC maximization assume that the model function is linear in the original feature space. However, this assumption is not suitable for nonlinear separable problems. Although there have been some nonlinear methods of AUC maximization, scaling up nonlinear AUC maximization is still an open question. To address this challenging problem, in this paper, we propose a novel large-scale nonlinear AUC maximization method (named as TSAM) based on the triply stochastic gradient descents. Specifically, we first use the random Fourier feature to approximate the kernel function. After that, we use the triply stochastic gradients w.r.t. the pairwise loss and random feature to iteratively update the solution. Finally, we prove that TSAM converges to the optimal solution with the rate of O(1/t) \mathcal {O}(1/t) after tt iterations. Experimental results on a variety of benchmark datasets not only confirm the scalability of TSAM, but also show a significant reduction of computational time compared with existing batch learning algorithms, while retaining the similar generalization performance.

中文翻译：

---

通过三重随机梯度实现大规模非线性 AUC 最大化


学习提高不平衡数据的AUC性能是一个重要的机器学习研究问题。大多数 AUC 最大化方法都假设模型函数在原始特征空间中是线性的。然而，该假设不适用于非线性可分离问题。尽管已经有一些 AUC 最大化的非线性方法，但扩大非线性 AUC 最大化仍然是一个悬而未决的问题。为了解决这个具有挑战性的问题，在本文中，我们提出了一种基于三重随机梯度下降的新型大规模非线性 AUC 最大化方法（称为 TSAM）。具体来说，我们首先使用随机傅里叶特征来近似核函数。之后，我们使用三重随机梯度与成对损失和随机特征来迭代更新解决方案。最后，我们证明 TSAM 在 tt 次迭代后以 O(1/t) \mathcal {O}(1/t) 的速率收敛到最优解。各种基准数据集上的实验结果不仅证实了 TSAM 的可扩展性，而且与现有的批量学习算法相比，计算时间显着减少，同时保留了相似的泛化性能。