当前位置: X-MOL 学术Sci. Program. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Fast Training Logistic Regression via Adaptive Sampling
Scientific Programming ( IF 1.672 ) Pub Date : 2021-05-30 , DOI: 10.1155/2021/9991859
Yunsheng Song 1 , Xiaohan Kong 1 , Shuoping Huang 1 , Chao Zhang 1
Affiliation  

Logistic regression has been widely used in artificial intelligence and machine learning due to its deep theoretical basis and good practical performance. Its training process aims to solve a large-scale optimization problem characterized by a likelihood function, where the gradient descent approach is the most commonly used. However, when the data size is large, it is very time-consuming because it computes the gradient using all the training data in every iteration. Though this difficulty can be solved by random sampling, the appropriate sampled examples size is difficult to be predetermined and the obtained could be not robust. To overcome this deficiency, we propose a novel algorithm for fast training logistic regression via adaptive sampling. The proposed method decomposes the problem of gradient estimation into several subproblems according to its dimension; then, each subproblem is solved independently by adaptive sampling. Each element of the gradient estimation is obtained by successively sampling a fixed volume training example multiple times until it satisfies its stopping criteria. The final estimation is combined with the results of all the subproblems. It is proved that the obtained gradient estimation is a robust estimation, and it could keep the objective function value decreasing in the iterative calculation. Compared with the representative algorithms using random sampling, the experimental results show that this algorithm obtains comparable classification performance with much less training time.

中文翻译:

通过自适应采样快速训练逻辑回归

Logistic回归因其深厚的理论基础和良好的实用性能而被广泛应用于人工智能和机器学习中。其训练过程旨在解决以似然函数为特征的大规模优化问题,其中梯度下降法是最常用的。但是,当数据量很大时,它非常耗时,因为它在每次迭代中都使用所有训练数据来计算梯度。虽然这个困难可以通过随机抽样来解决,但合适的抽样样本大小很难预先确定,并且获得的结果可能不稳健。为了克服这一缺陷,我们提出了一种通过自适应采样快速训练逻辑回归的新算法。提出的方法将梯度估计问题按照维度分解为若干子问题;然后,每个子问题通过自适应采样独立解决。梯度估计的每个元素是通过对固定体积的训练样本连续多次采样直到满足其停止标准来获得的。最终估计与所有子问题的结果结合在一起。证明得到的梯度估计是一个鲁棒的估计,在迭代计算中可以保持目标函数值递减。与使用随机抽样的代表性算法相比,实验结果表明,该算法以更少的训练时间获得了可比的分类性能。每个子问题通过自适应采样独立解决。梯度估计的每个元素是通过对固定体积的训练样本连续多次采样直到满足其停止标准来获得的。最终估计与所有子问题的结果相结合。证明得到的梯度估计是一个鲁棒的估计,在迭代计算中可以保持目标函数值递减。与使用随机抽样的代表性算法相比,实验结果表明,该算法以更少的训练时间获得了可比的分类性能。每个子问题通过自适应采样独立解决。梯度估计的每个元素是通过对固定体积的训练样本连续多次采样直到满足其停止标准来获得的。最终估计与所有子问题的结果相结合。证明得到的梯度估计是一个鲁棒的估计,在迭代计算中可以保持目标函数值递减。与使用随机抽样的代表性算法相比,实验结果表明,该算法以更少的训练时间获得了可比的分类性能。最终估计与所有子问题的结果相结合。证明得到的梯度估计是一个鲁棒的估计,在迭代计算中可以保持目标函数值递减。与使用随机抽样的代表性算法相比,实验结果表明,该算法以更少的训练时间获得了可比的分类性能。最终估计与所有子问题的结果相结合。证明得到的梯度估计是一个鲁棒的估计,在迭代计算中可以保持目标函数值递减。与使用随机抽样的代表性算法相比,实验结果表明,该算法以更少的训练时间获得了可比的分类性能。
更新日期:2021-05-30
down
wechat
bug