Abstract A framework previously introduced in Wilson et al. (2018) for solving a sequence of stochastic optimization problems with bounded changes in the minimizers is extended and applied to machine learning problems such as regression and classification. The stochastic optimization problems arising in these machine learning problems are solved using algorithms such as stochastic gradient descent (SGD). A method based on estimates of the change in the minimizers and properties of the optimization algorithm is introduced for adaptively selecting the number of samples at each time step to ensure that the excess risk—that is, the expected gap between the loss achieved by the approximate minimizer produced by the optimization algorithm and the exact minimizer—does not exceed a target level. A bound is developed to show that the estimate of the change in the minimizers is non trivial provided that the excess risk is small enough. Extensions relevant to the machine learning setting are considered, including a cost-based approach to select the number of samples with a cost budget over a fixed horizon, and an approach to applying cross-validation for model selection. Finally, experiments with synthetic and real data are used to validate the algorithms.

中文翻译：

---

自适应顺序机器学习

摘要 先前在 Wilson 等人中引入的框架。(2018) 用于解决最小化器有界变化的一系列随机优化问题被扩展并应用于机器学习问题，如回归和分类。这些机器学习问题中出现的随机优化问题可以使用诸如随机梯度下降 (SGD) 之类的算法来解决。引入了一种基于对优化算法的极小值和属性变化的估计的方法，用于自适应地选择每个时间步的样本数量，以确保超额风险——即通过近似实现的损失之间的预期差距优化算法产生的最小化器和精确最小化器——不超过目标水平。开发了一个界限以表明如果超额风险足够小，则对最小化变量的变化的估计是不重要的。考虑了与机器学习设置相关的扩展，包括在固定范围内选择具有成本预算的样本数量的基于成本的方法，以及将交叉验证应用于模型选择的方法。最后，使用合成数据和真实数据的实验来验证算法。