We develop a new primitive for stochastic optimization: a low-bias, low-cost estimator of the minimizer $x_\star$ of any Lipschitz strongly-convex function. In particular, we use a multilevel Monte-Carlo approach due to Blanchet and Glynn to turn any optimal stochastic gradient method into an estimator of $x_\star$ with bias $\delta$, variance $O(\log(1/\delta))$, and an expected sampling cost of $O(\log(1/\delta))$ stochastic gradient evaluations. As an immediate consequence, we obtain cheap and nearly unbiased gradient estimators for the Moreau-Yoshida envelope of any Lipschitz convex function, allowing us to perform dimension-free randomized smoothing. We demonstrate the potential of our estimator through four applications. First, we develop a method for minimizing the maximum of $N$ functions, improving on recent results and matching a lower bound up logarithmic factors. Second and third, we recover state-of-the-art rates for projection-efficient and gradient-efficient optimization using simple algorithms with a transparent analysis. Finally, we show that an improved version of our estimator would yield a nearly linear-time, optimal-utility, differentially-private non-smooth stochastic optimization method.

中文翻译：

---

减少随机偏差的梯度方法

我们为随机优化开发了一个新的原语：任何 Lipschitz 强凸函数的极小值 $x_\star$ 的低偏差、低成本估计器。特别是，由于 Blanchet 和 Glynn，我们使用多级蒙特卡洛方法将任何最优随机梯度方法转换为 $x_\star$ 的估计量，偏差 $\delta$，方差 $O(\log(1/\delta ))$，以及 $O(\log(1/\delta))$ 随机梯度评估的预期采样成本。作为一个直接的结果，我们获得了任何 Lipschitz 凸函数的 Moreau-Yoshida 包络的廉价且几乎无偏的梯度估计量，使我们能够执行无维数随机平滑。我们通过四个应用程序展示了我们的估算器的潜力。首先，我们开发了一种最小化 $N$ 函数最大值的方法，改进最近的结果并匹配对数因子的下限。第二和第三，我们使用具有透明分析的简单算法恢复投影效率和梯度效率优化的最先进速率。最后，我们展示了我们的估计器的改进版本将产生接近线性时间、最优效用、差异私有的非平滑随机优化方法。