Stochastic approximation methods play a central role in maximum likelihood estimation problems involving intractable likelihood functions, such as marginal likelihoods arising in problems with missing or incomplete data, and in parametric empirical Bayesian estimation. Combined with Markov chain Monte Carlo algorithms, these stochastic optimisation methods have been successfully applied to a wide range of problems in science and industry. However, this strategy scales poorly to large problems because of methodological and theoretical difficulties related to using high-dimensional Markov chain Monte Carlo algorithms within a stochastic approximation scheme. This paper proposes to address these difficulties by using unadjusted Langevin algorithms to construct the stochastic approximation. This leads to a highly efficient stochastic optimisation methodology with favourable convergence properties that can be quantified explicitly and easily checked. The proposed methodology is demonstrated with three experiments, including a challenging application to statistical audio analysis and a sparse Bayesian logistic regression with random effects problem.

随机逼近方法在涉及难处理的似然函数的最大似然估计问题中（例如因数据缺失或不完整的问题而产生的边际似然）以及参数经验贝叶斯估计中起着核心作用。结合马尔可夫链蒙特卡洛算法，这些随机优化方法已成功应用于科学和工业中的各种问题。但是，由于与在随机逼近方案中使用高维马尔可夫链蒙特卡罗算法相关的方法和理论上的困难，该策略无法很好地解决大型问题。本文提出通过使用未经调整的Langevin算法构造随机逼近来解决这些困难。这导致了一种高效的随机优化方法，该方法具有良好的收敛性，可以显式量化并轻松对其进行检查。通过三个实验证明了所提出的方法，包括具有挑战性的统计音频分析应用和具有随机效应问题的稀疏贝叶斯逻辑回归。