SIAM Journal on Scientific Computing, Volume 43, Issue 1, Page A26-A53, January 2021.
As an important Markov chain Monte Carlo (MCMC) method, the stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, SGLD typically suffers from a slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian smoothing technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieves strictly smaller discretization error in 2-Wasserstein distance, although its mixing rate can be slightly slower. Experiments on both synthetic and real datasets verify our theoretical results and demonstrate the superior performance of LS-SGLD on different machine learning tasks including posterior sampling, Bayesian logistic regression, and training Bayesian convolutional neural networks. The code is available at https://github.com/BaoWangMath/LS-MCMC.

中文翻译：

---

拉普拉斯平滑随机梯度马尔可夫链蒙特卡罗

SIAM科学计算杂志，第43卷，第1期，第A26-A53页，2021年1月。
作为一种重要的马尔可夫链蒙特卡罗（MCMC）方法，随机梯度Langevin动力学（SGLD）算法在贝叶斯学习和后验采样中取得了巨大的成功。但是，由于随机梯度导致SGLD差异较大，因此SGLD通常会出现收敛速度较慢的情况。为了减轻这些缺点，我们利用了最近开发的拉普拉斯平滑技术，并提出了一种拉普拉斯平滑随机梯度朗文动力学（LS-SGLD）算法。我们证明，对于从对数凹面和非对数凹面密度进行采样，尽管LS-SGLD的混合速率可能稍慢，但它们在2-Wasserstein距离中实现了严格较小的离散化误差。在合成数据集和真实数据集上进行的实验验证了我们的理论结果，并证明了LS-SGLD在不同的机器学习任务（包括后采样，贝叶斯逻辑回归和训练贝叶斯卷积神经网络）上的优越性能。该代码位于https://github.com/BaoWangMath/LS-MCMC。