Stochastic gradient Langevin dynamics (SGLD) is a computationally efficient sampler for Bayesian posterior inference given a large scale dataset and a complex model. Although SGLD is designed for unbounded random variables, practical models often incorporate variables within a bounded domain, such as non-negative or a finite interval. The use of variable transformation is a typical way to handle such a bounded variable. This paper reveals that several mapping approaches commonly used in the literature produce erroneous samples from theoretical and empirical perspectives. We show that the change of random variable in discretization using an invertible Lipschitz mapping function overcomes the pitfall as well as attains the weak convergence, while the other methods are numerically unstable or cannot be justified theoretically. Experiments demonstrate its efficacy for widely-used models with bounded latent variables, including Bayesian non-negative matrix factorization and binary neural networks.

中文翻译：

---

变换随机梯度 MCMC 的弱逼近

在给定大规模数据集和复杂模型的情况下，随机梯度朗之万动力学 (SGLD) 是一种计算效率高的贝叶斯后验推理采样器。尽管 SGLD 是为无界随机变量设计的，但实际模型通常将变量包含在有界域中，例如非负或有限区间。使用变量变换是处理这种有界变量的典型方法。本文揭示了文献中常用的几种映射方法从理论和经验的角度产生了错误的样本。我们表明，使用可逆 Lipschitz 映射函数在离散化中随机变量的变化克服了陷阱并实现了弱收敛，而其他方法在数值上不稳定或在理论上无法证明。