当前位置: X-MOL 学术J. Comput. Graph. Stat. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Manifold Optimization-Assisted Gaussian Variational Approximation
Journal of Computational and Graphical Statistics ( IF 2.4 ) Pub Date : 2021-06-21 , DOI: 10.1080/10618600.2021.1923516
Bingxin Zhou 1 , Junbin Gao 1 , Minh-Ngoc Tran 1 , Richard Gerlach 1
Affiliation  

Abstract

Gaussian variational approximation is a popular methodology to approximate posterior distributions in Bayesian inference, especially in high-dimensional and large data settings. To control the computational cost, while being able to capture the correlations among the variables, the low rank plus diagonal structure was introduced in the previous literature for the Gaussian covariance matrix. For a specific Bayesian learning task, the uniqueness of the solution is usually ensured by imposing stringent constraints on the parameterized covariance matrix, which could break down during the optimization process. In this article, we consider two special covariance structures by applying the Stiefel manifold and Grassmann manifold constraints, to address the optimization difficulty in such factorization architectures. To speed up the updating process with minimum hyperparameter-tuning efforts, we design two new schemes of Riemannian stochastic gradient descent methods and compare them with other existing methods of optimizing on manifolds. In addition to fixing the identification issue, results from both simulation and empirical experiments prove the ability of the proposed methods of obtaining competitive accuracy and comparable converge speed in both high-dimensional and large-scale learning tasks. Supplementary materials for this article are available online.



中文翻译:

流形优化辅助高斯变分逼近

摘要

高斯变分逼近是一种流行的方法来逼近贝叶斯推理中的后验分布,尤其是在高维和大数据设置中。为了控制计算成本,同时能够捕捉变量之间的相关性,在之前的高斯协方差矩阵的文献中引入了低秩加对角结构。对于特定的贝叶斯学习任务,解决方案的唯一性通常是通过对参数化协方差矩阵施加严格的约束来确保的,这可能会在优化过程中崩溃。在本文中,我们通过应用 Stiefel 流形和 Grassmann 流形约束来考虑两种特殊的协方差结构,以解决此类分解架构中的优化困难。为了以最小的超参数调整工作加快更新过程,我们设计了两种黎曼随机梯度下降方法的新方案,并将它们与其他现有的流形优化方法进行比较。除了解决识别问题之外,模拟和经验实验的结果证明了所提出的方法在高维和大规模学习任务中获得竞争准确性和可比收敛速度的能力。本文的补充材料可在线获取。模拟和实证实验的结果证明了所提出的方法在高维和大规模学习任务中获得具有竞争力的准确性和可比的收敛速度的能力。本文的补充材料可在线获取。模拟和实证实验的结果证明了所提出的方法在高维和大规模学习任务中获得具有竞争力的准确性和可比的收敛速度的能力。本文的补充材料可在线获取。

更新日期:2021-06-21
down
wechat
bug