This work considers variational Bayesian inference as an inexpensive and scalable alternative to a fully Bayesian approach in the context of sparsity-promoting priors. In particular, the priors considered arise from scale mixtures of Normal distributions with a generalized inverse Gaussian mixing distribution. This includes the variational Bayesian LASSO as an inexpensive and scalable alternative to the Bayesian LASSO introduced in [56]. It also includes priors which more strongly promote sparsity. For linear models the method requires only the iterative solution of deterministic least squares problems. Furthermore, for $n\rightarrow \infty$ data points and p unknown covariates the method can be implemented exactly online with a cost of O(p$^3$) in computation and O(p$^2$) in memory. For large p an approximation is able to achieve promising results for a cost of O(p) in both computation and memory. Strategies for hyper-parameter tuning are also considered. The method is implemented for real and simulated data. It is shown that the performance in terms of variable selection and uncertainty quantification of the variational Bayesian LASSO can be comparable to the Bayesian LASSO for problems which are tractable with that method, and for a fraction of the cost. The present method comfortably handles n = p = 131,073 on a laptop in minutes, and n = 10$^5$, p = 10$^6$ overnight.

中文翻译：

---

稀疏在线变分贝叶斯回归

在稀疏性先验的背景下，这项工作认为变分贝叶斯推理是完全贝叶斯方法的一种廉价且可扩展的替代方法。特别地，所考虑的先验来自具有正态分布的比例混合以及广义逆高斯混合分布。这包括变体贝叶斯LASSO，它是[56]中引入的贝叶斯LASSO的廉价且可扩展的替代方案。它还包括更能促进稀疏性的先验。对于线性模型，该方法仅需要确定性最小二乘问题的迭代解。此外，对于$ n \ rightarrow \ infty $个数据点和p个未知协变量，该方法可以完全在线实现，计算成本为O（p $ ^ 3 $），存储器成本为O（p $ ^ 2 $）。对于大的p，在计算和存储方面都可以以O（p）的代价获得近似结果。还考虑了超参数调整的策略。该方法适用于真实和模拟数据。结果表明，就变量选择和不确定性量化而言，贝叶斯LASSO的性能可与贝叶斯LASSO相媲美，这是该方法可解决的问题，而且成本低廉。本方法在几分钟内舒适地在笔记本电脑上处理n = p = 131,073，n = 10 $ ^ 5 $，p = 10 $ ^ 6 $过夜。结果表明，就变量选择和不确定性量化而言，贝叶斯LASSO的性能可与贝叶斯LASSO相媲美，这是该方法可解决的问题，而且成本低廉。本方法在几分钟内舒适地在笔记本电脑上处理n = p = 131,073，n = 10 $ ^ 5 $，p = 10 $ ^ 6 $过夜。结果表明，就变量选择和不确定性量化而言，贝叶斯LASSO的性能可与贝叶斯LASSO相媲美，这是该方法可解决的问题，而且成本低廉。本方法在几分钟内舒适地在笔记本电脑上处理n = p = 131,073，n = 10 $ ^ 5 $，p = 10 $ ^ 6 $过夜。