Abstract

In the past decade, many Bayesian shrinkage models have been developed for linear regression problems where the number of covariates, p, is large. Computation of the intractable posterior is often done with three-block Gibbs samplers (3BG), based on representing the shrinkage priors as scale mixtures of Normal distributions. An alternative computing tool is a state of the art Hamiltonian Monte Carlo (HMC) method, which can be easily implemented in the Stan software. However, we found both existing methods to be inefficient and often impractical for large p problems. Following the general idea of Rajaratnam et al., we propose two-block Gibbs samplers (2BG) for three commonly used shrinkage models, namely, the Bayesian group lasso, the Bayesian sparse group lasso, and the Bayesian fused lasso models. We demonstrate with simulated and real data examples that the Markov chains underlying 2BG’s converge much faster than that of 3BG’s, and no worse than that of HMC. At the same time, the computing costs of 2BG’s per iteration are as low as that of 3BG’s, and can be several orders of magnitude lower than that of HMC. As a result, the newly proposed 2BG is the only practical computing solution to do Bayesian shrinkage analysis for datasets with large p. Further, we provide theoretical justifications for the superior performance of 2BG’s. We establish geometric ergodicity of Markov chains associated with the 2BG for each of the three Bayesian shrinkage models. We also prove, for most cases of the Bayesian group lasso and the Bayesian sparse group lasso model, the Markov operators for the 2BG chains are trace-class. Whereas for all cases of all three Bayesian shrinkage models, the Markov operator for the 3BG chains is not even Hilbert–Schmidt. Supplementary materials for this article are available online.

摘要

在过去的十年中，许多贝叶斯收缩模型被开发用于线性回归问题，其中协变量p的数量很大。基于将收缩先验表示为正态分布的尺度混合，难以计算的后验通常使用三块吉布斯采样器 (3BG) 完成。另一种计算工具是最先进的哈密顿蒙特卡罗 (HMC) 方法，它可以在 Stan 软件中轻松实现。然而，我们发现现有的两种方法都效率低下，而且对于大p往往不切实际问题。遵循 Rajaratnam 等人的总体思路，我们为三种常用的收缩模型提出了两块 Gibbs 采样器（2BG），即贝叶斯组套索、贝叶斯稀疏组套索和贝叶斯融合套索模型。我们通过模拟和真实数据示例证明了 2BG 下的马尔可夫链的收敛速度比 3BG 快得多，并且不比 HMC 差。同时，2BG每次迭代的计算成本和3BG一样低，可以比HMC低几个数量级。因此，新提出的 2BG 是对大p数据集进行贝叶斯收缩分析的唯一实用计算解决方案. 此外，我们为 2BG 的卓越性能提供了理论依据。我们为三个贝叶斯收缩模型中的每一个建立了与 2BG 相关的马尔可夫链的几何遍历性。我们还证明，对于贝叶斯群套索模型和贝叶斯稀疏群套索模型的大多数情况，2BG 链的马尔可夫算子是迹类的。而对于所有三个贝叶斯收缩模型的所有情况，3BG 链的马尔可夫算子甚至不是 Hilbert-Schmidt。本文的补充材料可在线获取。