Abstract

We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse truth at the optimal rate and gives optimal prediction of the response vector. The empirical performance of our algorithm is studied, showing that it works comparably well as other state-of-the-art Bayesian variable selection methods. We also numerically demonstrate that the widely used coordinate-ascent variational inference algorithm can be highly sensitive to the parameter updating order, leading to potentially poor performance. To mitigate this, we propose a novel prioritized updating scheme that uses a data-driven updating order and performs better in simulations. The variational algorithm is implemented in the R package sparsevb. Supplementary materials for this article are available online.

摘要

我们研究了稀疏高维线性回归中贝叶斯模型选择先验的平均场尖峰和平板变分贝叶斯 (VB) 近似。在设计矩阵的相容条件下，平均场 VB 近似得到了预言不等式，这意味着它以最优速率收敛到稀疏真值，并给出了响应向量的最优预测。研究了我们算法的经验性能，表明它与其他最先进的贝叶斯变量选择方法相当有效。我们还用数值证明了广泛使用的坐标上升变分推理算法可能对参数更新顺序高度敏感，从而可能导致性能不佳。为了减轻这种情况，我们提出了一种新颖的优先更新方案，它使用数据驱动的更新顺序并在模拟中表现更好。变分算法在 R 包 sparsevb 中实现。本文的补充材料可在线获取。