This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal features, we analytically demonstrate favorable performance with regard to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.

这项工作考虑了在贝叶斯回归中施加稀疏性的方法及其在非线性系统识别中的应用。我们首先回顾一下自动相关性确定（ARD），并分析性地证明需要进行额外的正则化或阈值化以实现稀疏模型。然后，我们讨论基于ARD的两类方法，它们基于ARD来学习线性问题的简约解。在正交特征的情况下，我们分析性地证明了在稀疏解的线性系统中学习少量有效项的良好性能。提出了几个示例问题，以比较在具有数百个元素的基础上针对ARD的优点和局限性提出的方法集。