Abstract

This article develops a novel stochastic tree ensemble method for nonlinear regression, referred to as accelerated Bayesian additive regression trees, or XBART. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursive partitioning algorithms, XBART attains state-of-the-art performance at prediction and function estimation. Simulation studies demonstrate that XBART provides accurate point-wise estimates of the mean function and does so faster than popular alternatives, such as BART, XGBoost, and neural networks (using Keras) on a variety of test functions. Additionally, it is demonstrated that using XBART to initialize the standard BART MCMC algorithm considerably improves credible interval coverage and reduces total run-time. Finally, two basic theoretical results are established: the single tree version of the model is asymptotically consistent and the Markov chain produced by the ensemble version of the algorithm has a unique stationary distribution.

摘要

本文开发了一种用于非线性回归的新型随机树集成方法，称为加速贝叶斯加性回归树或 XBART。通过将贝叶斯建模的正则化和随机搜索策略与递归分区算法的计算高效技术相结合，XBART 在预测和函数估计方面获得了最先进的性能。模拟研究表明，XBART 提供均值函数的准确逐点估计，并且在各种测试函数上比流行的替代方法（例如 BART、XGBoost 和神经网络（使用 Keras））更快。此外，证明使用 XBART 初始化标准 BART MCMC 算法可显着提高可信区间覆盖率并减少总运行时间。最后，