Abstract
In the Bayesian framework, the marginal likelihood plays an important role in variable selection and model comparison. The marginal likelihood is the marginal density of the data after integrating out the parameters over the parameter space. However, this quantity is often analytically intractable due to the complexity of the model. In this paper, we first examine the properties of the inflated density ratio (IDR) method, which is a Monte Carlo method for computing the marginal likelihood using a single MC or Markov chain Monte Carlo (MCMC) sample. We then develop a variation of the IDR estimator, called the dimension reduced inflated density ratio (Dr.IDR) estimator. We further propose a more general identity and then obtain a general dimension reduced (GDr) estimator. Simulation studies are conducted to examine empirical performance of the IDR estimator as well as the Dr.IDR and GDr estimators. We further demonstrate the usefulness of the GDr estimator for computing the normalizing constants in a case study on the inequality-constrained analysis of variance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arima, S., & Tardella, L. (2014). Idr for marginal likelihood in Bayesian phylogenetics. In M.-H. Chen, L. Kuo, & P. Lewis (Eds.), Bayesian Phylogenetics—Methods, Algorithm, and Applications (pp. 25–57). Boca Raton: CRC Press.
Chen, M.-H. (1994). Importance-weighted marginal Bayesian posterior density estimation. Journal of the American Statistical Association, 89, 818–824.
Chen, M.-H. (2005). Computing marginal likelihoods from a single MCMC output. Statistica Neerlandica, 59, 16–29.
Chen, M.-H., & Kim, S. (2008). The Bayes factor versus other model selection criteria for the selection of constrained models. In H. Hoijtink, I. Klugkist, & P. Boelen (Eds.), Bayesian Evaluation of Informative Hypotheses (pp. 155–180). Berlin: Springer.
Chib, S. (1995). Marginal likelihood from the Gibbs output. Journal of the American Statistical Association, 90, 1313–1321.
Chib, S., & Jeliazkov, I. (2001). Marginal likelihood from the Metropolis-Hastings output. Journal of the American Statistical Association, 96, 270–281.
Fan, Y., Wu, R., Chen, M.-H., Kuo, L., & Lewis, P. O. (2011). Choosing among partition models in Bayesian phylogenetics. Molecular Biology and Evolution, 28, 523–532.
Gelfand, A. E., Smith, A. F., & Lee, T.-M. (1992). Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. Journal of the American Statistical Association, 87, 523–532.
Gelman, A., & Meng, X.-L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statistical Science, 13, 163–185.
Hoijtink, H., Klugkist, I., & Boelen, P. (2008). Bayesian Evaluation of Informative Hypotheses. Berlin: Springer Science & Business Media.
Kahn, H. (1950). Random sampling Monte Carlo techniques in neutron attenuation problems. Nucleonics, 6, 27–37.
Lartillot, N., & Philippe, H. (2006). Computing Bayes factors using thermodynamic integration. Systematic Biology, 55, 195–207.
Newton, M. A., & Raftery, A. E. (1994). Approximate Bayesian inference by the weighted likelihood bootstrap. Journal of the Royal Statistical Society, Series B, 56, 3–48.
Petris, G., & Tardella, L. (2007). New perspectives for estimating normalizing constants via posterior simulation, Technical Report, Universita I di Roma La Sapienza.
Petris, G., & Tardella, L. (2003). A geometric approach to transdimensional Markov chain Monte Carlo. The Canadian Journal of Statistics, 31(4), 469–482.
Schmeiser, B. W., Avramidis, T. N., & Hashem, S. Overlapping batch statistics. In Proceedings of the 22nd Conference on Winter Simulation (pp. 395–398) IEEE Press.
Wang, Y.-B., Chen, M.-H., Kuo, L., & Lewis, P. O. (2018). A new Monte Carlo method for estimating marginal likelihoods. Bayesian Analysis, 13, 311–333.
Wang, Y.-B., Chen, M.-H., Kuo, L., & Lewis, P. O. (2019). Partition weighted approach for estimating the marginal posterior density with applications. Journal of Computational and Graphical Statistics,. https://doi.org/10.1080/10618600.2018.1529600. (In press).
Xie, W., Lewis, P. O., Fan, Y., Kuo, L., & Chen, M.-H. (2011). Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology, 60, 150–160.
Acknowledgements
The authors gratefully thank the Editor in Chief, the Editor, the Associate Editor, and the three anonymous reviewers for their constructive comments and suggestions that help improve the article. This material is based upon work partially supported by the National Science Foundation under Grant No. DEB-1354146. Dr. M.-H. Chen’s research was also partially supported by NIH Grants #GM70335 and #P01CA142538.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, YB., Chen, MH., Shi, W. et al. Inflated density ratio and its variation and generalization for computing marginal likelihoods. J. Korean Stat. Soc. 49, 244–263 (2020). https://doi.org/10.1007/s42952-019-00013-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42952-019-00013-z