Skip to main content
SpringerLink
Log in

Markov chain Monte Carlo algorithms with sequential proposals

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Availability of data and materials

The German credit dataset used in Sect. 4.3.2 is available from the UCI repository https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data).

References

  • Andrieu, C., Atchade, Y.: On the efficiency of adaptive MCMC algorithms. Electron. Commun. Probab. 12, 336–349 (2007)

    MathSciNet  MATH  Google Scholar 

  • Andrieu, C., Livingstone, S.: Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario. (2019). arXiv preprint arXiv:1906.06197

  • Andrieu, C., Moulines, É.: On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16, 1462–1505 (2006)

    MathSciNet  MATH  Google Scholar 

  • Andrieu, C., Thoms, J.: A tutorial on adaptive MCMC. Stat. Comput. 18, 343–373 (2008)

    MathSciNet  Google Scholar 

  • Atchadé, Y., Fort, G.: Limit theorems for some adaptive MCMC algorithms with subgeometric kernels. Bernoulli 16, 116–154 (2010)

    MathSciNet  MATH  Google Scholar 

  • Atchadé, Y.F., Fort, G.: Limit theorems for some adaptive MCMC algorithms with subgeometric kernels: Part II. Bernoulli 18, 975–1001 (2012)

    MathSciNet  MATH  Google Scholar 

  • Atchadé, Y.F., Rosenthal, J.S.: On adaptive Markov chain Monte Carlo algorithms. Bernoulli 11, 815–828 (2005)

    MathSciNet  MATH  Google Scholar 

  • Beskos, A., Pillai, N., Roberts, G., Sanz-Serna, J.-M., Stuart, A.: Optimal tuning of the hybrid Monte Carlo algorithm. Bernoulli 19, 1501–1534 (2013)

    MathSciNet  MATH  Google Scholar 

  • Betancourt, M.: A conceptual introduction to Hamiltonian Monte Carlo. (2017). arXiv preprint arXiv:1701.02434

  • Bouchard-Côté, A., Vollmer, S.J., Doucet, A.: The bouncy particle sampler: a nonreversible rejection-free Markov chain Monte Carlo method. J. Am. Stat. Assoc. 113, 855–867 (2018)

    MathSciNet  MATH  Google Scholar 

  • Calderhead, B.: A general construction for parallelizing Metropolis-Hastings algorithms. Proc. Natl. Acad. Sci. 111, 17408–17413 (2014)

    Google Scholar 

  • Campos, C.M., Sanz-Serna, J.: Extra chance generalized hybrid Monte Carlo. J. Comput. Phys. 281, 365–374 (2015)

    MathSciNet  MATH  Google Scholar 

  • Carpenter, B., Gelman, A., Hoffman, M.D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., Riddell, A.: Stan: a probabilistic programming language. J. Stat. Softw. 76, 1 (2017)

    Google Scholar 

  • Dua, D., Graff, C.: UCI machine learning repository. (2017). http://archive.ics.uci.edu/ml. Accessed 17 Jan 2020

  • Duane, S., Kennedy, A.D., Pendleton, B.J., Roweth, D.: Hybrid Monte Carlo. Phys. Lett. B 195, 216–222 (1987)

    MathSciNet  Google Scholar 

  • Fang, Y., Sanz-Serna, J.M., Skeel, R.D.: Compressible generalized hybrid Monte Carlo. J. Chem. Phys. 140, 174108 (2014)

    Google Scholar 

  • Flegal, J.M., Hughes, J., Vats, D., Dai, N.: mcmcse: Monte Carlo Standard Errors for MCMC. Riverside, CA, Denver, CO, Coventry, UK, and Minneapolis, MN. R package version 1.3-2 (2017)

  • Geyer, C.J.: Markov chain Monte Carlo maximum likelihood. Retrieved from the University of Minnesota Digital Conservancy, Interface Foundation of North America (1991)

  • Goodman, J., Weare, J.: Ensemble samplers with affine invariance. Commun. Appl. Math. Comput. Sci. 5, 65–80 (2010)

    MathSciNet  MATH  Google Scholar 

  • Green, P.J., Mira, A.: Delayed rejection in reversible jump Metropolis-Hastings. Biometrika 88, 1035–1053 (2001)

    MathSciNet  MATH  Google Scholar 

  • Gupta, S., Irbäc, A., Karsch, F., Petersson, B.: The acceptance probability in the hybrid Monte Carlo method. Phys. Lett. B 242, 437–443 (1990)

    Google Scholar 

  • Haario, H., Saksman, E., Tamminen, J.: An adaptive Metropolis algorithm. Bernoulli 7, 223–242 (2001)

    MathSciNet  MATH  Google Scholar 

  • Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)

    MathSciNet  MATH  Google Scholar 

  • Hoffman, M.D., Gelman, A.: The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15, 1593–1623 (2014)

    MathSciNet  MATH  Google Scholar 

  • Horowitz, A.M.: A generalized guided Monte Carlo algorithm. Phys. Lett. B 268, 247–252 (1991)

    Google Scholar 

  • Hukushima, K., Nemoto, K.: Exchange Monte Carlo method and application to spin glass simulations. J. Phys. Soc. Jpn. 65, 1604–1608 (1996)

    Google Scholar 

  • Kou, S., Zhou, Q., Wong, W.H., et al.: Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Stat. 34, 1581–1619 (2006)

    MathSciNet  MATH  Google Scholar 

  • Leimkuhler, B., Reich, S.: Simulating Hamiltonian dynamics, vol. 14. Cambridge University Press, Cambridge (2004)

    MATH  Google Scholar 

  • Liouville, J.: Note on the theory of the variation of arbitrary constants. Journal de Mathématiques Pures et Appliquées 3, 342–349 (1838)

    Google Scholar 

  • Liu, J.S., Liang, F., Wong, W.H.: The multiple-try method and local optimization in Metropolis sampling. J. Am. Stat. Assoc. 95, 121–134 (2000)

    MathSciNet  MATH  Google Scholar 

  • Marinari, E., Parisi, G.: Simulated tempering: a new Monte Carlo scheme. EPL (Europhys. Lett.) 19, 451 (1992)

    Google Scholar 

  • Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    MATH  Google Scholar 

  • Mira, A.: On metropolis-hastings algorithms with delayed rejection. Metron 59, 231–241 (2001)

    MathSciNet  MATH  Google Scholar 

  • Mira, A., Møller, J., Roberts, G.O.: Perfect slice samplers. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 63, 593–606 (2001)

  • Neal, R.M.: An improved acceptance procedure for the hybrid Monte Carlo algorithm. J. Comput. Phys. 111, 194–203 (1994)

    MathSciNet  MATH  Google Scholar 

  • Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003)

    MathSciNet  MATH  Google Scholar 

  • Neal, R.M.: MCMC using Hamiltonian dynamics. In: Brooks, S., Gelman, A., Jones, G., Meng, X.-L. (eds.) Handbook of Markov chain Monte Carlo, pp. 113–162. CRC press (2011)

  • Peskun, P.H.: Optimum Monte-Carlo sampling using markov chains. Biometrika 60, 607–612 (1973)

    MathSciNet  MATH  Google Scholar 

  • Peters, E.A., et al.: Rejection-free Monte Carlo sampling for general potentials. Phys. Rev. E 85, 026703 (2012)

    Google Scholar 

  • Plummer, M., Best, N., Cowles, K., Vines, K.: CODA: convergence diagnosis and output analysis for MCMC. R News 6, 7–11 (2006)

    Google Scholar 

  • R Core Team.: R: a Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2018). https://www.R-project.org/

  • Roberts, G.O., Rosenthal, J.S.: Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 60, 255–268 (1998)

    MathSciNet  MATH  Google Scholar 

  • Roberts, G.O., Rosenthal, J.S.: Convergence of slice sampler Markov chains. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 61, 643–660 (1999)

    MathSciNet  MATH  Google Scholar 

  • Roberts, G.O., Rosenthal, J.S.: Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. J. Appl. Probab. 44, 458–475 (2007)

    MathSciNet  MATH  Google Scholar 

  • Roberts, G.O., Gelman, A., Gilks, W.R.: Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7, 110–120 (1997)

    MathSciNet  MATH  Google Scholar 

  • Sexton, J., Weingarten, D.: Hamiltonian evolution for the hybrid Monte Carlo algorithm. Nucl. Phys. B 380, 665–677 (1992)

    Google Scholar 

  • Sherlock, C., Fearnhead, P., Roberts, G.O.: The random walk Metropolis: Linking theory and practice through a case study. Stat. Sci. 25, 172–190 (2010)

    MathSciNet  MATH  Google Scholar 

  • Sohl-Dickstein, J., Mudigonda, M., DeWeese, M.R.: Hamiltonian Monte Carlo without detailed balance. (2014). arXiv preprint arXiv:1409.5191

  • Tierney, L.: A note on Metropolis-Hastings kernels for general state spaces. Ann. Appl. Probab. 8, 1–9 (1998)

    MathSciNet  MATH  Google Scholar 

  • Tierney, L., Mira, A.: Some adaptive Monte Carlo methods for Bayesian inference. Stat. Med. 18, 2507–2515 (1999)

    Google Scholar 

  • Vanetti, P., Bouchard-Côté, A., Deligiannidis, G., Doucet, A.: Piecewise deterministic Markov chain Monte Carlo. (2017) arXiv preprint arXiv:1707.05296

Download references

Acknowledgements

This work was supported by National Science Foundation Grants DMS-1513040 and DMS-1308918. The authors thank Edward Ionides, Aaron King, and Stilian Stoev for comments on an earlier draft of this manuscript. The authors also thank Jesús María Sanz-Serna for informing us about related references.

Author information

Authors and Affiliations

  1. Boston University, Boston, USA

    Joonha Park & Yves Atchadé

Authors
  1. Joonha Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yves Atchadé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joonha Park.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Code availability

The source codes used in this work are available at https://github.com/joonhap/spMCMC.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 372 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, J., Atchadé, Y. Markov chain Monte Carlo algorithms with sequential proposals. Stat Comput 30, 1325–1345 (2020). https://doi.org/10.1007/s11222-020-09948-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-020-09948-4

Keywords

Search

Navigation