Skip to main content
Log in

Bayesian multiple changepoints detection for Markov jump processes

  • Original paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

A Bayesian multiple changepoint model for the Markov jump process is formulated as a Markov double chain model in continuous time. Inference for this type of multiple changepoint model is based on a two-block Gibbs sampling scheme. We suggest a continuous-time version of forward-filtering backward-sampling (FFBS) algorithm for sampling the full trajectories of the latent Markov chain via inverse transformation. We also suggest a continuous-time version of Viterbi algorithm for this Markov double chain model, which is viable for obtaining the MAP estimation of the set of locations of changepoints. The model formulation and the continuous-time version of forward-filtering backward-sampling algorithm and Viterbi algorithm can be extended to simultaneously monitor the structural breaks of multiple Markov jump processes, which may have either variable transition rate matrix or identical transition rate matrix. We then perform a numerical study to demonstrate the methods. In numerical examples, we compare the performance of the FFBS algorithm via inverse transformation with the FFBS algorithm via uniformization. The two algorithms performs well in different cases. These models are potentially viable for modelling the credit rating dynamics in credit risk management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Arnesen P, Holsclaw T, Smyth P (2016) Bayesian detection of change points in finite-state Markov chains for multiple sequences. Technometrics 58(2):205–213

    Article  MathSciNet  Google Scholar 

  • Ball F, Milne RK, Yeo GF (1994) Continuous time Markov chains in a random environment, with application to ion channel modelling. Adv Appl Probab 26(4):919–946

    Article  MathSciNet  Google Scholar 

  • Barry D, Hartigan J (1992) Product partition models for change point problems. Ann Stat 20:260–279

    Article  MathSciNet  Google Scholar 

  • Bebbington MS (2007) Identifying volcanic regimes using hidden Markov models. Geophys J Int 171:921–942

    Article  Google Scholar 

  • Biernacki C, Celeux G, Govaert G (2000) Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell 22(7):719–725

    Article  Google Scholar 

  • Chib S (1998) Estimation and comparison of multiple change-point models. J Econ 86:221–241

    Article  MathSciNet  Google Scholar 

  • Ephraim Y, Roberts WJJ (2009) An EM algorithm for Markov modulated Markov processes. IEEE Trans Signal Process 57(2):463–470

    Article  MathSciNet  Google Scholar 

  • Fearnhead P (2006) Exact and efficient Bayesian inference for multiple changepoint problems. Stat Comput 16:203–213

    Article  MathSciNet  Google Scholar 

  • Fearnhead P, Sherlock C (2006) An exact Gibbs sampler for the Markov-modulated Poisson process. J R Statist Soc B 68(5):767–784

    Article  MathSciNet  Google Scholar 

  • Fitzpatrick M, Marchev D (2013) Efficient Bayesian estimation of the multivariate double chain Markov model. Stat Comput 23(4):467–480

    Article  MathSciNet  Google Scholar 

  • Green PJ (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711–732

    Article  MathSciNet  Google Scholar 

  • Jensen A (1953) Markoff chains as an aid in the study of markoff processes. Scand Actuar J 1953(Supp 1):87–91

    Article  MathSciNet  Google Scholar 

  • Lai TL, Xing H (2011) A simple Bayesian approach to multiple change-points. Statistica Sinica 21:539–569

    Article  MathSciNet  Google Scholar 

  • Lu S (2019) A Bayesian multiple changepoint model for marked poisson processes with applications to deep earthquakes. Stoch Environ Res Risk Assess 33(1):59–72

    Article  Google Scholar 

  • Perkins TJ (2009) Maximum likelihood trajectories for continuous-time Markov chains. Adv Neural Inf Process Syst 22:1437–1445

    Google Scholar 

  • Polansky AM (2007) Detecting change-points in Markov chains. Comput Stat Data Anal 51:6013–6026

    Article  MathSciNet  Google Scholar 

  • Rao V, Teh YW (2013) Fast MCMC sampling for Markov jump processes and extensions. J Mach Learn Res 14:3295–3320

    MathSciNet  MATH  Google Scholar 

  • Rigaill G, Lebarbier E, Robin S (2012) Exact posterior distributions and model selection criteria for multiple change-point detection problems. Stat Comput 22:917–929

    Article  MathSciNet  Google Scholar 

  • Scott SL (2002) Bayesian methods for hidden Markov models: recursive computing in the 21st century. J Am Stat Assoc 97:337–351

    Article  MathSciNet  Google Scholar 

  • Stephens DA (1994) Bayesian retrospective multiple-changepoint identification. Appl Stat 43:159–178

    Article  Google Scholar 

  • Van Dijk NM, Van Brummelen SP, Boucherie RJ (2018) Uniformization: Basics, extensions and applications. Perform Eval 118:8–32

    Article  Google Scholar 

  • Viterbi AJ (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory 13(2):260–269

    Article  Google Scholar 

  • Xing H, Sun N, Chen Y (2012) Credit rating dynamics in the presence of unknown structural breaks. J Bank Finance 36:78–89

    Article  Google Scholar 

  • Zhang NR, Siegmund DO (2007) A modified Bayes information criterion with applications to the analysis of comparative genomic hybridization data. Biometrics 63(1):22–32

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

One Referee’s suggestions are acknowledged.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lu Shaochuan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Note that \(\{X(t), Y^{(1}(t), Y^{(2}(t)\}\) is a Markov Process. The density of the observable process on the space of sample path can be evaluated as follows. Assume that the process jumps N times in [0, T]. The likelihood can then be obtained from the joint density of the random variables \(\{(Y_1,\Delta t_1), (Y_2,\Delta t_2), \ldots , (Y_n,\Delta t_n)\}\). An expression for the density \(P\{x_{k+1}, Y_{k+1}, \Delta t_k\}\) can be derived from a similar argument in (Ball et al. 1994). Specifically, suppose that at time \(t=0\), the latent Markov chain is in state \(X(0)=i\) and the observable process \(Y^{(1}(0)=u, Y^{(2}(0)=l\). \(\Delta t_1=t\) is the time of the first jump of the observed Markov jump process, e.g. \(Y^{(1}(t)\). The conditional probability of the event \(\{x(t)=j, Y^{(1}(t)=v, Y^{(2}(t)=l, \Delta t_1\in [t, t+dt), v\ne u\}\) given \(\{x(0)=i, Y^{(1}(0)=u, Y^{(2}(0)=l\}\) is of interest. The first case is given by the second term on the right of the above equation. The first term on the right-hand of the above equation is for the case when the latent chain has no jump in [0, T]. We denote the corresponding density by \(f_{ij}^{(u,v;l,l)}(t)\).

$$\begin{aligned} f_{ij}^{(u,v;l,l)}(t)= & {} e^{-q_it}\left[ d_u^{(1)}(i)e^{-d_u^{(1)}(i)t}e^{-d_l^{(2)}(i)t}\right] \frac{d_{uv}^{(1)}(i)}{d_u^{(1)}(i)}\delta _{ij}\\&+\,\int _{0}^{t} e^{-(d_u^{(1)}(i)+d_l^{(2)}(i))s}q_ie^{-q_is}\sum \limits _{k\ne i}\frac{q_{ik}}{q_i}f_{kj}^{(u,v;l,l)}(t-s)\,ds,\nonumber \end{aligned}$$
(28)

where \(\delta _{ij}\) is the Kronecker delta function. The above expression may be explained as follows. The first jump of the observable process, e.g. \(Y^{(1}(t)\), from state u to v at time t may occur while the underlying Markov chain first jumps to some intermediate state \(k\ne j\) at some random time \(s < t\), before it eventually the final state j at time t. The jump of the observed process may also occur while the latent Markov chain remains in \(x(0)=i\) for the case \(j=i\). With a variable substitution, the second term of the right of the equality is given by

$$\begin{aligned}&\int _{0}^{t} e^{-(d_u^{(1)}(i)+d_l^{(2)}(i))s}q_ie^{-q_is}\sum \limits _{k\ne i}\frac{q_{ik}}{q_i}f_{kj}^{(u,v;l,l)}(t-s)\,ds\\&\quad = \int _{0}^{t} e^{-(d_u^{(1)}(i)+d_l^{(2)}(i))(t-r)}q_ie^{-q_i(t-r)}\sum \limits _{k\ne i}\frac{q_{ik}}{q_i}f_{kj}^{(u,v;l,l)}(r)\,dr\nonumber \end{aligned}$$
(29)

By differentiating \(f_{ij}^{(u,v;l,l)}(t)\) with respect to t, we obtain

$$\begin{aligned} \frac{df_{ij}^{(u,v;l,l)}(t)}{dt} = -(q_i+d_u^{(1)}(i)+d_l^{(2)}(i))f_{ij}^{(u,v;l,l)}(t) + \sum \limits _{k\ne i}q_{ik}f_{kj}^{(u,v;l,l)}(t). \end{aligned}$$

so,

$$\begin{aligned} \frac{df^{(u,v;l,l)}(t)}{dt}=(Q+D_{uu}^{(1)}+D_{ll}^{(2)})f^{(u,v;l,l)}(t). \end{aligned}$$

With initial value \(f^{(u,v;l,l)}(0)=D_{uv}^{(1)}\), we have

$$\begin{aligned} f^{(u,v;l,l)}(t) = \exp \{(Q+D_{uu}^{(1)}+D_{ll}^{(2)})t\}D_{uv}^{(1)}. \end{aligned}$$

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shaochuan, L. Bayesian multiple changepoints detection for Markov jump processes. Comput Stat 35, 1501–1523 (2020). https://doi.org/10.1007/s00180-020-00956-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-020-00956-6

Keywords

Navigation