Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

Nima Nonejad

doi:10.1515/jtse-2013-0024

Published by De Gruyter March 17, 2015

Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

Nima Nonejad

From the journal Journal of Time Series Econometrics

https://doi.org/10.1515/jtse-2013-0024

Showing a limited preview of this publication:

Abstract

This paper details particle Markov chain Monte Carlo (PMCMC) techniques for analysis of unobserved component time series models using several economic data sets. The objective of this paper is to explain the basics of the methodology and provide computational applications that justify applying PMCMC in practice. For instance, we use PMCMC to estimate a stochastic volatility model with a leverage effect, Student-t distributed errors or serial dependence. We also model time series characteristics of monthly US inflation rate by considering a heteroskedastic ARFIMA model where heteroskedasticity is specified by means of a Gaussian stochastic volatility process.

Keywords: Bayes; Gibbs; Metropolis–Hastings; particle filter; unobserved components

JEL Classification: C11; C22; C63

Funding statement: Funding: Danish National Research Foundation (Grant/Award Number: DNRF78).

Appendix

A.1 Simulation Results, SV-MA(1) and SVM

In this section, we present simulation results for SV-MA(1) and SVM. We simulate T=1000 observations from these models, and report the true DGP parameters along with PMMH parameter estimates in Table 10. In each case, we also estimate a plain SV model for comparison. Overall, we see that PMMH works very well as parameter estimates are close to their respective true values. In each case the corresponding model outperforms the plain SV model in terms of ML.

Finally, we analyze the performance of PMMH with respect to the number of particles, M. We do this by estimating SV-MA(1) using M=1, M=10, M=100 and M=1000. In all of these cases, we choose N=20000. We see that using very low values of M appear to be insufficient, see Figure 6. For instance, the chain gets stuck on a specific parameter value almost throughout the sample for M=1. For M=100, we get better results. However, the chain still gets stuck for a considerable time. However, we see drastic improvements in the performance of the algorithm for M=1000. We also run the PMMH algorithm for M=2000. We get almost identical results as for M=1000.

Table 10:

Simulation evidence.

Parameter	DGP: stochastic volatility with MA(1) errors, SV-MA(1)					DGP: stochastic volatility in mean, SVM
		SV		SV-MA(1)			SV		SVM
	true	θˉ	RB	θˉ	RB	true	θˉ	RB	θˉ	RB
μ	0.20	0.4168	4.31	0.2756	4.40	0.20	0.7287	23.72	0.2883	5.24
		[0.2129,0.6045]		[0.0419,0.5011]			[0.4106,1.0132]		[0.0580,0.5191]
ϕ	0.98	0.9701	5.07	0.9781	4.55	0.98	0.9815	21.63	0.9790	4.86
		[0.9515,0.9861]		[0.9635,0.9906]			[0.9651,0.9928]		[0.9655,0.9904]
σ2	0.01	0.0171	4.86	0.0104	5.22	0.01	0.0087		0.0108	5.38
		[0.0078,0.0291]		[0.0048,0.0172]			[0.0043,0.0159]	14.04	[0.0055,0.0174]
ψ	0.40			0.4591	5.57
				[0.4221,0.4963]
β						0.80			0.7533	5.09
									[0.7090,0.8001]

log(L)		–1,666.1		–1,581.0			–1,811.0		–1,586.8
log(ML), a=0.75		–1,671.2		–1,592.4			–1,817.5		–1,598.5
log(ML), a=0.99		–1,673.8		–1,592.2			–1,817.2		–1,598.2
M–H ratio		0.42		0.38			0.33		0.37

^[1]

Figure 6:

Estimation results, SV-MA(1) model. Each column shows posterior draws of the parameter of interest for different number of particles, M.

A.2 Particle Gibbs with Ancestor Sampling

In order to ease the notation burden, we consider the plain stochastic volatility (SV) model

[10]yt=expαt/2ϵt,ϵt∼N0,1

[11]αt+1=μ+ϕαt−μ+σηt,ηt∼N0,1,

where θ=μ,ϕ,σ2′. Within the PG-AS framework, we approach estimating eqs [10] and [11] directly by first drawing α1,...,αT∼pα1,...,αT|θ,YT using the conditional particle filter with ancestor sampling, CPF-AS. Thereafter, we draw pθ|α1,...,αT,YT using standard Gibbs sampling techniques. Let i=1,...,N denote the number of Gibbs sampling iterations, j=1,...,M denote the number of particles, and let pyt|θ,αt,Yt−1 denote the density of yt given θ, αt and Yt−1. Finally, let α1i−1,...,αTi−1 be a fixed reference trajectory of α1,...,αT sampled at iteration i−1 of the Gibbs sampler. The steps of CPF-AS for the SV model are as follows

ift=1
1. Draw α1j|α0j,θ for j=1,...,M−1 and set α1M=α1i−1.
2. Set w1j=τ1j/Σk=1Mτ1k, where τ1j=py1|θ,α1j,Y0 for j=1,...,M.
else fort=2to T do
1. Resample {αt−1j}j=1M−1 using indices δtj, where pδtj=k∝wt−1k.
2. Draw αtj|αt−1δtj,θ for j=1,...,M−1.
3. Set αt(M)=αti−1.
4. Draw δtM from pδtM=j∝wt−1jpαti−1|αt−1j,θ.
5. Set α1j,...,αtj=α1δtj,...,αt−1δtj,αtj, wtj=τtj/Σk=1Mτtk.
end for
Sample α1i,...,αTi|θ,YT with pα1i,...,αTi=α1j,...,αTj|θ,YT∝wTj.

Notice that CPF-AS is akin to a standard particle filter, but with the difference that α1(M),...,αT(M) is specified a priori and serves as a reference trajectory. Hence, we use only M−1 particles at each step. Furthermore, whereas in the particle Gibbs algorithm of Andrieu, Doucet, and Holenstein (2010), we set δtM=M, in PG-AS, we sample a new value for the index variable, δtM, in an ancestor sampling step, (d). Finally, we can include more unobserved processes in the model of interest. All we need to do is to modify (a), (b), (c), (e), and thus draw particles for each process. At the same time, we still have one sets of weights. Thereafter, we can sample the unobserved process using step 4.

References

Andrieu, C., A.Doucet, and R.Holenstein. 2010. “Particle Markov Chain Monte Carlo Methods (with Discussion).” Journal of the Royal Statistical Society B72 (3): 1–33.Search in Google Scholar

Bos, C. S. 2011. “A Bayesian Analysis of Unobserved Component Models Using Ox.” Journal of Statistical Software41 (13): 1–24.10.2139/ssrn.1776046Search in Google Scholar

Bos, C. S. 2013GnuDraw.http://www.tinbergen.nl/~cbos/gnudraw.htmlSearch in Google Scholar

Bos, C. S., S. J.Koopman, and M.Ooms. 2012. “Long Memory with Stochastic Variance Model: A Recursive Analysis for U.S. Inflation.” Computational Statistics and Data Analysis76 (3): 144–57.Search in Google Scholar

Chan, J. 2013. “Moving Average Stochastic Volatility Models with Application to Inflation Forecast.” Journal of Econometrics176 (2): 162–72.10.1016/j.jeconom.2013.05.003Search in Google Scholar

Chan, J. 2014. “The Stochastic Volatility in Mean Model with Time-Varying Parameters: An Application to Inflation Modeling.” Working paper, Research School of Economics, Australian National University.10.2139/ssrn.2579988Search in Google Scholar

Chib, S., and E.Greenberg. 1995. “Understanding the Metropolis-Hastings Algorithm.” The American Statistician49 (4): 327–35.Search in Google Scholar

Chib, S., F.Nadari, and N.Shephard. 2002. “Markov Chain Monte Carlo Methods for Stochastic Volatility Models.” Journal of Econometrics108 (2): 281–316.10.1016/S0304-4076(01)00137-3Search in Google Scholar

Creal, D. 2012. “A Survey of Sequential Monte Carlo Methods for Economics and Finance.” Econometric Reviews31 (3): 245–96.10.1080/07474938.2011.607333Search in Google Scholar

Del Moral, P. 2004. Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. New York: Springer-Verlag.10.1007/978-1-4684-9393-1Search in Google Scholar

Doornik, J. A. 2009. An Object-Oriented Matrix Language Ox 6. London: Timberlake.Search in Google Scholar

Doucet, A., S. J.Godsill, and C.Andrieu. 2000. “On Sequential Monte Carlo Sampling Methods for Bayesian Filtering.” Statistics and Computing10 (3): 197–208.10.1023/A:1008935410038Search in Google Scholar

Flury, T., and N.Shephard. 2011. “Bayesian Inference Based Only on Simulated Likelihood: Particle Filter Analysis of Dynamic Economic Models.” Econometric Theory27 (5): 933–56.10.1017/S0266466610000599Search in Google Scholar

Fouque, J. P., C. H.Han, and M.Molina. 2010. “MCMC Estimation of Multiscale Stochastic Volatility Models.” Handbook of Quantitative Finance and Risk Management 1109–20. New York: Springer.10.1007/978-0-387-77117-5_71Search in Google Scholar

Gelfand, A., and D.Dey. 1994. “Bayesian Model Choice: Asymptotics and Exact Calculations.” Journal of the Royal Statistical Society B56 (3): 501–14.10.1111/j.2517-6161.1994.tb01996.xSearch in Google Scholar

Geweke, J. 2005. Contemporary Bayesian Econometrics and Statistics. New Jersey: John Wiley & Sons Ltd.10.1002/0471744735Search in Google Scholar

Grassi, S., and T.Proietti. 2010. “Has the Volatility of U.S. Inflation Changed and How?” Journal of Time Series Econometrics2 (1): Article 6.10.2202/1941-1928.1050Search in Google Scholar

Hansen, P. R., A.Lunde, and J. M.Nason. 2011. “The Model Confidence Set.” Econometrica79 (2): 453–97.10.3982/ECTA5771Search in Google Scholar

Kass, R. E., and A. E.Raftery. 1995. “Bayes Factors.” Journal of the American Statistical Association90: 773–95.10.1080/01621459.1995.10476572Search in Google Scholar

Kim, C. J., and C. R.Nelson. 1999. State Space Models with Regime Switching Classical and Gibbs Sampling Approaches with Applications. Cambridge, MA: MIT Press.Search in Google Scholar

Kim, S., N.Shephard, and S.Chib. 1998. “Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models.” Review of Economic Studies65 (3): 361–93.10.1111/1467-937X.00050Search in Google Scholar

Koop, G. 2003. Bayesian Econometrics. England: John Wiley & Sons Ltd.Search in Google Scholar

Koop, G., R.Leon-Gonzalez, and R. W.Strachan. 2010. “Dynamic probabilities of restrictions in state space models: an application to the Phillips curve.” Journal of Business and Economic Statistics28 (3): 370–79.10.1198/jbes.2009.07335Search in Google Scholar

Koopman, S. J., and E. H.Uspensky. 2002. “The Stochastic Volatility in Mean Model: Empirical Evidence from International Stock Markets.” Journal of Applied Econometrics17 (6): 667–89.10.1002/jae.652Search in Google Scholar

Lindsten, F., M. I.Jordan, and T. B.Schön. 2012. “Ancestor Sampling for Particle Gibbs.” Advances in Neural Information Processing Systems (NIPS25: 2600–8.Search in Google Scholar

Lindsten, F., and T. B.Schön. 2013. “Backward Simulation Methods for Monte Carlo Statistical Inference.” Foundations and Trends in Machine Learning6 (1): 1–14.10.1561/2200000045Search in Google Scholar

Robert, C., and G.Casella. 1999. Monte Carlo Statistical Methods. Berlin: Springer.10.1007/978-1-4757-3071-5Search in Google Scholar

So, M. K. P., C. W. S.Chen, and M.Chen. 2005. “A Bayesian Threshold Nonlinearity Test for Financial Time Series.” Journal of Forecasting24 (1): 61–75.10.1002/for.939Search in Google Scholar

Stock, J. H., and M. W.Watson. 2007. “Why Has U.S. Inflation Become Harder to Forecast?” Journal of Money, Credit, and Banking39 (1): 3–34.10.1111/j.1538-4616.2007.00014.xSearch in Google Scholar

Published Online: 2015-3-17

Published in Print: 2016-1-1

Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

Abstract

Appendix

A.1 Simulation Results, SV-MA(1) and SVM

A.2 Particle Gibbs with Ancestor Sampling

References

Journal and Issue

Articles in the same Issue