Abstract
This paper presents a Bayesian sampling approach to bandwidth estimation for the local linear estimator of the regression function in a nonparametric regression model. In the Bayesian sampling approach, the error density is approximated by a location-mixture density of Gaussian densities with means the individual errors and variance a constant parameter. This mixture density has the form of a kernel density estimator of errors and is referred to as the kernel-form error density (c.f. Zhang, X., M. L. King, and H. L. Shang. 2014. “A Sampling Algorithm for Bandwidth Estimation in a Nonparametric Regression Model with a Flexible Error Density.” Computational Statistics & Data Analysis 78: 218–34.). While (Zhang, X., M. L. King, and H. L. Shang. 2014. “A Sampling Algorithm for Bandwidth Estimation in a Nonparametric Regression Model with a Flexible Error Density.” Computational Statistics & Data Analysis 78: 218–34) use the local constant (also known as the Nadaraya-Watson) estimator to estimate the regression function, we extend this to the local linear estimator, which produces more accurate estimation. The proposed investigation is motivated by the lack of data-driven methods for simultaneously choosing bandwidths in the local linear estimator of the regression function and kernel-form error density. Treating bandwidths as parameters, we derive an approximate (pseudo) likelihood and a posterior. A simulation study shows that the proposed bandwidth estimation outperforms the rule-of-thumb and cross-validation methods under the criterion of integrated squared errors. The proposed bandwidth estimation method is validated through a nonparametric regression model involving firm ownership concentration, and a model involving state-price density estimation.
Funding source: Australian Research Council
Award Identifier / Grant number: DP1095838
Award Identifier / Grant number: DP130104229
-
Author contribution: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
-
Research funding: None declared.
-
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.
References
Aït-Sahalia, Y., and A. W. Lo. 1998. “Nonparametric Estimation of State-Price Densities Implicit in Financial Asset Prices.” The Journal of Finance 53: 499–547.10.3386/w5351Search in Google Scholar
Bowman, A. W., and A. Azzalini. 1997. Applied Smoothing Techniques for Data Analysis. London: Oxford University Press.Search in Google Scholar
Cheng, F. 2004. “Weak and Strong Uniform Consistency of a Kernel Error Density Estimator in Nonparametric Regression.” Journal of Statistical Planning and Inference 119: 95–107.10.1016/S0378-3758(02)00417-2Search in Google Scholar
Chernozhukov, V., and H. Hong. 2003. “An MCMC Approach to Classical Estimation.” Journal of Econometrics 115: 293–346.10.1016/S0304-4076(03)00100-3Search in Google Scholar
Chib, S. 1995. “Marginal Likelihood from the Gibbs Output.” Journal of the American Statistical Association: Theory and Methods 90: 1313–21.10.1080/01621459.1995.10476635Search in Google Scholar
Efromovich, S. 2005. “Estimation of the Density of Regression Errors.” The Annals of Statistics 33: 2194–227.10.1214/009053605000000435Search in Google Scholar
Fan, J., and I. Gijbels. 1996. Local Polynomial Modelling and its Applications. Boca Raton: Chapman & Hall.Search in Google Scholar
Faraway, J. J., and M. Jhun. 1990. “Bootstrap Choice of Bandwidth for Density Estimation.” Journal of the American Statistical Association: Theory and Methods 85: 1119–22.10.1080/01621459.1990.10474983Search in Google Scholar
Garthwaite, P. H., Y. Fan, and S. A. Sisson. 2016. “Adaptive Optimal Scaling of Metropolis-Hastings Algorithms Using the Robbins-Monro Process.” Communications in Statistics - Theory and Methods 45: 5098–111.10.1080/03610926.2014.936562Search in Google Scholar
Gelman, A., G. O. Roberts, and W. R. Gilks. 1996. “Efficient Metropolis Jumping Rules.” Bayesian Statistics 5: 599–607.Search in Google Scholar
Geweke, J. F. 1999. “Using Simulation Methods for Bayesian Econometric Models: Inference, Development, and Communication.” Econometric Reviews 18: 1–73.10.1080/07474939908800428Search in Google Scholar
Geweke, J. 2010. Complete and Incomplete Econometric Models. New Jersey: Princeton University Press.10.1515/9781400835249Search in Google Scholar
Gourieroux, C., A. Monfort, and A. Trognon. 1984. “Pseudo Maximum Likelihood Methods: Theory.” Econometrica 52: 681–700.10.2307/1913471Search in Google Scholar
Hall, P., S. N. Lahiri, and J. Polzehl. 1995. “On Bandwidth Choice in Nonparametric Regression with Both Short-And Long-Range Dependent Errors.” The Annals of Statistics 23: 1921–36.10.1214/aos/1034713640Search in Google Scholar
Härdle, W. 1990. Applied Nonparametric Regression. Cambridge: Cambridge University Press.10.1017/CCOL0521382483Search in Google Scholar
Härdle, W. 1991. Smoothing Techniques with Implementation in S. New York: Springer-Verlag.10.1007/978-1-4612-4432-5Search in Google Scholar
Härdle, W., and E. Mammen. 1993. “Comparing Nonparametric Verus Parametric Regression Fits.” The Annals of Statistics 21: 1926–47.10.1214/aos/1176349403Search in Google Scholar
Härdle, W., and J. S. Marron. 1985. “Optimal Bandwidth Selection in Nonparametric Regression Function Estimation.” The Annals of Statistics 13: 1465–81.10.1214/aos/1176349748Search in Google Scholar
Härdle, W., and M. Müller. 2000. “Multivariate and Semiparametric Kernel Regression.” In Smoothing and Regression: Approaches, Computation, and Application, edited by M. G. Schimek, 357–91. New York: John Wiley & Sons.10.1002/9781118150658.ch12Search in Google Scholar
Hayfield, T., and J. S. Racine. 2008. “Nonparametric Econometrics: The np Package.” Journal of Statistical Software 27.10.18637/jss.v027.i05Search in Google Scholar
Herrmann, E., J. Engel, M. P. Wand, and T. Gasser. 1995. “A Bandwidth Selector for Bivariate Kernel Regression.” Journal of the Royal Statistical Society: Series B 57: 171–80.10.1111/j.2517-6161.1995.tb02022.xSearch in Google Scholar
Huynh, K., P. Kervella, and J. Zheng. 2002. “Estimating State Price Densities with Nonparametric Regression.” In Applied Quantitative Finance, edited by W. Härdle, T. Kleinow and G. Stahl, 171–96. Heidelberg: Springer-Verlag.10.1007/978-3-662-05021-7_8Search in Google Scholar
Jaki, T., and R. W. West. 2008. “Maximum Kernel Likelihood Estimation.” Journal of Computational & Graphical Statistics 17: 976–93.10.1198/106186008X387057Search in Google Scholar
Jaki, T., and R. W. West. 2011. “Symmetric Maximum Kernel Likelihood Estimation.” Journal of Statistical Computation and Simulation 81: 193–206.10.1080/00949650903232664Search in Google Scholar
Kass, R. E., and A. E. Raftery. 1995. “Bayes Factors.” Journal of the American Statistical Association: Review Paper 90: 773–95.10.1080/01621459.1995.10476572Search in Google Scholar
Kim, S., N. Shephard, and S. Chib. 1998. “Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models.” The Review of Economic Studies 65: 361–93.10.1111/1467-937X.00050Search in Google Scholar
Kuruwita, C. N., K. B. Kulasekera, and C. M. Gallagher. 2011. “Generalized Varying Coefficient Models with Unknown Link Function.” Biometrika 98: 701–10.10.1093/biomet/asr031Search in Google Scholar
Lee, C., and P. J. Neal. 2018. “Optimal Scaling of the Independence Sampler: Theory and Practice.” Bernoulli 24: 1636–52.10.3150/16-BEJ908Search in Google Scholar
Li, Q., and J. Racine. 2004. “Cross-validated Local Linear Nonparametric Regression.” Statistica Sinica 14: 485–512.Search in Google Scholar
Li, Q., and J. Zhou. 2005. “The Uniqueness of Cross-validation Selected Smoothing Parameters in Kernel Estimation of Nonparametric Models.” Econometric Theory 21: 1017–25.10.1017/S0266466605050504Search in Google Scholar
Linton, O., and Z. Xiao. 2007. “A Nonparametric Regression Estimator that Adapts to Error Distribution of Unknown Form.” Econometric Theory 23: 371–413.10.1017/S026646660707017XSearch in Google Scholar
Martins-Filho, C., F. Yao, and M. Torero. 2018. “Nonparametric Estimation of Conditional Value-at-risk and Expected Shortfall Based on Extreme Value Theory.” Econometric Theory 34: 23–67.10.1017/S0266466616000517Search in Google Scholar
Müller, U. 2013. “Risk of Bayesian Inference in Misspecified Models, and the Sandwich Covariance Matrix.” Econometrica 81: 1805–49.10.3982/ECTA9097Search in Google Scholar
Nadaraya, E. A. 1964. “On Estimating Regression.” Theory of Probability and Its Applications 9: 141–2.10.1137/1109020Search in Google Scholar
Padgett, W. J., and L. A. Thombs. 1986. “Smooth Nonparametric Quantile Estimation under Censoring: Simulations and Bootstrap Methods.” Communications in Statistics – Simulation and Computation 15: 1003–25.10.21236/ADA169935Search in Google Scholar
R Core Team. 2020. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing, https://www.R-project.org/.Search in Google Scholar
Roberts, G. O. 1996. “Markov Chain Concepts Related to Sampling Algorithms.” In Markov Chain Monte Carlo in Practice, edited by W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, 45–57. London: Chapman & Hall.Search in Google Scholar
Roberts, G. O., and J. S. Rosenthal. 2001. “Optimal Scaling for Various Metropolis-Hastings Algorithms.” Statistical Science 16: 351–67.10.1214/ss/1015346320Search in Google Scholar
Roberts, G. O., and J. S. Rosenthal. 2009. “Examples of Adaptive MCMC.” Journal of Computational & Graphical Statistics 18: 349–67.10.1198/jcgs.2009.06134Search in Google Scholar
Roberts, G. O., A. Gelman, and W. R. Gilks. 1997. “Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithm.” The Annals of Applied Probability 7: 110–20.10.1214/aoap/1034625254Search in Google Scholar
Romano, J. P. 1988. “On Weak Convergence and Optimality of Kernel Density Eestimate of the Mode.” The Annals of Statistics 16: 629–47.10.1214/aos/1176350824Search in Google Scholar
Sain, S. R., K. A. Baggerly, and D. W. Scott. 1994. “Cross-validation of Multivariate Densities.” Journal of the American Statistical Association: Theory and Methods 89: 807–17.10.1080/01621459.1994.10476814Search in Google Scholar
Samb, R. 2010. “Nonparametric Kernel Estimation of the Probability Density Function of Regression Errors Using Estimated Residuals,” Working paper, Universite Pierre et Marie Curie, LSTA, https://arxiv.org/abs/1010.0439.Search in Google Scholar
Samb, R.. 2011. “Nonparametric Estimation of the Density of Regression Errors.” Comptes Rendus Mathematique 349: 1281–5.10.1016/j.crma.2011.10.017Search in Google Scholar
Scott, D. W. 1992. Multivariate Density Estimation: Theory, Practice, and Visualization. New York: Wiley.10.1002/9780470316849Search in Google Scholar
Silverman, B. W. 1986. Density Estimation for Statistics and Data Analysis. New York: Chapman & Hall.Search in Google Scholar
Wahba, G., and S. Wold. 1975. “A Completely Automatic French Curve: Fitting Spline Functions by Cross Validation.” Communications in Statistics – Theory and Methods 4: 1–17.10.1080/03610927508827223Search in Google Scholar
Watson, G. S. 1964. “Smooth Regression Analysis,” Sankhya: The Indian Journal of Statistics, Series A 26: 359–72.Search in Google Scholar
Yafeh, Y., and O. Yosha. 2003. “Large Shareholders and Banks: Who Monitors and How?” The Economic Journal 113: 128–46.10.1111/1468-0297.00087Search in Google Scholar
Yuan, A., and J. G. de Gooijer. 2007. “Semiparametric Regression with Kernel Error Model.” Scandinavian Journal of Statistics 34: 841–69.10.1111/j.1467-9469.2006.00531.xSearch in Google Scholar
Zhang, X., R. D. Brooks, and M. L. King. 2009. “A Bayesian Approach to Bandwidth Selection for Multivariate Kernel Regression with an Application to State-price Density Estimation.” Journal of Econometrics 153: 21–32.10.1016/j.jeconom.2009.04.004Search in Google Scholar
Zhang, X., M. L. King, and R. J. Hyndman. 2006. “A Bayesian Approach to Bandwidth Selection for Multivariate Kernel Density Estimation.” Computational Statistics & Data Analysis 50: 3009–31.10.1016/j.csda.2005.06.019Search in Google Scholar
Zhang, X., M. L. King, and H. L. Shang. 2014. “A Sampling Algorithm for Bandwidth Estimation in a Nonparametric Regression Model with a Flexible Error Density.” Computational Statistics & Data Analysis 78: 218–34.10.1016/j.csda.2014.04.016Search in Google Scholar
Zhang, X., M. L. King, and H. L. Shang. 2016. “Bayesian Bandwidth Selection for a Nonparametric Regression Model with Mixed Types of Regressors.” Econometrics 4, Article 24.10.3390/econometrics4020024Search in Google Scholar
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/snde-2018-0050).
© 2020 Walter de Gruyter GmbH, Berlin/Boston
