Abstract
M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arcones MA (2000) Asymptotic distribution of regression M-estimators. J Stat Plan inference 97:235–261
Bai ZD, Wu Y (1994) Limiting behavior of M-estimators of regression coefficients in high dimensional linear models I. Scale-dependent case. J Multivar Anal 51:211–239
Cai TT, Yuan M (2011) Optimal estimation of the mean function based on discretely sampled functional data: phase transition. Ann Stat 39:2330–2355
Cox DD (1983) Asymptotics for M-type smoothing splines. Ann Stat 11:530–551
Cunningham J, Eubank RL, Hsing T (1991) M-type smoothing splines with auxiliary scale estimation. Comput Stat Data Anal 11:43–51
De Vito S, Massera E, Piga M, Martinotto L, Di Francia G (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B: Chem 129:750–757
Eggermont PPB, LaRiccia VN (2009) Maximum penalized likelihood estimation, volume II: regression. Springer, New York
Eubank RL (1999) Nonparametric regression and spline smoothing, 2nd edn. CRC Press, New York
Fu A, Narasimhan B, Boyd S (2020) CVXR: an R package for disciplined convex optimization. J Stat Soft 94:1–34
Ghement IR, Ruiz M, Zamar R (2008) Robust estimation of error scale in nonparametric regression models. J Stat Plan Inference 138:3200–3216
Green PJ, Silverman BW (1994) Nonparametric regression and generalized linear models: a roughness penalty approach. Chapman & Hall, Suffolk
Hastie TJ, Tibshirani RJ, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York
He X, Shi P (1995) Asymptotics for M-type regression splines with auxiliary scale estimation. Indian J Stat Ser A 57:452–461
Huber PJ (1979) Robust Smoothing. In: Launer RL, Wilkinson GN (eds) Robustness in statistics. Academic Press, New York, pp 33–47
Huber PJ, Ronchetti EM (2009) Robust statistics, 2nd edn. Wiley, Hoboken
Maronna RA (2011) Robust ridge regression for high-dimensional data. Technometrics 53:44–53
Maronna RA, Zamar RH (2002) Robust estimates of location and dispersion for high-dimensional datasets. Technometrics 44:307–317
Maronna RA, Martin D, Salibián-Barrera M, Yohai VJ (2019) Robust statistics: theory and methods, 2nd edn. Wiley, Chichester
Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New York
Nychka D, Martin D, Haaland P, O’Connell M (1995) A nonparametric regression approach to syringe grading for quality improvement. J Am Stat Assoc 90:1171–1178
Oh H-S, Nyckha DeW (2007) The role of pseudo data for robust smoothing with application to wavelet regression. Biometrika 94:893–904
Pollard D (1991) Asymptotics for least absolute deviation regression estimators. Econom Theory 7:186–199
R Core Team (2018) R: A Language and Environment for Statistical Computing,R Foundation for Statistical Computing. Austria, Vienna
Rice J, Rosenblatt M (1981) Integrated mean squared error of a smoothing spline. J Approx Theory 33:353–369
Silverman BW (1996) Smoothed functional principal components analysis by choice of norm. Ann Stat 24:1–24
Stone CJ (1982) Optimal global rates of convergence for nonparametric regression. Ann Stat 10:1040–1053
Stone CJ (1985) Additive regression and other nonparametric models. Ann Stat 13:689–705
van de Geer S (2000) Empirical processes in M-estimation. Cambridge University Press, New York
van de Geer S (2002) M-estimation using penalties or sieves. J Stat Plan Inference 108:55–69
Wahba G (1990) Spline models for observational data. SIAM, Philadelphia
Welsh AH (1989) On M-processes and M-estimation. Ann Stat 17:337–361
Yohai VJ, Maronna RA (1979) Asymptotic behavior of M-estimators for the linear model. Ann Stat 7:258–268
Yohai VJ, Zamar RH (1988) High breakdown-point estimates of regression by means of the minimization of an efficient scale. J Am Stat Assoc 83:406–413
Acknowledgements
The author is grateful to two anonymous referees and an associate editor for valuable comments and suggestions that greatly improved the paper both with respect to content and accessibility. He is also indebted to Stefan Van Aelst for helpful conversations. Support from Grant C16/15/068 of KU Leuven is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Software availability
R-scripts reproducing the simulation experiments and real-data examples are available in https://github.com/ioanniskalogridis/Smoothing-splines.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kalogridis, I. Asymptotics for M-type smoothing splines with non-smooth objective functions. TEST 31, 373–389 (2022). https://doi.org/10.1007/s11749-021-00782-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11749-021-00782-y