Skip to main content
SpringerLink
Log in

Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

In general, the solution to a regression problem is the minimizer of a given loss criterion and depends on the specified loss function. The nonparametric isotonic regression problem is special, in that optimal solutions can be found by solely specifying a functional. These solutions will then be minimizers under all loss functions simultaneously as long as the loss functions have the requested functional as the Bayes act. For the functional, the only requirement is that it can be defined via an identification function, with examples including the expectation, quantile, and expectile functionals. Generalizing classical results, we characterize the optimal solutions to the isotonic regression problem for identifiable functionals by rigorously treating these functionals as set-valued. The results hold in the case of totally or partially ordered explanatory variables. For total orders, we show that any solution resulting from the pool-adjacent-violators algorithm is optimal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Ayer, M., Brunk, H. D., Ewing, G. M., Reid, W. T., Silverman, E. (1955). An empirical distribution function for sampling with incomplete information. Annals of Mathematical Statistics, 26, 641–647.

    Article  MathSciNet  Google Scholar 

  • Barlow, R. E., Bartholomew, D. J., Bremner, J. M., Brunk, H. D. (1972). Statistical inference under order restrictions. London: Wiley.

    MATH  Google Scholar 

  • Bartholomew, D. J. (1959a). A test of homogeneity for ordered alternatives. Biometrika, 46, 36–48.

    Article  MathSciNet  Google Scholar 

  • Bartholomew, D. J. (1959b). A test of homogeneity for ordered alternatives. II. Biometrika, 46, 328–335.

    Article  MathSciNet  Google Scholar 

  • Bellec, P. C. (2018). Sharp oracle inequalities for least squares estimators in shape restricted regression. The Annals of Statistics, 46, 745–780.

    MathSciNet  MATH  Google Scholar 

  • Brümmer, N., Du Preez, J. (2013). The PAV algorithm optimizes binary proper scoring rules. arXiv:1304.2331.

  • Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Annals of Mathematical Statistics, 26, 607–616.

    Article  MathSciNet  Google Scholar 

  • Dawid, A. P. (2016). Contribution to the discussion of Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings by Ehm, W., Gneiting, T., Jordan, A. and Krüger, F. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.

    Article  MathSciNet  Google Scholar 

  • Ehm, W., Gneiting, T., Jordan, A., Krüger, F. (2016). Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.

    Article  MathSciNet  Google Scholar 

  • Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association, 106, 746–762.

    Article  MathSciNet  Google Scholar 

  • Groeneboom, P., Jongbloed, G. (2014). Nonparametric estimation under shape constraints. New York: Cambridge University Press.

    Book  Google Scholar 

  • Guntuboyina, A., Sen, B. (2018). Nonparametric shape-restricted regression. Statistical Science, 33, 568–594.

    Article  MathSciNet  Google Scholar 

  • Gurney, A. J. T., Griffin, T. G. (2011). Pathfinding through congruences. Relational and Algebraic Methods in Computer Science (Vol. 6663, pp. 180–195). Heidelberg: Springer.

  • Han, Q., Wang, T., Chatterjee, S., Samworth, R. J. (2019). Isotonic regression in general dimensions. The Annals of Statistics, 47, 2440–2471.

    MathSciNet  MATH  Google Scholar 

  • Huber, P. J. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 35, 73–101.

    Article  MathSciNet  Google Scholar 

  • Kyng, R., Rao, A., Sachdeva, S. (2015). Fast, provable algorithms for isotonic regression in all \({L}_{p}\)-norms. Advances in Neural Information Processing Systems 28 (pp. 2719–2727). Red Hook: Curran Associates Inc.

  • Luss, R., Rosset, S. (2014). Generalized isotonic regression. Journal of Computational and Graphical Statistics, 23, 192–210.

    Article  MathSciNet  Google Scholar 

  • Luss, R., Rosset, S. (2017). Bounded isotonic regression. Electronic Journal of Statistics, 11, 4488–4514.

    Article  MathSciNet  Google Scholar 

  • Miles, R. E. (1959). The complete amalgamation into blocks, by weighted means, of a finite set of real numbers. Biometrika, 46, 317–327.

    Article  MathSciNet  Google Scholar 

  • Mösching, A., Dümbgen, L. (2020). Monotone least squares and isotonic quantiles. Electronic Journal of Statistics, 14, 24–49.

    Article  MathSciNet  Google Scholar 

  • Newey, W. K., Powell, J. L. (1987). Asymmetric least squares estimation and testing. Econometrica, 55, 819–847.

    Article  MathSciNet  Google Scholar 

  • Pardalos, P. M., Xue, G. (1999). Algorithms for a class of isotonic regression problems. Algorithmica, 23, 211–222.

    Article  MathSciNet  Google Scholar 

  • Patton, A. J. (2011). Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics, 160, 246–256.

    Article  MathSciNet  Google Scholar 

  • Patton, A. J. (2020). Comparing possibly misspecified forecasts. Journal of Business & Economic Statistics, 38, 796–809.

    Article  MathSciNet  Google Scholar 

  • Polonik, W. (1998). The silhouette, concentration functions and ML-density estimation under order restrictions. The Annals of Statistics, 26, 1857–1877.

    Article  MathSciNet  Google Scholar 

  • Robertson, T., Wright, F. T. (1973). Multiple isotonic median regression. The Annals of Statistics, 1, 422–432.

    Article  MathSciNet  Google Scholar 

  • Robertson, T., Wright, F. T. (1980). Algorithms in order restricted statistical inference and the Cauchy mean value property. The Annals of Statistics, 8, 645–651.

    Article  MathSciNet  Google Scholar 

  • Savage, L. J. (1971). Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66, 783–801.

    Article  MathSciNet  Google Scholar 

  • Stout, Q. F. (2015). Isotonic regression for multiple independent variables. Algorithmica, 71, 450–470.

    Article  MathSciNet  Google Scholar 

  • Sysoev, O., Burdakov, O., Grimvall, A. (2011). A segmentation-based algorithm for large-scale partially ordered monotonic regression. Computational Statistics & Data Analysis, 55, 2463–2476.

    Article  MathSciNet  Google Scholar 

  • Thompson, W. A., Jr. (1962). The problem of negative estimates of variance components. Annals of Mathematical Statistics, 33, 273–289.

    Article  MathSciNet  Google Scholar 

  • van Eeden, C. (1958). Testing and estimating ordered parameters of probability distributions. Amsterdam: Mathematical Centre.

    MATH  Google Scholar 

  • Ziegel, J. F. (2016). Contribution to the discussion of Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings by Ehm, W., Gneiting, T., Jordan, A. and Krüger, F. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank two reviewers, Tilmann Gneiting, Alexandre Mösching and Lutz Dümbgen for inspiring discussions and valuable comments. Alexander I. Jordan acknowledges the support of the Klaus Tschira Foundation. Anja Mühlemann and Johanna F. Ziegel gratefully acknowledge financial support from the Swiss National Science Foundation.

Author information

Authors and Affiliations

  1. Computational Statistics (CST) Group, Heidelberg Institute for Theoretical Studies, Schloss-Wolfsbrunnenweg 35, 69118, Heidelberg, Germany

    Alexander I. Jordan

  2. Institute of Mathematical Statistics and Actuarial Science, University of Bern, Alpeneggstrasse 22, 3012, Bern, Switzerland

    Anja Mühlemann & Johanna F. Ziegel

Authors
  1. Alexander I. Jordan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anja Mühlemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johanna F. Ziegel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander I. Jordan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jordan, A.I., Mühlemann, A. & Ziegel, J.F. Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals. Ann Inst Stat Math 74, 489–514 (2022). https://doi.org/10.1007/s10463-021-00808-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-021-00808-0

Keywords

Search

Navigation