Skip to main content
SpringerLink
Log in

Deville and Särndal’s calibration: revisiting a 25-years-old successful optimization problem

  • Invited Paper
  • Published:
TEST Aims and scope Submit manuscript

Abstract

In 1992, in a famous paper, Deville and Särndal proposed the calibration method in order to adjust samples on known population totals. This paper had a very important impact in the theory and practice of survey statistics. In this paper, we propose a rigorous formalization of the calibration problem viewed as an optimization problem. We examine the main calibration functions and we discuss the question of the existence of solutions. We also propose an alternate way of solving the optimization problem given by the calibration principle. We finally present a set of simulations in order to compare the different methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Andersson C, Lennart N (1998) A user’s guide to clan 97: a SAS-program for computation of point- and standard error estimates in sample survey. Technical report, Statistics Sweden

  • Beaumont JF, Bocci C (2008) Another look at ridge calibration. Metron 66(1):5–20

    MATH  Google Scholar 

  • Berger YG (2018) Empirical likelihood approaches in survey sampling. Surv Statistician 78:22–31

    Google Scholar 

  • Berger YG, De La Riva Torres O (2016) Empirical likelihood confidence intervals for complex sampling designs. J Roy Stat Soc B78(2):319–341

    MathSciNet  MATH  Google Scholar 

  • Bethlehem JG, Keller WJ (1987) Linear weighting of sample survey data. J Off Stat 3:141–153

    Google Scholar 

  • Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Brick MJ (2013) Unit nonresponse and weighting adjustments: a critical review. J Off Stat 29(3):329–353

    Google Scholar 

  • Cassel CM, Särndal CE, Wretman JH (1976) Some results on generalized difference estimation and generalized regression estimation for finite population. Biometrika 63:615–620

    MathSciNet  MATH  Google Scholar 

  • Chang T, Kott PS (2008) Using calibration weighting to adjust for nonresponse under a plausible model. Biometrika 95:555–571

    MathSciNet  MATH  Google Scholar 

  • Chaudhuri S, Handcock MS, Rendall MS (2008) Generalized linear models incorporating population level information: an empirical-likelihood-based approach. J R Stat Soc Ser B (Stat Methodol) 70(2):311–328

    MathSciNet  MATH  Google Scholar 

  • Chen J, Qin J (1993) Empirical likelihood estimation for finite populations and the effective usage of auxiliary information. Biometrika 80:107–116

    MathSciNet  MATH  Google Scholar 

  • Chen J, Sitter RR (1999) A pseudo empirical likelihood approach to the effective use of auxiliary information in complex surveys. Statistica Sinica 9:385–406

    MathSciNet  MATH  Google Scholar 

  • Chen J, Wu C (1999) Estimation of distribution function and quantiles using the model-calibrated pseudo empirical likelihood method. Statistica Sinica 12:1223–1239

    MathSciNet  MATH  Google Scholar 

  • Chen J, Sitter RR, Wu C (2002) Using empirical likelihood methods to obtain range restricted weights in regression estimators for surveys. Biometrika 89(1):230–237

    MathSciNet  MATH  Google Scholar 

  • Cholakian V (1980) Un exemple d’application de diverses méthodes d’ajustement d’un tableau à des marges imposées. Les Cahiers de l’Analyse des Données 5:173–176

    Google Scholar 

  • Cornfield J (1944) On samples from finite populations. J Am Stat Assoc 39:236–239

    MathSciNet  MATH  Google Scholar 

  • Davies G, Gillard J, Zhigljavsky A (2015) Calibration in survey sampling as an optimization problem. In: Migdalis A, Karakitsiou A (eds) Optimization, control, and applications in the information age. Springer, New York, pp 67–89

    MATH  Google Scholar 

  • Deming WE (1948) Statistical adjustment of data. Wiley, New York

    MATH  Google Scholar 

  • Deming WE (1950) Some theory of sampling. Dover Publications, New York

    MATH  Google Scholar 

  • Deming WE, Stephan FF (1940) On a least square adjustment of sampled frequency table when the expected marginal totals are known. Ann Math Stat 11:427–444

    MathSciNet  MATH  Google Scholar 

  • Demnati A, Rao JNK (2004) Linearization variance estimators for survey data (with discussion). Surv Methodol 30:17–34

    Google Scholar 

  • Deville JC (1988) Estimation linéaire et redressement sur informations auxiliaires d’enquêtes par sondage. In: Monfort A, Laffond JJ (eds) Mélanges économiques: Essais en l’honneur de Edmond Malinvaud. Economica, Paris, pp 915–927

    Google Scholar 

  • Deville JC (1998) La correction de la non-réponse par calage ou par échantillonnage équilibré. Technical report, Insee, Paris, recueil de la Section des méthodes d’enquête

  • Deville JC (2000) Generalized calibration and application to weighting for non-response. In: Compstat - proceedings in computational statistics: 14th symposium held in Utrecht. The Netherlands, Springer, New York, pp 65–76

    Google Scholar 

  • Deville JC (2002) La correction de la nonréponse par calage généralisé. Actes des Journées de Méthodologie Statistique. Insee-Méthodes, Paris, pp 3–20

    Google Scholar 

  • Deville JC (2004) Calage, calage généralisé et hypercalage. Technical report, Insee, Paris

  • Deville JC, Särndal CE (1990) Estimateur par calage et technique de ratissage généralisé dans les enquêtes par sondage. Technical report, Insee, Paris

  • Deville JC, Särndal CE (1992) Calibration estimators in survey sampling. J Am Stat Assoc 87:376–382

    MathSciNet  MATH  Google Scholar 

  • Deville JC, Särndal CE, Sautory O (1993) Generalized raking procedure in survey sampling. J Am Stat Assoc 88:1013–1020

    MATH  Google Scholar 

  • Dupont F (1994) Calibration used as a nonresponse adjustment. In: Diday E, Lechevallier Y, Schader M, Bertrand P, Burtschy B (eds) New approaches in classification and data analysis. Springer, Berlin, pp 539–548

    Google Scholar 

  • Estevao VM, Särndal CE (2000) A functional form approach to calibration. J Off Stat 16:379–399

    Google Scholar 

  • Estevao VM, Särndal CE (2006) Survey estimates by calibration on complex auxiliary information. Int Stat Rev 74:127–147

    Google Scholar 

  • Estevao VM, Hidiroglou MA, Särndal CE (1995) Methodological principles for a generalized estimation system at statistics canada. J Off Stat 11:181–204

    Google Scholar 

  • Froment R, Lenclud B (1976) Ajustement de tableaux statistiques. Annales de l’Insee 22–23:29–53

    MathSciNet  Google Scholar 

  • Fuller WA, Isaki CT (1981) Currents topics in survey sampling. In: Krewski D, Platek R, Rao JNK (eds) Survey design under superpopulation models. Academic Press, New York, pp 196–226

    MATH  Google Scholar 

  • Fuller WA, Loughin MM, Baker HD (1994) Regression weighting in the presence of nonresponse with application to the 19871988 nationwide food consumption survey. Surv Methodol 20:75–85

    Google Scholar 

  • Goga C, Shehzad MA (2010) Overview of ridge regression estimators in survey sampling. Technical report, Université de Bourgogne, Dijon, France

  • Graf M (2011) Use of survey weights for the analysis of compositional data. In: Pawlowsky-Glahn V, Buccianti A (eds) Compositional data analysis: theory and applications. Wiley, Chichester, pp 114–127

    Google Scholar 

  • Guandalini A, Tillé Y (2017) Design-based estimators calibrated on estimated totals from multiple surveys. Int Stat Rev 85:250–269

    MathSciNet  Google Scholar 

  • Guggemos F, Tillé Y (2010) Penalized calibration in survey sampling: design-based estimation assisted by mixed models. J Stat Plan Inference 140(11):3199–3212

    MathSciNet  MATH  Google Scholar 

  • Güler O (2010) Foundations of optimization, vol 258. Graduate texts in mathematics. Springer, New York

    MATH  Google Scholar 

  • Hartley HO, Rao JNK (1968) A new estimation theory for sample survey. Biometrika 55:547–557

    MATH  Google Scholar 

  • Haziza D, Lesage É (2016) A discussion of weighting procedures for unit nonresponse. J Off Stat 32(1):129–145

    Google Scholar 

  • Horvitz DG, Thompson DJ (1952) A generalization of sampling without replacement from a finite universe. J Am Stat Assoc 47:663–685

    MathSciNet  MATH  Google Scholar 

  • Huang ET, Fuller WA (1978) Non-negative regression estimation for sample survey data. In: Proceedings of the social statistics section of the American Statistical Association, pp 300–305

  • Isaki CT, Fuller WA (1982) Survey design under a regression population model. J Am Stat Assoc 77:89–96

    MATH  Google Scholar 

  • Kim JK, Park H (2010) Calibration estimation in survey sampling. Int Stat Rev/Revue Internationale de Statistique 78(1):21–39

    Google Scholar 

  • Kott PS (1994) A note on handling nonresponse in surveys. J Am Stat Assoc 89:693–696

    MathSciNet  MATH  Google Scholar 

  • Kott PS (2006) Using calibration weighting to adjust for nonresponse and coverage errors. Surv Methodol 32:133–142

    Google Scholar 

  • Kott PS (2009) Calibration weighting: combining probability samples and linear prediction models. In: Pfeffermann D, Rao CR (eds) Handbook of statistics, Part B: sampling, vol 29. Elsevier/North-Holland, New York, Amsterdam, pp 55–82

    Google Scholar 

  • Kott PS, Chang T (2010) Using calibration weighting to adjust for nonignorable unit nonresponse. J Am Stat Assoc 105(491):1265–1275. https://doi.org/10.1198/jasa.2010.tm09016

    Article  MathSciNet  MATH  Google Scholar 

  • Le Guennec J, Sautory O (2002) CALMAR2: une nouvelle version de la macro CALMAR de redressement d’échantillon par calage. Actes des Journées de Méthodologie. Insee, Paris, Paris, pp 33–38

    Google Scholar 

  • Lemel Y (1976) Une généralisation de la méthode du quotient pour le redressement des enquêtes par sondages. Annales de l’Insee 22–23:273–281

    Google Scholar 

  • Lesage É, Haziza D, D’Haultfoeuille X (2018) A cautionary tale on instrumental calibration for the treatment of nonignorable unit nonresponse in surveys. J Am Stat Assoc 114:1–28

    MathSciNet  MATH  Google Scholar 

  • Lumley T (2010) Survey: analysis of complex survey samples. R package version 3.23-0, The Comprehensive R Archive Network

  • Lundström S, Särndal CE (1999) Calibration as a standard method for treatment of nonresponse. J Off Stat 15:305–327

    Google Scholar 

  • Madre JL (1980) Méthodes d’ajustement d’un tableau à des marges. Les Cahiers de l’Analyse des Données 5:87–99

    Google Scholar 

  • Matei A, Tillé Y (2007) Computational aspects of order \(\pi ps\) sampling schemes. Comput Stat Data Anal 51:3703–3717

    MathSciNet  MATH  Google Scholar 

  • Matei A, Tillé Y (2016) The R sampling package, Version 2.8. Université de Neuchâtel, Neuchâtel

    Google Scholar 

  • Narain RD (1951) On sampling without replacement with varying probabilities. J Indian Soc Agric Stat 3:169–174

    MathSciNet  Google Scholar 

  • Nascimento Silva PLD, Skinner CJ (1997) Variable selection for regression estimation in finite populations. Surv Methodol 23(1):23–32

    Google Scholar 

  • Nieuwenbroek NJ, Boonstra HJ (2002) Bascula 4. 0 for weighting sample survey data with estimation of variances. Surv Statistician Softw Rev 46:6–11

    Google Scholar 

  • Owen AB (1988) Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75(2):237–249

    MathSciNet  MATH  Google Scholar 

  • Park M, Yang M (2008) Ridge regression estimation for survey samples. Commun Stat Theory Methods 37(4):532–543

    MathSciNet  MATH  Google Scholar 

  • Rebecq A (2017) icarus: Calibrates and Reweights Units in Samples, likelihood, estimation. R package version 0.3.0. The Comprehensive R Archive Network

  • Roy G, Vanheuverzwyn A (2001) Redressement par la macro CALMAR : applications et pistes d’amélioration. Traitements des fichiers d’enquêtes, éditions PUG pp 31–46

  • Särndal CE (1980) On \(\pi \)-inverse weighting versus best linear unbiased weighting in probability sampling. Biometrika 67:639–650

    MathSciNet  MATH  Google Scholar 

  • Särndal CE (2007) The calibration approach in survey theory and practice. Surv Methodol 33:99–119

    Google Scholar 

  • Särndal CE, Lundström S (2005) Estimation in surveys with nonresponse. Wiley, New York

    MATH  Google Scholar 

  • Särndal CE, Swensson B, Wretman JH (1992) Model assisted survey sampling. Springer, New York

    MATH  Google Scholar 

  • Sautory O (1993) La macro calmar, redressement d’échantillon par calage sur marges. Tech. rep., Séries des documents de travail de la Direction des Statistiques Démographiques et Sociales, F9310, Insee, Paris

  • Sautory O, Le Guennec J (2003) La macro CALMAR2: redressement d’un échantillon par calage sur marges - documentation de l’utilisateur. Technical report, Insee, Paris

  • Shah BV (1981) SESUDAAN, standard errors program for computing of standardized rates from sample survey data. Research Triangle Institute, Research Triangle Park

    Google Scholar 

  • Shah BV, Holt MM, Folsom RE (1977) Inference about regression models from sample survey data. Bull Int Stat Inst 47(3):43–57

    Google Scholar 

  • Shah BV, Folsom RE, Harrell F, Dillard C (1984) Survey data analysis software for logistic regression. Research Triangle Institute, Research Triangle Park

    Google Scholar 

  • Shah BV, Folsom RE, LaVange LM, Wheeless SC, Boyle KE, Williams RL (1993) Statistical methods and mathematical algorithms used in SUDAAN. Research Triangle Institute, Research Triangle Park

    Google Scholar 

  • Shah BV, Barnwell BG, Bieler GS et al (1997) SUDAAN user’s manual, release 7.5., vol 67. Research Triangle Institute, Research Triangle Park

    Google Scholar 

  • Stephan FF (1942) An iterative method of adjusting sample frequency data tables when expected marginal totals are known. Ann Math Stat 13:166–178

    MATH  Google Scholar 

  • Thionet P (1959) L’ajustement des résultats des sondages sur ceux des dénombrements. Revue de l’Institut International de Statistique 27:8–25

    MathSciNet  MATH  Google Scholar 

  • Thionet P (1976) Construction et reconstruction de tableaux statistiques. Annales de l’Insee 22–23:5–27

    MathSciNet  Google Scholar 

  • Tillé Y, Matei A (2016) sampling: Survey Sampling. http://CRAN.R-project.org/package=sampling, R package version 2.8

  • Vanderhoeft C (2001) Generalised calibration at Statistics Belgium SPSS module gCALIBS and current practices. Technical report, statistics Belgium working paper no. 3

  • Vanderhoeft C, Waeytens E, Museux JM (2001) Generalised calibration with SPSS 9. 0 for Windows baser. In: Droesbeke JJ, Lebart L (eds) Enquêtes. Modèles et Applications. Dunod, Paris, pp 404–415

    Google Scholar 

  • Wu C, Rao JNK (2006) Pseudo-empirical likelihood ratio confidence intervals for complex surveys. Can J Stat/La revue canadienne de statistique 34(3):359–376

    MathSciNet  MATH  Google Scholar 

  • Wu C, Sitter RR (2001) A model-calibration approach to using complete auxiliary information from survey data. J Am Stat Assoc 96:185–193

    MathSciNet  MATH  Google Scholar 

  • Yule GU (1912) On the methods of measuring association between two attributes. J Roy Stat Soc 75(6):579–652

    Google Scholar 

Download references

Acknowledgements

The authors thank the Swiss Federal Statistical Office (FSO) which partially supported this work as well as the three reviewers for their comments and efforts towards improving our manuscript.

Author information

Authors and Affiliations

  1. Institut de statistique, Université de Neuchâtel, Av. de Bellevaux 51, 2000, Neuchâtel, Switzerland

    Denis Devaud & Yves Tillé

Authors
  1. Denis Devaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yves Tillé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Denis Devaud.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This invited paper is discussed in comments available at: http://dx.doi.org/10.1007/s11749-019-00682-2; http://dx.doi.org/10.1007/s11749-019-00683-1; http://dx.doi.org/10.1007/s11749-019-00684-0; http://dx.doi.org/10.1007/s11749-019-00686-y; http://dx.doi.org/10.1007/s11749-019-00687-x

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Devaud, D., Tillé, Y. Deville and Särndal’s calibration: revisiting a 25-years-old successful optimization problem. TEST 28, 1033–1065 (2019). https://doi.org/10.1007/s11749-019-00681-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11749-019-00681-3

Keywords

Mathematics Subject Classification

Search

Navigation