Skip to main content
SpringerLink
Log in

Robust Subsampling ANOVA Methods for Sensitivity Analysis of Water Resource and Environmental Models

  • Published:
Water Resources Management Aims and scope Submit manuscript

Abstract

Sensitivity analysis is an important component for modelling water resource and environmental processes. Analysis of Variance (ANOVA), has been widely used for global sensitivity analysis for various models. However, the applicability of ANOVA is restricted by this biased variance estimator. To address this issue, the subsampling based ANOVA method are developed in this study, in which multiple subsampling(single-, multiple- and full-subsampling) techniques are proposed to diminish the effect of the biased variance estimator of ANOVA. Two case studies including one simplified regression model and one hydrological model are used to illustrate the applicability of the proposed approaches. Results indicate that: (1) the subsampling procedures effectively diminish the biases resulting from traditional ANOVA method; (2) among the proposed subsampling approaches, the full-subsampling ANOVA has the most robust performance; (3) compared with Sobol’s method, the subsampling ANOVA methods can significantly reduce the calculation requirements while achieve similar sensitivity characterization for model parameters. This study serves as a first basis for the application of subsampling ANOVA methods to sensitivity analysis for water resource and environmental models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Bahremand A, De Smedt F (2008) Distributed hydrological modeling and sensitivity analysis in Torysa watershed. Slovakia Water Resour Manag 22:393–408

    Article  Google Scholar 

  • Bennett KE, Urrego Blanco JR, Jonko A, Bohn TJ, Atchley A, Urban NM, Middleton R (2018) Global sensitivity of simulated water balance indicators under future climate change in the Colorado Basin. Water Resour Res 54(1):132–149

  • Borgonovo E, Plischke E (2016) Sensitivity analysis: a review of recent advances. Eur J Oper Res 248:869–887

    Article  Google Scholar 

  • Bosshard T, Carambia M, Goergen K, Kotlarski S, Krahe P, Zappa M, Schär C (2013) Quantifying uncertainty sources in an ensemble of hydrological climate-impact projections. Water Resour Res 49:1523–1536. https://doi.org/10.1029/2011wr011533

    Article  Google Scholar 

  • Chen X, MolinaCristóbal A, Guenov MD, Riaz A (2019) Efficient method for variance-based sensitivity analysis. Reliab Eng Syst Saf 181:97–115. https://doi.org/10.1016/j.ress.2018.06.016

    Article  Google Scholar 

  • Chowdhury K (2019) Supervised machine learning and heuristic algorithms for outlier detection in irregular spatiotemporal datasets. J Environ Inform 33:1–16. https://doi.org/10.3808/jei.201700375

  • Dessai S, Hulme M (2007) Assessing the robustness of adaptation decisions to climate change uncertainties: a case study on water resources management in the East of England. Glob Environ Chang 17:59–72

    Article  Google Scholar 

  • Đukić V, Radić Z (2016) Sensitivity analysis of a physically based distributed model. Water Resour Manag 30:1669–1684

    Article  Google Scholar 

  • Fan YR, Huang GH, Baetz BW, Li YP, Huang K, Li Z, Chen X, Xiong LH (2016) Parameter Uncertainty and Temporal Dynamics of Sensitivity for Hydrologic Models: a Hybrid Sequential Data Assimilation and Probabilistic Collocation Method. Environ Model Softw 86:30–49. https://doi.org/10.1016/j.envsoft.2016.09.012

  • Fan YR, Huang GH, Li YP, Baetz BW, Huang K (2020) Uncertainty Characterization and Partition in Multivariate Risk Inference: A Factorial Bayesian Copula Framework. Environ Res 183:109215. https://doi.org/10.1016/j.envres.2020.109215

  • Fan YR, Huang K, Huang GH, Li Y, Wang F (2019) An uncertainty partition approach for inferring interactive hydrologic risks. Hydrol Earth Syst Sci Discuss 1–58. https://doi.org/10.5194/hess-2019-434

  • Gamerith V, Neumann MB, Muschalla D (2013) Applying global sensitivity analysis to the modelling of flow and water quality in sewers. Water Res 47:4600–4611. https://doi.org/10.1016/j.watres.2013.04.054

    Article  Google Scholar 

  • Giuntoli I, Vidal JP, Prudhomme C, Hannah DM (2015) Future hydrological extremes: the uncertainty from multiple global climate and global hydrological models. Earth Syst Dyn 6:267–285

    Article  Google Scholar 

  • Hamby DM (1995) A comparison of sensitivity analysis techniques. Health Phys 68:195–204

    Article  Google Scholar 

  • Herman JD, Kollat JB, Reed PM, Wagener T (2013) Technical note: method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models. Hydrol Earth Syst Sci 17:2893–2903. https://doi.org/10.5194/hess-17-2893-2013

    Article  Google Scholar 

  • Hipel KW, McLeod AI (1994) Time series modelling of water resources and environmental systems, vol 45. Elsevier, New York

    Book  Google Scholar 

  • Khaiter P, Erechtchoukova M (2019) Conceptualizing an environmental software modeling framework for sustainable management using UML. J Environ Inform 34:123–138. https://doi.org/10.3808/jei.201800400

  • Khorashadi Zadeh F, Nossent J, Sarrazin F, Pianosi F, van Griensven A, Wagener T, Bauwens W (2017) Comparison of variance-based and moment-independent global sensitivity analysis approaches by application to the SWAT model. Environ Model Softw 91:210–222. https://doi.org/10.1016/j.envsoft.2017.02.001

    Article  Google Scholar 

  • Li Z et al (2015) Development of a stepwise-clustered hydrological inference model. J Hydrol Eng 20:04015008

    Article  Google Scholar 

  • Lindenschmidt K, Rokaya P (2019) A stochastic hydraulic modelling approach to determining the probable maximum staging of ice-jam floods. J Environ Inform 34:45–54. https://doi.org/10.3808/jei.201900416

  • Liu Y, Chaubey I, Bowling LC, Bralts VF, Engel BA (2016) Sensitivity and uncertainty analysis of the L-THIA-LID 2.1 model. Water Resour Manag 30:4927–4949

    Article  Google Scholar 

  • Maqsood I, Huang GH, Huang YF, Chen B (2005) ITOM: an interval-parameter two-stage optimization model for stochastic planning of water resources systems. Stoch Env Res Risk A 19(2):125–133

  • Morris MD (1991) Factorial sampling plans for preliminary computational experiments. Technometrics 33:161–174

    Article  Google Scholar 

  • Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models part I – a discussion of principles. J Hydrol 10:282–290

    Article  Google Scholar 

  • Oladyshkin S, De Barros F, Nowak W (2012) Global sensitivity analysis: a flexible and efficient framework with an example from stochastic hydrogeology. Adv Water Resour 37:10–22

    Article  Google Scholar 

  • Oudin L, Perrin C, Mathevet T, Andréassian V, Michel C (2006) Impact of biased and randomly corrupted inputs on the efficiency and the parameters of watershed models. J Hydrol 320:62–83. https://doi.org/10.1016/j.jhydrol.2005.07.016

    Article  Google Scholar 

  • Pappenberger F, Beven KJ, Ratto M, Matgen P (2008) Multi-method global sensitivity analysis of flood inundation models. Adv Water Resour 31:1–14

    Article  Google Scholar 

  • Perrin C, Michel C, Andréassian V (2003) Improvement of a parsimonious model for streamflow simulation. J Hydrol 279:275–289. https://doi.org/10.1016/s0022-1694(03)00225-7

    Article  Google Scholar 

  • Pianosi F, Beven K, Freer J, Hall JW, Rougier J, Stephenson DB, Wagener T (2016) Sensitivity analysis of environmental models: a systematic review with practical workflow. Environ Model Softw 79:214–232. https://doi.org/10.1016/j.envsoft.2016.02.008

    Article  Google Scholar 

  • Qi W, Zhang C, Fu G, Sweetapple C, Zhou H (2016a) Evaluation of global fine-resolution precipitation products and their uncertainty quantification in ensemble discharge simulations. Hydrol Earth Syst Sci 20:903–920. https://doi.org/10.5194/hess-20-903-2016

    Article  Google Scholar 

  • Qi W, Zhang C, Fu G, Zhou H (2016b) Imprecise probabilistic estimation of design floods with epistemic uncertainties. Water Resour Res 52(6):4823–4844. https://doi.org/10.1002/2015WR017663

  • Qi W, Zhang C, Fu G, Zhou H (2016c) Quantifying dynamic sensitivity of optimization algorithm parameters to improve hydrological model calibration. J Hydrol 533:213–223. https://doi.org/10.1016/j.jhydrol.2015.11.052

    Article  Google Scholar 

  • Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S (2010) Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput Phys Commun 181:259–270. https://doi.org/10.1016/j.cpc.2009.09.018

    Article  Google Scholar 

  • Saltelli A et al (2019) Why so many published sensitivity analyses are false: a systematic review of sensitivity analysis practices. Environ Model Softw 114:29–39. https://doi.org/10.1016/j.envsoft.2019.01.012

    Article  Google Scholar 

  • Shin M, Guillaume JHA, Croke BFW, Jakeman AJ (2013) Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R. J Hydrol 503:135–152. https://doi.org/10.1016/j.jhydrol.2013.08.047

    Article  Google Scholar 

  • Sobol’ BIM (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comput Exp 1(4):407–414

  • Tang T, Reed P, Wagener T, Van Werkhoven K (2006) Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol Earth Syst Sci Discuss 3:3333–3395

    Article  Google Scholar 

  • Tang Y, Reed PM, Wagener T, van Werkhoven K (2008) Comparison of parameter sensitivity analysis methods for lumped watershed model. In: World environmental and water resources Congress 2008: Ahupua’A, pp 1–8. American Society of Civil Engineers. Honolulu, Hawaii. https://doi.org/10.1061/40976(316)612

  • Tao Z et al (2011) Estimation of carbon sinks in chemical weathering in a humid subtropical mountainous basin. Chin Sci Bull 56:3774–3782. https://doi.org/10.1007/s11434-010-4318-6

    Article  Google Scholar 

  • Tian W (2013) A review of sensitivity analysis methods in building energy analysis. Renew Sust Energ Rev 20:411–419

    Article  Google Scholar 

  • Tsakiris G (1982) A method for applying crop sensitivity factors in irrigation scheduling. Agric Water Manag 5:335–343

    Article  Google Scholar 

  • Tsakiris G, Spiliotis M (2017) Uncertainty in the analysis of urban water supply and distribution systems. J Hydroinformatics 19:823–837

    Article  Google Scholar 

  • Uusitalo L, Lehikoinen A, Helle I, Myrberg K (2015) An overview of methods to evaluate uncertainty of deterministic models in decision support. Environ Model Softw 63:24–31. https://doi.org/10.1016/j.envsoft.2014.09.017

    Article  Google Scholar 

  • Vega M, Pardo R, Barrado E, Debán L (1998) Assessment of seasonal and polluting effects on the quality of river water by exploratory data analysis. Water Res 32:3581–3592

    Article  Google Scholar 

  • Vitale D, Bilancia M, Papale D (2019) A multiple imputation strategy for eddy covariance data. J Environ Inform 34:68–87. https://doi.org/10.3808/jei.201800391

  • Wang S, Ancell BC, Huang GH, Baetz BW (2018) Improving robustness of hydrologic ensemble predictions through probabilistic pre- and Postprocessing in sequential data assimilation. Water Resour Res 54(3): 2129–2151. https://doi.org/10.1002/2018WR022546

  • Weng SQ, Huang GH, Li YP (2010) An integrated scenario-based multi-criteria decision support system for water resources management and planning–A case study in the Haihe River Basin. Expert Syst Appl 37(12):8242–8254

  • Wu SM, Huang GH, Guo HC (1997) An interactive inexact-fuzzy approach for multiobjective planning of water resource systems. Water Sci Technol 36(5):235–242

  • Wu H, Chen B, Snelgrove K, Lye LM (2019) Quantification of uncertainty propagation effects during statistical downscaling of precipitation and temperature to hydrological modeling. J Environ Inform 34:139–148. https://doi.org/10.3808/jei.201600347

  • Xu L, Li G, Mays LW (2001) Optimal operation of soil aquifer treatment systems considering parameter uncertainty. Water Resour Manag 15:123–147

    Article  Google Scholar 

  • Zhang Z, Zhang Q, Singh VP, Shi P (2018) River flow modelling: comparison of performance and evaluation of uncertainty using data-driven models and conceptual hydrological model. Stoch Env Res Risk A 32:2667–2682

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Key Research and Development Plan (2016YFC0502800, 2016YFA0601502), the Natural Sciences Foundation (51520105013, 51679087), and the Natural Science and Engineering Research Council of Canada. All information used in this research is available in the Hydrological Data of Pearl River Basin, Annual Hydrology Report.

Author information

Authors and Affiliations

  1. State Key Joint Laboratory of Environmental Simulation and Pollution Control, CEEER-URBNU, College of Environment, Beijing Normal University, Beijing, 100875, China

    F. Wang, G. H. Huang & Y. P. Li

  2. Institute for Energy, Environment and Sustainable Communities, University of Regina, Regina, Saskatchewan, Canada

    G. H. Huang

  3. Department of Civil and Environmental Engineering, Brunel University London, Middlesex, Uxbridge, UB8 3PH, UK

    Y. Fan

Authors
  1. F. Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. H. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Y. Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Y. P. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. H. Huang.

Ethics declarations

Conflict of Interest

The authors have no conflict of interest to declare.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 302 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, F., Huang, G.H., Fan, Y. et al. Robust Subsampling ANOVA Methods for Sensitivity Analysis of Water Resource and Environmental Models. Water Resour Manage 34, 3199–3217 (2020). https://doi.org/10.1007/s11269-020-02608-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11269-020-02608-2

Keywords

Search

Navigation