Abstract
The objective of the study is to verify a hypothesis that a hydrological model, which successfully passed a comprehensive evaluation test (CE-test), is more suitable for climate impact study than that which failed the test. In our study, the CE-test is a specially designed model evaluation procedure, including a set of enhanced tests of model performance and robustness. The hypothesis verification is carried out with two models, ECOMAG and SWAP, which are applied for the Lena and Mackenzie River basins. The following three versions of every model are compared: (1) version A with a priori assigned parameters (without any calibration); (2) version B calibrated against streamflow observations at the basin outlets only, and (3) version C calibrated against streamflow observations at several gauges within the basins. We found that the B and C versions were successful in passing the CE-test, while the A versions failed the test. The C versions performed better than the B versions, especially at the monthly time scale. Then, all model versions were forced by global climate model (GCM) ensemble data to simulate flow projections for the twenty-first century and assess the projection uncertainty. Summarizing the results, we found that the differences in projections (in terms of mean annual changes in discharge and their uncertainties) between A version and two other versions were nearly three times larger than the differences between the B and C versions. Thus, the CE-test results together with the estimated differences in projections give us reason to conclude that the successful comprehensive evaluation of a model increases its confidence and suitability for impact assessment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andréassian V, Perrin C, Berthet L et al (2009) Crash tests for a standardized evaluation of hydrological models. Hydrol Earth Syst Sci 13(10):1757–1764. https://doi.org/10.5194/hess-13-1757-2009
Bartholomé E, Belward A (2005) GLC2000: a new approach to global land cover mapping from Earth observation data. Int J Remote Sens 26(9):1959–1977
Birhanu D, Kim H, Jang C, Park S (2018) Does the complexity of evapotranspiration and hydrological models enhance robustness? Sustainability 10:2837. https://doi.org/10.3390/su10082837
Brigode P, Oudin L, Perrin C (2013) Hydrological model parameter instability: a source of additional uncertainty in estimating the hydrological impacts of climate change? J Hydrol 476:410–425. https://doi.org/10.2016/j.jhydrol.2012.11.012
Champeaux JL, Masson V, Chauvin F (2005) ECOCLIMAP: a global database of land surface parameters at 1 km resolution. Meteorol Appl 12(1):29–32. https://doi.org/10.1017/S1350482705001519
Coron L, Andréassian V, Perrin C et al (2012) Crash testing hydrological models in contrasted climate conditions: an experiment on 216 Australian catchments. Water Resour Res 48(W05552). https://doi.org/10.1029/2011WR011721
Coron L, Andréassian V, Perrin C et al (2014) On the lack of robustness of hydrologic models regarding water balance simulation: a diagnostic approach applied to three models of increasing complexity on 20 mountainous catchments. Hydrol Earth Syst Sci 18:727–746. https://doi.org/10.5194/hess-18-727-2014
Cosby B, Hornberger GM, Clapp RB, Ginn TR (1984) A statistical exploration of the relationships of soil moisture characteristics to the physical properties of soils. Water Resour Res 20(6):682–690. https://doi.org/10.1029/WR020i006p00682
Duan Q, Schaake J, Andréassian V et al (2006) Model Parameter Estimation Experiment (MOPEX): an overview of science strategy and major results from the second and third workshops. J Hydrol 320(1–2):3–17. https://doi.org/10.1016/j.jhydrol.2005.07.031
Fischer G, Velthuizen H, Shah M, Nachtergaele F (2008) Global agro-ecological zones assessment for agriculture (GAEZ 2008) IIASA. Laxenburg and FAO, Austria and Rome
Gelfan A, Gustafsson D, Motovilov Y et al (2017) Climate change impact on the water regime of two great Arctic rivers: modeling and uncertainty issues. Clim Chang 141(3):499–515. https://doi.org/10.1007/s10584-016-1710-5
Gelfan A, Millionshchikova T (2018) Validation of a hydrological model intended for impact study: problem statement and solution example for Selenga River basin. Water Res 45(S1):90–101. https://doi.org/10.1134/S0097807818050354
Gupta HV, Kling H, Yilmaz KK et al (2009) Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling. J Hydrol 377:80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003
Gusev YM, Nasonova ON (1998) The land surface parameterization scheme SWAP: description and partial validation. Glob Planet Chang 19(1–4):63–86
Gusev YM, Nasonova ON (2003) Modelling heat and water exchange in the boreal spruce forest by the land-surface model SWAP. J Hydrol 280(1–4):162–191
Gusev EM, Nasonova ON, Dzhogan LY (2006) The simulation of runoff from small catchments in the permafrost zone by the SWAP model. Water Res 33(2):115–126. https://doi.org/10.1134/S0097807806020011
Gusev EM, Nasonova ON, Dzhogan LY (2016) Physically based modeling of many-year dynamics of daily streamflow and snow water equivalent in the Lena R. basin. Water Res 43(1):21–32. https://doi.org/10.1134/S0097807816010085
Gusev YM, Nasonova ON, Kovalev EE, Aizel GV (2018) Modelling river runoff and estimating its weather-related uncertainty for 11 large-scale rivers located in different regions of the globe. Hydrol Res 49(4):1072–1087. https://doi.org/10.2166/nh.2017.015
Huang S, Kumar R, Flörke M et al (2017) Evaluation of an ensemble of regional hydrological models in 12 large-scale river basins worldwide. Clim Chang 141:381–397. https://doi.org/10.1007/s10584-016-1841-8
Kalugin AS, Motovilov YG (2018) Runoff formation model for the Amur River basin. Water Res 45(2):149–159. https://doi.org/10.1134/S0097
Klemeš V (1986) Operational testing of hydrological simulation models. Hydrol Sci J 31:13–24
Krysanova V, Donnelly C, Gelfan A et al (2018) How the performance of hydrological models relates to credibility of projections under climate change. Hydrol Sci J 63(5):696–720. https://doi.org/10.1080/02626667.2018.1446214
Kundzewicz ZW (1986) The hydrology of tomorrow. Hydrol Sci J 31(2):223–235
Kundzewicz ZW, Krysanova V, Benestad RE et al (2018) Uncertainty in climate change impacts on water resources. Environ Sci Pol 79:1–8. https://doi.org/10.1016/j.envsci.2017.10.008
Lange S (2018) Bias correction of surface downwelling longwave and shortwave radiation for the EWEMBI dataset. Earth Sys Dynam 9(2):627–645
Merz R, Parajka J, Blöschl G (2011) Time stability of catchment model parameters: implications for climate impact analyses. Water Resour Res 47(W02531). https://doi.org/10.1029/2010WR009505
Moriasi DN, Zeckoski RW, Arnold JG et al (2015) Models: performance measures and evaluation criteria. Trans ASABE 58(6):1763–1785. https://doi.org/10.13031/trans.58.10715
Motovilov YG (2016) Hydrological simulation of river basins at different spatial scales: 1. Generalization and averaging algorithms. Water Res 43(3):429–437. https://doi.org/10.1134/S0097807816030118
Motovilov YG, Gottschalk L, Engeland K, Rodhe A (1999) Validation of a distributed hydrological model against spatial observations. Agric For Meteorol 98-99:257–277. https://doi.org/10.1016/S0168-1923(99)00102-1
Refsgaard JC, Knudsen J (1996) Operational validation and intercomparison of different types of hydrological models. Water Resour Res 32(7):2189–2202. https://doi.org/10.1029/96WR00896
Refsgaard JC, Madsen H, Andréassian V et al (2013) A framework for testing the ability of models to project climate change and its impacts. Clim Chang 122:271–282. https://doi.org/10.1007/s10584-013-0990-2
Santer BD, Wigley TML, Boyle JS et al (2000) Statistical significance of trends and trend differences. J Geophys Res 105(D6):7337–7356. https://doi.org/10.1029/1999JD901105
Schlosser CA, Slater AG, Robock A et al (2000) Simulations of a boreal grassland hydrology at Valdai, Russia: PILPS phase 2(d). Mon Weather Rev 128(2):301–321
Seibert J (2003) Reliability of model predictions outside calibration conditions. Nord Hydrol 34:477–492. https://doi.org/10.2166/nh.2003.0019
Shi X, Qin T, Nie H et al (2019) Changes in major global river discharges directed into the ocean. J Environ Res Publ Health 16:1469. https://doi.org/10.3390/ijerph16081469
Shiklomanov AI et al (2006) Cold region river discharge uncertainty estimates from large Russian rivers. J Hydrol 326:231–256
Smith MB, Seo DJ, Koren VI et al (2004) The distributed model intercomparison project (DMIP): motivation and experiment design. J Hydrol 298(1–4):4–26. https://doi.org/10.1016/j.jhydrol.2004.03.040
Tananaev NI, Makarieva OM, Lebedeva LS (2016) Trends in annual and extreme flows in the Lena River basin, Northern Eurasia. Geophys Res Lett 43(10):764–772. https://doi.org/10.1002/2016GL070796
Thirel G, Andréassian V, Perrin C et al (2015) Hydrology under change: an evaluation protocol to investigate how hydrological models deal with changing catchments. Hydrol Sci J 60(7–8):1184–1199. https://doi.org/10.1080/02626667.2014.967248
Vaze J, Post DA, Chiew FHS et al (2010) Climate nonstationarity – validity of calibrated rainfall-runoff models for use in climatic changes studies. J Hydrol 394(3–4):447–457. https://doi.org/10.1016/j.jhydrol.2010.09.018
Vormoor K, Heistermann M, Bronstert A, Lawrence D (2018) Hydrological model parameter (in)stability – “crash testing” the HBV model under contrasting flood seasonality conditions. Hydrol Sci J 63(7):991–1007. https://doi.org/10.1080/02626667.2018.1466056
Wagener T, McIntyre M, Lees MJ et al (2003) Towards reduced uncertainty in conceptual rainfall-runoff modeling: dynamic identifiability analysis. Hydrol Proced 17:455–476
Woo MK, Thorne R (2014) Winter flows in the Mackenzie drainage system. Arctic 67:238–256
Xu C (1999) Operational testing of a water balance model for predicting climate change impacts. Agric For Meteorol 98-99:295–304. https://doi.org/10.1016/S0168-1923(99)00106-9
Yang D et al (2005) Bias-corrections of long-term (1973-2004) daily precipitation data over the northern regions. Geophys Res Lett 32:L19501. https://doi.org/10.1029/2005GL024057
Yilmaz KK, Gupta HV, Wagener T et al (2008) A process-based diagnostic approach to model evaluation: application to the NWS distributed hydrologic model. Water Resour Res 44(W09417). https://doi.org/10.1029/2007WR006716
Acknowledgments
The authors are grateful to three anonymous reviewers and guest editor Dr. V. Krysanova for their critical comments. Also, we would like to thank Dr. V. Krysanova for constructive discussions that motivated us to think about the study issue far before the earliest draft of the paper, as well as for her invitation to contribute in the Climatic Change Special Issue.
The numerical experiments were designed within the framework of the State Assignment theme № 0147-2019-0001.
The present work was carried out within the framework of the Panta Rhei Research Initiative of the International Association of Hydrological Sciences (IAHS).
Funding
Simulations by the SWAP model were financially supported by the Russian Science Foundation (Grant 16-17-10039). Simulations by the ECOMAG model were financially supported by the Russian Science Foundation (Grant 19-17-00215). Analysis of hydrological projections and their uncertainties was financially supported by the Ministry of Science and Higher Education of the Russian Federation (Grant MK-1753.2020.5).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of a Special Issue on “How evaluation of hydrological models influences results of climate impact assessment,” edited by Valentina Krysanova, Fred Hattermann, and Zbigniew Kundzewicz
Supplementary information
ESM 1
(DOCX 1.45 mb)
Rights and permissions
About this article
Cite this article
Gelfan, A., Kalugin, A., Krylenko, I. et al. Does a successful comprehensive evaluation increase confidence in a hydrological model intended for climate impact assessment?. Climatic Change 163, 1165–1185 (2020). https://doi.org/10.1007/s10584-020-02930-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10584-020-02930-z