Position paper: Sensitivity analysis of spatially distributed environmental models- a pragmatic framework for the exploration of uncertainty sources

Highlights

• A pragmatic framework of sensitivity analyses is provided for spatially distributed environmental models.

• The framework prescribes sequential steps in which important considerations are highlighted.

• The framework benefits users of sensitivity analyses in environmental modeling and GIS communities.

Abstract

Sensitivity analysis (SA) has been used to evaluate the behavior and quality of environmental models by estimating the contributions of potential uncertainty sources to quantities of interest (QoI) in the model output. Although there is an increasing literature on applying SA in environmental modeling, a pragmatic and specific framework for spatially distributed environmental models (SD-EMs) is lacking and remains a challenge. This article reviews the SA literature for the purposes of providing a step-by-step pragmatic framework to guide SA, with an emphasis on addressing potential uncertainty sources related to spatial datasets and the consequent impact on model predictive uncertainty in SD-EMs. The framework includes: identifying potential uncertainty sources; selecting appropriate SA methods and QoI in prediction according to SA purposes and SD-EM properties; propagating perturbations of the selected potential uncertainty sources by considering the spatial structure; and verifying the SA measures based on post-processing. The proposed framework was applied to a SWAT (Soil and Water Assessment Tool) application to demonstrate the sensitivities of the selected QoI to spatial inputs, including both raster and vector datasets - for example, DEM and meteorological information - and SWAT (sub)model parameters. The framework should benefit SA users not only in environmental modeling areas but in other modeling domains such as those embraced by geographical information system communities.

Introduction

Sensitivity analysis (SA) and uncertainty analysis (UA) are important tools for investigating model behavior, testing model hypotheses, and exploring the potential for simplifying models (Wagener and Pianosi, 2019). Uncertainty is intrinsic to all modeling work that involves representing natural processes and/or human behavior. Sources of uncertainty that need to be considered in such exercises are model input datasets, model structure, and model parameters. SA studies the influence of input factors (e.g., parameters, forcing, initial value of model states, model resolution, and model structure such as different parameterization schemes of a model or submodel) on model outputs. It is considered a key practice in the assessment of environmental models (Chen et al., 2019; Gan et al., 2014; Jakeman et al., 2006; Matott et al., 2009; Oakley and O'Hagan, 2004; Pianosi et al., 2016; Yue et al., 2020). In comparison, UA quantifies the uncertainty of model outputs from input datasets and model parameters, typically characterized by empirical probability distributions and/or confidence bounds for the model parameters and outputs. UA can be considered an extension of SA with the uncertainty distributions for the input factors being used as the perturbations. Therefore, SA can be used to indicate when uncertainty in input factors matters in terms of the impact on the uncertainty in the outputs. Care must however be taken in the interpretation of SA results as sensitivities can be dependent on parameter ranges selected, model structure assumed, length of data period examined and its climatic forcing (Shin et al., 2013).

The use of SA and UA in environmental modeling has become of particular importance due to the highly complex nature of environmental systems, and the attendant complexity of models typically invoked to represent them. This is especially the case for spatially distributed environmental models (referred to from hereon as SD-EMs), where there tends to be a considerable number of model parameters due to their spatially variant nature, and substantial uncertainty in the model and its predictions. Uncertainty and sensitivity related studies in environmental modeling are rising in popularity because of the growing awareness of the importance of models in supporting informed decision making (Douglas-Smith et al., 2020), coupled with the fact that current process-based environmental models are typically, and perhaps necessarily, deterministic in their representation (Farmer and Vogel, 2016; Uusitalo et al., 2015).

This paper focuses on Monte Carlo simulation-based SA of SD-EMs, which is a valuable tool per se and one that can also inform uncertainty analyses. Monte Carlo simulation-based approaches are widely applied due to their ease of implementation, yet there is a lack of a comprehensive pragmatic framework for conducting such approaches for SD-EMs (Yang et al., 2018). An SD-EM is intrinsically tied to the spatial dimensions of producing and utilizing data that represent the spatially distributed nature of the modeling context. Grid-based digital elevation models (DEMs), site-specific point measurements, and remotely sensed images are examples of such data. However, SAs are rarely conducted for DEM and DEM-derived parameters even though the inherent scale and errors of a spatial dataset and/or of the whole environmental model can have a significant impact on model outputs (Tran et al., 2018). A crucial issue to take into account regarding spatial datasets is the spatial structure of their uncertainty. Generally, spatial datasets are characterized by spatial dependence (i.e., spatial coherence), and their uncertainties are also spatially autocorrelated (Oksanen and Sarjakoski, 2005a; Wechsler, 2007). Thus, ignoring such characteristics can lead to erroneous estimation of sensitivity measurements. Moreover, because spatial datasets often determine the uncertainty in model resolution and structures through their boundaries, discretization and scale, exploring uncertainty related to spatial datasets can partly account for model uncertainty in SD-EMs.

This article introduces a pragmatic framework for the application of SA to an SD-EM, using a scenario/simulation-based approach to investigate the significance of potential uncertainties in the model inputs, which can not only explore model and data assumptions transparently but also be an informative precursor to a more thorough UA. The objectives of the framework are to provide sufficient information and background in order to guide the selection of more appropriate choices at each step of the SA process: potential uncertainty source identification; selection of SA method(s) and quantities of interest (QoI); perturbation propagation; and SA evaluation and post-processing. The framework emphasizes the following aspects: it attempts to address potential uncertainty sources related to spatial datasets; and assists in propagating the potential uncertainty sources by considering their likely spatial structure. Therefore, the framework helps to explore the impact of potential uncertainty of spatial datasets in an SD-EM, and to compare their relative impacts with the usual factors in SA (e.g., model parameters). The framework is intended to benefit both non-experts and SA users in environmental modeling and geographical information system (GIS) communities.

The remainder of this article is organized as follows. Section 2 broadly introduces the pragmatic framework for applying SA to SD-EMs, covering potential uncertainty source identification, selection of SA method and QoI, perturbation propagation, and SA evaluation and post-processing. Then, from Sections 3 Identification of potential uncertainty sources, 4 Selection of SA method(s) and quantities of interest, 5 Perturbation propagation, 6 SA evaluation and post-processing, the detailed steps and their corresponding considerations are discussed. Section 7 provides a concise example of the SA framework. The article concludes in Section 8 with a discussion of future needs and opportunities.