Research papers
Improving daily streamflow simulations for data-scarce watersheds using the coupled SWAT-LSTM approach

https://doi.org/10.1016/j.jhydrol.2023.129734Get rights and content

Highlights

  • Daily streamflow in data-scarce watersheds is reproduced using coupled SWAT-LSTM.

  • Precipitation contributes the most to simulated streamflow.

  • SWAT-LSTM without retraining performs satisfactory in ungauged watersheds.

  • Transfer learning using few observations optimizes model in poorly gauged watersheds.

  • Consider both spatial proximity and physical similarity for robustness of source selection.

Abstract

There is a scarcity of streamflow data owing to the limited availability of gauge networks or delayed gauging in most parts of the world. To overcome this challenge and reproduce long-duration daily streamflow in both ungauged and poorly gauged watersheds, we proposed a novel approach that couples the process-based model Soil and Water Assessment Tool (SWAT) and the interpretable machine learning (ML) model long short-term memory (LSTM). The watershed process features generated by SWAT were combined with meteorological features as inputs for LSTM. The coupled SWAT-LSTM approach was first developed in a data-rich coastal watershed in Fujian Province, China. During the testing period, the obtained Nash-Sutcliffe efficiency coefficient (NSE) of SWAT-LSTM is 0.885, which outperformed other SWAT-MLs (e.g., backward propagation neural network, NSE = 0.843; random forest, NSE = 0.838) and calibrated SWAT (NSE = 0.706) used as comparators. Precipitation is considered the most important feature to local streamflow from a ML perspective. The pre-trained SWAT-LSTM presented satisfactory performances over 30 years of simulations in 24 hypothesized data-scarce watersheds. In ungauged watersheds, the NSE ranged from 0.474 to 0.898, with a mean of 0.685. In poorly gauged watersheds, the pre-trained SWAT-LSTM was optimized using limited local observations by introducing the transfer learning technique, and the NSE ranged from 0.591 to 0.918, with a mean of 0.760, which was markedly more accurate than the new trained models locally. Spatial proximity and physical similarity should be considered simultaneously when selecting the optimal source for data-scarce watersheds, as better performance can be achieved in less time than with tandem trained the observations of all sources. This study demonstrates that coupling SWAT with interpretable LSTM enhances the modeling confidence and provides a potential shortcut to achieving long-duration streamflow simulations in both ungauged and poorly gauged watersheds.

Introduction

Streamflow is a fundamental indicator of the terrestrial water cycle (Gudmundsson et al., 2017, Irving et al., 2018), and accessible streamflow data are crucial for flood/drought risk assessment, sustainable water resource management, and water resource optimization in the face of climate change and human activities (Feng et al., 2022b, Miao et al., 2022, Wiel et al., 2019). However, there is a significant geographical imbalance in the global availability of streamflow data (Do et al., 2018, Ma et al., 2021). Apart for the US and parts of Europe, there is a scarcity of streamflow gauge networks or late starting of gauging have led to the limited streamflow data (poorly gauged watersheds, PGWs) or even no data (ungauged watersheds, UWs) in many watersheds (Do et al., 2017). Compared with data-rich watersheds, these data-scarce watersheds also require adequate streamflow data to effectively manage water resources.

Model simulations are widely used to study streamflow variability patterns (Kasiviswanathan et al., 2016). Process-based models (PBMs) are classic and powerful tools for generating streamflow series by describing and quantifying realistic watershed hydrological processes using complex mathematical equations (Chen et al., 2022b, Fatichi et al., 2016, Swain and Patra, 2017). Among PBMs, a semi-distributed watershed model named Soil and Water Assessment Tool (SWAT) is widely used in hydrological modeling (Arnold et al., 1998, Tan et al., 2020, van Griensven et al., 2012, Xie et al., 2023). Nonetheless, the calibration process for SWAT is often time-consuming and complex owing to the large number of required parameters and their complex interactions (Rezaeianzadeh et al., 2013, Tokar and Markus, 2000). As another modeling approach, machine learning (ML) from the field of artificial intelligence have been successfully applied to water environment research over two decades (Adnan et al., 2020, Maier and Dandy, 2000). Generally, ML models require less expertise and time for developing and calculating (LeCun et al., 2015), and can solve highly nonlinear problems without considering the physical processes (Chen et al., 2022a). Furthermore, they generally demonstrate better performance than PBMs (Humphrey et al., 2016, Kratzert et al., 2018). In recent years, the long short-term memory (LSTM) model with unique internal structure, has demonstrated unparalleled popularity and robustness in hydrological modeling (Feng et al., 2022a, Xiang et al., 2020). However, both PBMs and MLs must be calibrated or trained with sufficient observations to achieve satisfactory performance (Qi et al., 2020). Without adequate calibration, the utility of PBMs cannot be determined owing to the scarcity of observations (Noori and Kalin, 2016), and serious errors may occur in peak and valley values, even if their simulations exhibit patterns similar to those of observations (Abbaspour et al., 2015, Zhang et al., 2020). Similarly, the accuracy of MLs in PGWs is difficult to guarantee, since it cannot provide sufficient observations for training, let alone in UWs. Therefore, accurate streamflow simulating remains a challenge, in data-scarce watersheds (de Lavenne et al., 2023, Yoon et al., 2022), neither PBM nor ML models are robust when developed directly.

Various regionalization approaches have been employed to develop models with good spatial transferability that can accurately simulate streamflow, by transferring knowledge from data-rich to data-scarce watersheds (Arsenault et al., 2023, Guo et al., 2020, Tegegne and Kim, 2018, Wu et al., 2022). For instance, Pool et al. (2021) evaluated 19 regionalization approaches to simulate streamflow in 671 UWs in the US and found that most watersheds benefited from careful source selection and obtained PBM parameters from suitably selected data-rich watersheds. While in PGWs, a novel approach named transfer learning (TL) has been introduced, which is an important paradigm in ML field (Shen, 2018, Weiss et al., 2016). TL involves feeding available information from PGWs into pre-trained ML models from data-rich watersheds for retraining and local adaptation/optimization to improve performance (Chen et al., 2021, Willard et al., 2021, Zhu et al., 2021). Additionally, the coupled modeling of PBMs and ML models has been shown to be a promising approach in hydrological-related research that is superior to individual modeling (Fang et al., 2022, Read et al., 2019, Reichstein et al., 2019, Wang et al., 2022). Researchers coupled PBMs with ML models to simulate streamflow in UWs (Noori and Kalin, 2016) and PGWs (Yang et al., 2020) and found that coupled modeling can amplify the advantages and overcome the limitations of each model, such as the cumbersome calibrating processes of PBMs and the high data demand of ML models. There is an urgent need to leverage the benefits of coupled modeling, combining classical and emerging techniques to improve the accuracy of streamflow simulations in data-scarce watersheds.

Most coupled modeling approaches have focused on using PBMs to generate long time series variables and then exploiting the nonlinear fitting capability of ML models to improve the streamflow simulation accuracy in data-scarce watersheds, without considering ML interpretations. For example, Noori and Kalin (2016) simulated streamflow in 25 UWs in the US by splitting SWAT-simulated streamflow into baseflow and direct flow and using these flows as the input features of artificial neural networks. Yang et al. (2020) used a calibrated geomorphology-based hydrological model to simulate streamflow and coupled climatic and watershed geographical attributes as inputs of artificial neural network to simulate streamflow in PGWs in Thailand. The aforementioned approaches obtained high accuracy but did not consider the corresponding interpretation of watershed hydrological processes. When developing ML models, it is necessary to consider their black-box attributes, which has triggered considerable debate on whether such models are sufficiently comprehensible, thus impeding their wide application in hydrology (Nearing et al., 2021, Rudin, 2019). Interpretability is a critical aspect of ML models development in hydrology, because it can provide insights into how black-box models generate outputs using the given inputs, thus enhancing researchers’ confidence in the simulations (Fleming et al., 2021, Xiong et al., 2022). To our knowledge, no previous research has coupled PBMs with ML models to interpret watershed processes and systematically analyze the streamflow simulation capabilities of the coupled approach in both UWs and PGWs.

This study proposes a novel approach that couples SWAT (a PBM) and LSTM (a ML model), hereinafter, SWAT-LSTM, that explicitly considers climatic and watershed processes and can be used to simulate streamflow in both UWs and PGWs. Watershed process-related variables were generated using SWAT and combined with meteorological features as inputs for LSTM. First, SWAT-LSTM was developed in a data-rich watershed (a coastal watershed in Fujian Province, southeastern China). After post-hoc interpretations of the pre-trained models, the streamflow simulation abilities of proposed approach for UWs and PGWs were examined in 24 surrounding watersheds distributed across three watersheds in Fujian Province. This study aimed to address two main research questions: (1) From the perspective of MLs, what are the key driving forces in watershed processes for streamflow variation? (2) Does the coupling of PBM and ML lead to satisfactory performance in streamflow simulations in data-scarce watersheds? This study demonstrated that the coupled modeling of a PBM and an interpretable ML based on watershed process-related knowledge can substantially improve the accuracy of streamflow simulations in both UWs and PGWs, thereby representing a successful approach to streamflow simulation in data-scarce watersheds.

Section snippets

Study area

The Jiulong River Watershed (JRW) is located in Fujian Province, southeastern China (Fig. 1b), and has a typical subtropical monsoon climate. The total area of this costal watershed is 14,741 km2, mean annual temperature is 20.9 °C, and mean annual precipitation is 1,600 mm. Three tributaries, the North, West, and South Rivers, converge downstream (southeast of the JRW) and feed into the Taiwan Strait. Among these three sub-watersheds, the North River watershed is the largest, with a drainage

Performance in data-rich watershed

The importance of the 12 input features was first quantified and ranked using SHAP, and the six features with negligible contributions were gradually deleted. The remaining six features, PCP, LATQ, PERC, GWQ, SURQ, and ET of the current day and previous 21 days were input features for each SWAT-MLs. Parameter tuning for the MLs and SWAT can be found in Text S1, and the performance of each model during the training/calibration and testing/validation periods is listed in Table 1. During the

Effectiveness of the proposed approach

The task of reproducing streamflow series for data-scarce watersheds is crucial for supporting local sustainable water management. To address the challenge of unsatisfactory performance in traditional singular modeling approaches (such as PBM or ML) due to the lack of observations (Grimaldi et al., 2021), we successfully coupled SWAT and LSTM in this study. This approach achieved complementarity, providing a new way to simulate streamflow in data-scarce watersheds.

One advantage of the proposed

Conclusion

In summary, our study demonstrates that the coupled SWAT and LSTM approach can effectively reproduce long-term streamflow in both ungauged and poorly gauged watersheds. Our SWAT-LSTM was first trained and tested in a data-rich watershed using over 60 years of local observations and could reasonably explain local watershed hydrological processes. In UWs, the pre-trained model from data-rich watershed performed satisfactorily based on inter-watershed hydrological consistency, and was far better

CRediT authorship contribution statement

Shengyue Chen: Conceptualization, Data curation, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing. Jinliang Huang: Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing. Jr-Chuan Huang: Methodology, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

The authors would like to extend their gratitude to the Fujian Provincial Water Resources Department for providing the streamflow data. This study was financially supported by the National Natural Science Foundation of China (Grant No. 41971231).

References (71)

  • N. Noori et al.

    Coupling SWAT and ANN models for enhanced daily streamflow prediction

    J. Hydrol.

    (2016)
  • J.B. Swain et al.

    Streamflow estimation in ungauged catchments using regionalization techniques

    J. Hydrol.

    (2017)
  • G. Tegegne et al.

    Modelling ungauged catchments using the catchment runoff response similarity

    J. Hydrol.

    (2018)
  • R.M. Adnan

    Least square support vector machine and multivariate adaptive regression splines for streamflow prediction in mountainous basin using hydro-meteorological data as inputs

    J. Hydrol.

    (2020)
  • J.G. Arnold et al.

    Large area hydrologic modeling and assessment part i: model development

    JAWRA J. Am. Water Resour. Assoc.

    (1998)
  • R. Arsenault et al.

    Continuous streamflow prediction in ungauged basins: long short-term memory neural networks clearly outperform traditional hydrological models

    Hydrol. Earth Syst. Sci.

    (2023)
  • M.A. Bannwarth

    Simulation of stream flow components in a mountainous catchment in northern Thailand with SWAT, using the ANSELM calibration approach

    Hydrol. Process.

    (2015)
  • T. Chai et al.

    Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature

    Geosci. Model Dev.

    (2014)
  • Z. Chen

    A transfer Learning-Based LSTM strategy for imputing Large-Scale consecutive missing data and its application in a water quality prediction system

    J. Hydrol.

    (2021)
  • X. Chen

    Comparison of deep learning models and a typical process-based model in glacio-hydrology simulation

    J. Hydrol.

    (2022)
  • S. Chen et al.

    Machine learning-based estimation of riverine nutrient concentrations and associated uncertainties caused by sampling frequencies

    PLoS One

    (2022)
  • CSSDC, 2020. Soil Science Database....
  • P. Das et al.

    Machine learning-based rainfall forecasting with multiple non-linear feature selection algorithms

    Water Resour. Manag.

    (2022)
  • A. de Lavenne et al.

    The transfR toolbox for transferring observed streamflow series to ungauged basins based on their hydrogeomorphology

    Environ. Model. Softw.

    (2023)
  • H.X. Do et al.

    The global streamflow indices and metadata archive (GSIM) – Part 1: the production of a daily streamflow archive and metadata

    Earth Syst. Sci. Data

    (2018)
  • L. Fang et al.

    Hybrid approach for flood susceptibility assessment in a flood-prone mountainous catchment in China

    J. Hydrol.

    (2022)
  • Z.-K. Feng

    Parallel cooperation search algorithm and artificial intelligence method for streamflow time series forecasting

    J. Hydrol.

    (2022)
  • D. Feng et al.

    Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy

    Water Resour. Res.

    (2022)
  • S.W. Fleming et al.

    Augmenting geophysical interpretation of data-driven operational water supply forecast modeling for a western US river using a hybrid machine learning approach

    J. Hydrol.

    (2021)
  • C. Fu et al.

    Investigations of uncertainty in SWAT hydrologic simulations: a case study of a Canadian Shield catchment

    Hydrol. Process.

    (2015)
  • S. Grimaldi et al.

    Continuous hydrologic modelling for design simulation in small and ungauged basins: a step forward and some tests for its practical use

    J. Hydrol.

    (2021)
  • L. Gudmundsson et al.

    Anthropogenic climate change detected in European renewable freshwater resources

    Nat. Clim. Chang.

    (2017)
  • Y. Guo et al.

    Regionalization of hydrological modeling for predicting streamflow in ungauged catchments: a comprehensive review

    WIREs Water

    (2020)
  • S. Hochreiter et al.

    Long short-term memory

    Neural Comput.

    (1997)
  • K. Irving

    A high-resolution streamflow and hydrological metrics dataset for ecological modeling using a regression model

    Sci. Data

    (2018)
  • Cited by (0)

    View full text