A scalable distributed parallel simulation tool for the SWAT model
Section snippets
Software availability
The source codes and tools developed in this research are freely available through the GNU general public license for the general public. They are hosted in GitHub and can be accessed through the following link: https://github.com/djzhang80/Spark-SWAT.
SWAT model
SWAT (Arnold and Fohrer, 2005; Arnold et al., 1998) is a semidistributed, watershed-scale hydrological model that was initially developed by the Agricultural Research Service of the United States Department of Agriculture to predict the impact of watershed management practices on water, sediments, nutrients, pesticides, and fecal bacterial yields in the agricultural landscapes of North America. Due to its distributed, physically based and open-access nature, it has been adapted and applied to
Test environment
A Spark cluster consisting of five virtual machines was established to test the performance of Spark-SWAT. These virtual machines are built on three physical servers by using VMware ESXi (version 4.1) software. There are 10 and 20 physical and logical cores for each physical server, respectively. The clock speed, random-access memory (RAM) and disk storage of each physical server are 2.2 GHz, 128 GB and 5 TB, respectively. Each of these physical servers runs VMware ESXi, which is an
Model result comparisons
In theory, the model outputs of undivided and split models should be identical. However, the model output comparison in this study shows that the stream flow, sediment and other chemicals are not identical (Fig. 6). Fig. 6a shows the simulated stream flows of the undivided and split models and errors between them (for clarity, only a half-year of simulated stream flow was plotted). As seen in this plot, for most cases, the simulated stream flows of these two models are identical; only a small
Conclusions
In this paper, we proposed a scheme for SWAT parallelization by splitting a watershed model into multiple subbasin models and orchestrating parallel simulations according to the watershed route network. We implemented a parallel-computing tool for SWAT (Spark-SWAT) by using an open-source general-purpose distributed cluster-computing framework (Spark) according to the proposed scheme. Based on synthetic models, Spark-SWAT was tested and evaluated with a small Spark cluster consisting of five
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was financially supported by the Natural Science Foundation of Fujian Province [grant number 2020J01779], the Science and Technology Project of Xiamen [grant number 3502Z20183056], and the Science and Technology Climbing Program of Xiamen University of Technology [grant number XPDKT19014].
References (40)
- et al.
A continental-scale hydrology and water quality model for Europe: calibration and uncertainty of a high-resolution large-scale SWAT model
J. Hydrol.
(2015) High-performance computing tools for the integrated assessment and modelling of social–ecological systems
Environ. Model. Software
(2013)- et al.
Calibration of SWAT models using the cloud
Environ. Model. Software
(2014) - et al.
SWAT-CS: revision and testing of SWAT for Canadian Shield catchments
J. Hydrol.
(2014) - et al.
Execution of compute-intensive applications into parallel machines
Inf. Sci.
(1997) - et al.
Global sensitivity analysis for large-scale socio-hydrological models using Hadoop
Environ. Model. Software
(2015) - et al.
Using a parallelized MCMC algorithm in R to identify appropriate likelihood functions for SWAT
Environ. Model. Software
(2013) - et al.
Sensitivity analysis in watershed model using SUFI-2 algorithm
Procedia Eng
(2016) - et al.
OpenMP-accelerated SWAT simulation using Intel C and FORTRAN compilers: development and benchmark
Comput. Geosci.
(2015) - et al.
A layered approach to parallel computing for spatially distributed hydrological modeling
Environ. Model. Software
(2014)
A layered approach to parallel computing for spatially distributed hydrological modeling
Environ. Model. Software
Estimation of theoretical maximum speedup ratio for parallel computing of grid-based distributed hydrological models
Comput. Geosci.
A two-level parallelization method for distributed hydrological models
Environ. Model. Software
Numerical assessment of metamodelling strategies in computationally intensive optimization
Environ. Model. Software
A parallelization framework for calibration of hydrological models
Environ. Model. Software
Autocalibration experiments using machine learning and high performance computing
Environ. Model. Software
A High-performance temporal-spatial discretization method for the parallel computing of river basins
Comput. Geosci.
Parallelization of a hydrological model using the message passing interface
Environ. Model. Software
Automating calibration, sensitivity and uncertainty analysis of complex models using the R package Flexible Modeling Environment (FME): SWAT as an example
Environ. Model. Software
SWAT-DayCent coupler: an integration tool for simultaneous hydro-biogeochemical modeling using SWAT and DayCent
Environ. Model. Software
Cited by (0)
- 1
Note: Lin and Zhang contributed equally to this work.