A novel deep neural network architecture for real-time water demand forecasting

Highlights

• A DNN structure based on GRU and k-means achieves a remarkable performance for WDF.

• With help of k-means method, WDF accuracy is preserved with low cost of complexity.

• Extending the data linearly reduces the prediction error at the extreme points.

Abstract

Short-term water demand forecasting (StWDF) is the foundation stone in the derivation of an optimal plan for controlling water supply systems. Deep learning (DL) approaches provide the most accurate solutions for this purpose. However, they suffer from complexity problem due to the massive number of parameters, in addition to the high forecasting error at the extreme points. In this work, an effective method to alleviate the error at these points is proposed. It is based on extending the data by inserting virtual data within the actual data to relieve the nonlinearity around them. To our knowledge, this is the first work that considers the problem related to the extreme points. Moreover, the water demand forecasting model proposed in this work is a novel DL model with relatively low complexity. The basic model uses the gated recurrent unit (GRU) to handle the sequential relationship in the historical demand data, while an unsupervised classification method, k-means, is introduced for the creation of new features to enhance the prediction accuracy with less number of parameters. Real data obtained from two different water plants in China are used to train and verify the model proposed. The prediction results and the comparison with the state-of-the-art illustrate that the method proposed reduces the complexity of the model six times of what achieved in the literature while conserving the same accuracy. Furthermore, it is found that extending the data set significantly reduces the error by about 30%. However, it increases the training time.

Introduction

Water scarcity has become a threat to humankind in recent decades. Many efforts in all possible directions are being made to compensate for this growing problem (Northey et al., 2016, González-Zeas et al., 2019). The major reliable strategies for that include water treatment (Zinatloo-Ajabshir et al., 2020), water desalination, and optimization of water management systems. Nanotechnology is the most powerful technology employed for water treatment, where researchers have done impressive work (Zinatloo-Ajabshir et al., 2020, Zinatloo-Ajabshir et al., 2017, Moshtaghi et al., 2016). On the other hand, StWDF is the foundation stone of the optimization of water management systems. Therefore, numerous researchers have directed their efforts towards this problem. Nowadays, deep learning (DL) is the most dominant approach, which provides the most promising solutions to a myriad of critical problems. To mention a few examples, Zhou (2020) has proposed deep learning method for forecasting the water quality in the presence of missing data. Friedel et al. (2020) have compared four machine learning methods to predict groundwater redox status in the agriculturally dominated regions of New Zealand. Wang et al. (2020) have used a deep belief network to forecast the depth of snow over in Alaska. Flood prediction is another crucial problem that utilizes deep learning approaches. An Encoder-Decoder based on Long Short-term-memory (LSTM) is proposed by Kao et al. (2020) for multi-step-ahead flood forecasting. While flood susceptibility modeling using deep learning is investigated in many studies, such as (Pham et al., 2021, Bui et al., 2020). LSTM is also used by Ni et al. (2020) for streamflow and rainfall prediction where two LSTM-based models are built, one combined wavelet network with LSTM and the other combined convolutional network with LSTM to achieve better performance.

StWDF is one of these problems that benefit most from DL to develop effective methods. However, some challenges might impede the success of DL-based solutions. Model complexity, the accumulative error when forecasting multi-steps, and the significant prediction error at the extreme points are some of these challenges that still need investigation. Model complexity and the size of the model in particular, become serious constraints when using federated learning approach, and bring some extra challenges to knowledge transformation technologies. Model complexity includes two types, time complexity and space complexity. Time complexity is brought on by the time required to train the model and by the data size required for training. The place complexity problem usually comes to the surface when the model has a massive number of parameters, which increases the model size, as well as the training time.

The second challenge is the accumulative error problem, which affects the multi-step prediction of water demand. When relying on the historical data of water demand for StWDF, the predicted values are involved in predicting the following values. Thus, the prediction error is compounded by the use of inaccurate values of the predicted water demand.

The third challenge is the significant error at the extreme points, which occur as a normal reflection of the nonlinearity of daily water demand. These points are recognized as periods where water demand is dramatically different from the average demand in the adjacent periods, making it difficult for the model to approximate the demand at these points. A few examples of extreme points are illustrated in Fig. 7, Fig. 8. The error at these points is usually unacceptable and may lead to severe problems in the distribution system.

In literature, complexity problem was not a critical issue when statistical methods such as Auto-regression integrated moving average (ARIMA) method and the seasonal version of it (SARIMA), in addition to Markov Chain, are used for StWDF (Pandey et al., 2021). However, their accuracy is not sufficiently satisfactory. Caiado (2010) has achieved $11\%$ of mean square percentage error for one-day prediction and $13.2\%$ for 7 -days ahead prediction by combining SARIMA with the generalized autoregressive conditional heteroscedasticity method for daily WDF. Arandia et al. (2016) also have achieved $4.21\%$ of mean absolute percentage error (MAPE) for 15-min prediction of water demand in Dublin Spain by involving some data assimilation technique to improve the performance of SRIMA method. However, their proposal does not work well with hourly prediction, where the best MAPE they have got is $38.12\%$. Brentan et al. (2017) have built their prediction model based on SVR and Fourier methods for on-line prediction of hourly water demand. Their model achieves MAPE of $3.41\%$for one-step prediction.

With the expansion in the application of machine learning (Zhou et al., 2020, He et al., 2019) and artificial neural networks (ANNs) (Salloom et al., 2020, Yu et al., 2020, He et al., 2017), several studies have proved that machine learning methods outdo the stochastic and the probabilistic models for WDF. Gagliardi et al. (2017) has proved the ANNs overcome Markov Chain-based models in terms of forecasting accuracy. Herrera et al. (2010) and Bai et al. (2015) have proved the efficiency of support vector regression method for hourly WDF. Guo et al. (2018) has compared statistical methods and conventional ANNs with deep learning method and proved that the DL methods give more accurate results when predicting water demand for short-horizon.

In fact, most researchers focus on improving prediction accuracy without paying too much attention to model complexity. Furthermore, a new trend that exacerbates this problem has started looming on the horizon recently, where some researchers comprise many machine-learning models in one system. Then, they chose a different one for different prediction periods based on probabilistic methods. Ambrosio et al. (2019) have used a combination of multilayer perceptron, SVM, ELM, random forests and adaptive neural fuzzy inference systems for hourly WDF. Antunes et al. (2018) have investigated combining SVR, ANN, k-nearest-neighbours, and random forest regression for real-time WDF. This strategy is meant to use the most accurate model in the most suitable prediction period. However, it requires a massive amount of computations and memory to save the parameters of all models and system configuration setting.

The error at extreme points also contributes to the worsening of the prediction accuracy; however, It has not attracted researchers’ attention. Guo et al. (2018) are the first to point out this problem in their work, but they have not provided any solution.

The accumulative error problem may occur when predicting several steps ahead based on the historical data of water demand. Some researchers tried to solve this problem individually by building a new neural network model and train it to make the predicted values approach the real ones. Obviously, this method increases the parameter of the system dramatically.

In fact, the accumulative error problem can be mitigated by involving several factors besides the historical data (Papageorgiou et al., 2015, Kley-Holsteg and Ziel, 2020), so the impact of the predicted values in the input can be reduced efficiently. Many factors influence water demand level (Dias et al., 2018), but only factors such as meteorological conditions and day type, which have weekly or daily distinguishable changes, have real impacts on StWDF (Romano and Kapelan, 2014). However, managers still rely on the historical data of water demand for StWDF, and ignore the other possible factors because of the difficulties of gathering data about them in real time, particularly in the 15-min interval. Moreover, some available information about some factors, such as meteorological information, are not sufficiently accurate for short time prediction (Rayner et al., 2005), making them unreliable.

In this work, we propose a novel DL model for StWDF. It is built based on the gated recurrent unit (GRU), and unsupervised classification method k-means. Involving data classification as a prior step has two major benefits, (i) it helps with creating new features to compensate for the leak of reliable features, which reflects positively on the prediction accuracy and the accumulative error. (ii) It creates a relationship between data of different days, which an ANN can be easily approximated with a small number of parameters, which in turn enhances the space complexity of the models.

Additionally, we investigate using a novel technique to alleviate the nonlinearity at the extreme points and reduce the error. It depends on inserting virtual data between the actual data so that the nonlinearity at these points is drastically declined.

The contribution of this paper can be summarized as follows:

• A novel DL model that provides a high prediction accuracy for both one step (15-min) and multi-step (96 steps ahead) prediction is proposed. It is built based on GRU neural network, supported by an unsupervised classification step to enhance the accuracy.

• A new technique for mitigating the prediction error at extreme points is proposed and investigated.

• The accumulative error problem, which occurs in multi-step prediction, is mitigated by means of classification step, which establishes new features to rely on in the prediction.

• A comparison with the state-of-the-art is carried out to show the effectiveness of our proposed methods in terms of accuracy and model complexity.

The rest of this paper is organized as follows: Section 2 describe the equipment used to carry out this research. Section 3 explains the research methodology and the methods proposed in this work. Section 4 clarifies the results of this research, including the structure of the prediction model and the evaluation results. While Section 5 provide an intensive discussion to illustrate the underlying cause of these results. Section 6 illustrates the significance of the results. Section 7 includes the conclusion and the planned future works.