A detection method for reservoir waterbodies vector data based on EGADS

2.1 Background

Anomaly detection can be simply described as follows: given a set of N data/objects, the process of finding data/objects that are significantly abnormal, isolated, or inconsistent with the general rules compared with the general data/objects [31]. There are three kinds of anomalies in time series: point anomalies, sequence anomalies, and pattern anomalies. Time series-oriented anomaly detection usually needs to achieve two things: time series analysis and effective abnormal data detection [31,32].

A time series is a column of ordered data recorded in a chronological order, which can be represented as the set X=(x1,x2,…,xn)T. The result of water vector extraction for long time series observation is a form of time series. Time series analysis is used to express the trend of a time series through the mathematical theory and method and predict its trend [33]. Time series modeling reveals the development law of a time series from the perspective of sequence autocorrelation, which is an important method to explore the statistical characteristics (trend, seasonality, periodicity, stationarity, and autocorrelation) of a time series. It is often used in hydrological forecasting, weather forecasting, market economic forecasting, and other fields [31,34,35].

Outliers are objects in the data set that deviate from most of the data. They are generated not because of random errors, but because of different mechanisms [36]. In practice, outliers often contain valuable information, and mining the hidden value in them is helpful for analyzing decision tasks. According to the existing studies, anomaly detection models can be divided into the following categories: the statistical-based methods [37,38] (establishing the data distribution model and screening anomalies through deviations); the distance-based method [39,40] (distance calculation of objects in data set through distance function and exception screening using a threshold); the density-based methods [41,42] (introducing local density and judging outlier degree of data by weight); and the method based on machine learning [43,44,45] (algorithms mainly based on the artificial neural network and the support vector machine). Studies have shown that the distance-based method and the machine learning algorithm tend to have high time complexity and low efficiency. Some literature state [31,32] that the time series anomaly detection based on prediction is the most concise and intuitive anomaly detection method, but this method relies on the prediction model and a reasonable threshold.

The extensible generic anomaly detection system (EGADS), which was designed by Yahoo, is an open-source extensible framework for automatic anomaly detection of massive time series data [46]. It consists of two main modules: the time series modeling module (TMM) and the anomaly detection module (ADM). In the EGADS plug-in framework, time series model and anomaly detection model can be inserted flexibly according to the specific application and data characteristics. As a java package, it can be easily integrated into the existing monitoring infrastructure. In this study, we used the EGADS framework as support to explore the anomaly mining method for vector data of reservoir waterbodies.

2.2 Vector detection of reservoir waterbodies based on EGADS

Facing abnormal phenomena, such as false and missed extractions and water level alarm in automatic water extraction, this study used the EGADS framework combined with the geometric characteristics of the water vector to design a detection method for vector data of reservoir waterbodies. First, rough inspection and processing are carried out on the water vector to find a preliminary solution to the most significant and obvious quality problems such as sawtooth, void, and irregular property structures and to obtain the initial area data for anomaly detection. Then, combining the EGADS framework plug-in outlier detection method, time series modeling is carried out on the area data of vector water to analyze and find the change rules of water data. The reliability of data is tested, based on this, using different anomaly detection models. Finally, abnormal data are found, and the alarm result is obtained. The specific research ideas are shown in Figure 1.

Figure 1

Detection flow chart for polygonal vector data of waterbodies in EGADS.

3.1 Data

The Water conservancy Gaofen satellite product service and distribution subsystem is one of the operational subsystems of the Gaofen Water conservancy Remote sensing application demonstration system (phase 1), which is developed by the Ministry of Water Resources. The system supports the management, query and sharing of domestic Gaofen satellite images (GF-1, GF-2, etc.), as well as the extraction, management, and query of water vectors nationwide. Based on the water body extraction algorithm with the water body index and automatic threshold, it can automatically extract water nationwide based on domestic Gaofen series of remote sensing images and provide near real-time nationwide water conservancy production and query.

The experimental data are obtained from the national water vector data automatically extracted by this system. Since the series of Gaofen satellites was launched as early as April 2013 and the study started at the end of 2017, we chose a data time range covering the full four years: November 2013 to November 2017. Our study focused on the polygonal vector data of reservoir waterbodies. First-level large reservoirs are the largest in terms of storage capacity, water area, and project scale, and the social and economic benefits generated are relatively more important. Due to the larger coverage, they are more likely to be blocked and affected by the cloud, vegetation, mountain shadow, and other objects, resulting in inaccurate extraction results, which are more suitable for research. Therefore, in this study, the Miyun reservoir (located in Beijing), a first-level large reservoir, was selected to construct the detection method for the waterbody vector data, to test the detection effect of different models provided by EGADS in the actual sequence, and to obtain a suitable detection method for water reservoir vector data.

3.2 Data preprocessing

The data from the Miyun Reservoir from October 2, 2013, to October 5, 2017 were preprocessed, and we obtained 160 datasets, including the attribute information of reservoir name, reservoir type, time, and area. We plotted the scatter plot for the water area time series as shown in Figure 2.

Figure 2

Scatter plot of area-time change of Miyun Reservoir from October 2013 to October 2017.

The area data for the Miyun reservoir (Figure 2) range from [20,100] km², and there were obvious deviations from the surrounding points. So, anomaly detection of the waterbody vector data is very necessary. Waterbody vectors with areas of approximately 20, 30, 40, 50, 60, 70, 80, 90, and 100 km² were selected for observation, and typical data are shown in Figure 3.

Figure 3

Water vector data for Miyun Reservoir (20–100 km²). (a) 2014-9-21, (b) 2013-12-12, (c) 2014-1-6, (d) 2015-8-21, (e) 2013-10-23, (f) 2014-9-10, (g) 2013-10-2, (h) 2017-3-3, (i) 2014-5-1.

There are two main reasons for the variation of the extraction areas of reservoir waterbody: (1) there is cloud cover in the remote sensing image or the image does not cover the reservoir completely (Figure 3(a)–(e) and (i)). We need to identify and remove this type of abnormal data; (2) drought, flood, agricultural irrigation, and artificial storage cause the amount of reservoir water to increase or decrease, resulting in changes in the water area (Figure 3(f)–(h)). We need to detect the abnormal changes in the reservoir waterbody under these circumstances to improve water management, dispatching, and emergency response.

4.1 Time series modeling

We used the area time series of the Miyun Reservoir for modeling using nine time series models of TMM. The results are shown in a scatter diagram (Figure 4) and sequence chart (Figure 5). The automatic forecast model is the optimal model chosen by EGADS from nine models according to the minimum deviation rule. However, this method has a single selection criterion, and it may not be applicable to the analysis of the waterbody vector data.

Figure 4

Scatter diagram of time series modeling results of Miyun Reservoir from October.

Figure 5

Sequence chart of time series modeling results of Miyun Reservoir from October 2013 to October 2017.

Figure 4 shows the scatter diagram of the results of time series modeling for each model; EGADS judged that the reservoir automatic prediction model is the Olympic model. As only the rate of change of the reservoir area was considered in our experiment, the result of the multiple linear regression model is consistent with those of the regression model.

Figure 5 shows the characteristic curves of time series of sample and modeling results. The fitting effect of each model has the following characteristics: (1) the fitting curve of the Olympic model (Figure 5(a)) in the early stages of the data over a short time interval (before December 2015) agrees well with the actual area curve, but the fitting effect is ordinary when the time interval of later data is large. At the same time, the fitting curve of the model can reflect the seasonal variation of the performance data. (2) The fitting curves of the regression model, multivariate linear regression model, and polynomial regression model (Figure 5(b)–(d)) are smooth curves, which are very inconsistent with the actual area curve. (4) The fitting curve of the first exponential smoothing model (Figure 5(e)) is similar to that of the Olympic model as a whole. However, when the time interval is large, the sudden change in the predicted value is more obvious than that of the Olympic model. (5) The fitting curve of the double exponential smoothing model (Figure 5(f)) is very consistent with the actual curve, and it reveals the overfitting phenomenon. (6) The fitting effect of the triple exponential smoothing model is worse than that of the simple exponential smoothing model, and the sudden change in the predicted value is obvious (the difference between the predicted value and the observed value is significant around April 2015). (7) The fitting curve of the moving average model in Figure 5(h) is similar to that of the first exponential smoothing model; however, the sudden change slows down in February 2015, February 2015, and April 2017. The curve also reflects the seasonal variation in the data. (8) Compared with the moving average model, the sudden change in the predicted data slows down further in the fitted curve of the weighted moving average model shown in Figure 5(i) and reflects the seasonal variation. (9) The automatic forecast model selected by the system is the Olympic model. In summary, the time series model suitable for the Miyun Reservoir should be selected from the Olympic model and the weighted moving average model.

After time series modeling, the prediction results of each model were evaluated. These five metrics all reflect the fitting ability of the model. Smaller absolute values of the results indicate a higher fitting ability of the model, but they may correspond to the overfitting phenomenon. By using the five metrics provided by TMM, we calculated the metric results of each model as presented in Table 4.

The fitness of each model, from high to low, is as follows: double exponential smoothing model, Olympic model and automatic forecast model, Regression model and polynomial regression model, polynomial regression model, moving average model, simple exponential smoothing model, triple exponential smoothing model, and weighted moving average model.

In our data source, there are some abnormal vector data caused by clouds, incomplete image coverage of the study area, or other reasons. They deviate greatly from the general law, thus disturbing the trend of normal time series. Therefore, we need to further evaluate the time series model. The dispersion degree of the model results was analyzed by calculating for the indicators presented in Table 5. Concentrated model results show that the reservoir water storage was closer to a stable actual state, indicating the fitness of the model.

The dispersion degree of the corrected results of each model, from smallest to largest, is as follows: polynomial regression model, regression model and multiple linear regression model, Olympic model and automatic forecast model, simple exponential smoothing model, moving average model, weighted moving average model, triple exponential smoothing model, double exponential smoothing model, and original data.

Considering the fitting curve characteristics, fitting degree, and the dispersion degree of the corrected results, the Olympic model is the most suitable time series model for the Miyun Reservoir. Meanwhile, the system’s automatic prediction model was also the Olympic model, but the system needs to run all the models before selecting the best model. Considering the computer performance and resource consumption, it is appropriate to directly adopt the Olympic model.

4.2 Outlier detection

After using the Olympic model for time series modeling, we used the four kinds of anomaly detection models to detect outliers for the Miyun Reservoir area vector data. We marked the outliers with a different color in the scatter plots (Figure 6). The occurrence date of the abnormal area data is presented in Table 6.

Figure 6

Abnormal area data for Miyun Reservoir from October 2013 to October 2017.

The characteristics and detection results of each model were analyzed. The purpose of the density-based spatial clustering of applications with noise (DBScan) model is to find outliers that are free from clusters; it only found near-zero abnormal values in our experiments. The adaptive kernel density change-point detection (AKDCPD) model considers the change points testing problem; it mistakenly regarded the area data on June 24, 2014, as an outlier and failed to detect most of the other abnormal values. The extreme low-density (ELD) model regards data in low-density regions as outliers, and it had the weakest anomaly detection effect in our experiment; only the smallest area was detected. The Kσ model effectively identified anomalies in high or low area values (all anomalies detected by other models were included). However, a few abnormal values had not been detected, and the reasons are explained in Section 4.4.2. In comparison, the optimal anomaly detection model for the Miyun Reservoir is Kσ model.

The aforementioned experiments show that in the framework of EGADS, combining the Olympic model and Kσ model into a single framework can effectively detect abnormal values in the area vector data for the Miyun Reservoir.

4.3 Application to other types of reservoirs

To further explore whether the optimal time series model and anomaly detection model are affected by reservoir scale in the framework of EGADS, we selected the Xishanwan Reservoir (second-level large; 99 datasets from November 8, 2013, to April 30, 2017), Dashuiyu Reservoir (third-level large; 157 datasets from October 2, 2013, to April 26, 2017), Xiagou Reservoir (fourth-level large; 83 datasets from May 2, 2014, to June 14, 2017), and Gulbeitou Reservoir (fifth-level large; 63 datasets from May 1, 2014, to May 13, 2017) for analysis and obtained the optimum models for waterbody vector data for each reservoir, which are presented as shown in Table 7.

Experiments show that although the fitting effect of each time series model is slightly different for the area data of reservoirs at different scales. Among the nine time series models, the Olympic model is the best in modeling performance for different scale reservoirs, in terms of seasonal variation, a high degree of fitting, and a small degree of dispersion. The Kσ model can effectively detect most of the abnormal values for all scales of reservoirs. Therefore, for reservoirs of different scales, the combination of the Olympic model and Kσ model in EGADS is suitable for detection of the abnormal values in the reservoir waterbody vector data.

4.4 Accuracy evaluation

4.4.2 Model evaluation and analysis

We extracted the Miyun Reservoir vector data for the sample and interpreted it manually: 106 are qualified vector data and 54 are abnormal vector data. Comparing the results of detection with those of manual interpretation, we got TY = 34, FN = 20, TN = 104, FY = 2 in the outliers of the Miyun Reservoir detected by the Olympic model and Kσ model. By substituting them into formula (1) and formula (2), sensitivity = 94.44% and specificity = 83.87% were derived. The results show that for the Miyun Reservoir area vector data, the method exhibited a 94.44% accuracy rate for abnormal data detection, and the vector data accuracy is improved from 66.25%, before the inspection, to 83.87%. The missed abnormal vector data (FN) and false abnormal vector data (FY) of the reservoir are shown in Figure 7 and 8, respectively.

Figure 7

Missed alarm vector data. (a) 2013-10-23, (b) 2013-11-7, (c) 2014-1-7, (d) 2014-3-27, (e) 2014-6-28, (f) 2014-9-17, (g) 2014-11-14, (h) 2015-8-21, (i) 2016-1-30, (j) 2016-1-31, (k) 2016-2-25, (l) 2016-3-8, (m) 2016-8-15, (n) 2016-8-26, (o) 2016-9-30, (p) 2016-11-25, (q) 2016-12-8, (r) 2016-12-16, (s) 2017-1-3, (t) 2017-4-18.

Figure 8

False alarm vector data. (a) 2014-2-14, (b) 2014-5-1.

From Figures 2 and 7, the anomalous vector data of missing alarm can be divided into three categories: (1) when reservoir imaging was less affected by cloud coverage or the limitation of the width in the remote sensing image, the extracted vector changed slightly (Figure 7(a)–(c) and (e)–(h)). This category of abnormal vector data can be detected by judging the inclusion relation of the frame vector of the original image and the detected cloud with the general water vector or the vector centroid. (2) The extracted vector data were divided into many fragmentary vectors, but their total area did not change significantly (Figure 7(q) and (s)). This category of abnormal vector data can be detected by judging the size of each vector area, number, and offset of vector centroids. (3) The approximate time and concentrated values of abnormal data result in the larger or smaller fitting value in time series modeling, which lead to some abnormal data not being detected (Figure 7(i)–(p), (r) and (t)). This category of abnormal vector data can be detected by judging the offset of vector centroids. (4) Model defects caused part of abnormal vector data to be undetected (Figure 7(d) and (g)), which accounts for a small proportion of the total anomaly data. This category of abnormal vector data can be detected by judging the size of each vector area and offset of vector centroids.

From Figure 8, we found that the false alarm anomaly data were vector data with larger areas: (1) the waterbody vectors of the Miyun Reservoir (Figure 8(a)) could be extracted from two images from the same date, but one was inaccurate because of the excessive cloudiness in the image, which affected the area of the waterbody vectors after fusion. (2) The water vector (Figure 8(b)) was disturbed by thin clouds in the image, which resulted in a larger extraction result, but we could not rule out the increase in the water surface. To obtain more accurate anomaly detection results, we should also classify these two types of anomaly data as anomalies to some extent.

Accuracy evaluation results show that EGADS can detect 94.44% accuracy rate of reservoir vector data anomalies and improve the vector data accuracy from 66.25 to 83.87%. The errors in terms of undetected abnormal data are small, which can be determined by vector space position or manual detection. Therefore, the detection method had passed the qualitative evaluation and can accurately detect most of the reservoir vector data anomalies and combining it with vector space location inspection can further improve the detection accuracy.