A deep sequence multi-distribution adversarial model for bearing abnormal condition detection

Highlights

• Proposed model that use normal data training are more suitable for practical applications.

• Apply adversarial learning and LSTM structure to anomaly detection.

• Multi-distribution features are utilized to construct an anomaly index.

• DSMDA can prove sufficient robustness for anomalies detection of low SNR signals.

Abstract

Time series anomaly detection is one of the key challenges in the field of condition monitoring. Many anomaly detection methods are inefficient and easy to lose effective information due to manual features extracting. Deep learning-based methods can solve the problem effectively, but the detection accuracy is still not satisfactory. In addition, most of the methods cannot take the time-ordered specialty into account which is significant for time-series-based anomaly detection. To address these issues, a novel method named deep sequence multi-distribution adversarial model (DSMDA) is proposed to improve the accuracy of anomaly detection in bearing condition monitoring. The proposed model utilizes the data reconstruction capability of the Variational Autoencoder (VAE) under the framework of generative adversarial network (GAN) to make full use of information. The feedforward neural network layer of VAE is replaced by the long-term and short-term memory (LSTM) layer, which uses the forgetting mechanism of LSTM to effectively avoid the false alarms caused by the excessive influence of the old sequences. Additionally, the fault-attention abnormal state index can be constructed by the real-time spatial distribution and latent spatial distribution features learned by the double discriminators. To verify the effectiveness of the proposed approach, experiments on two public datasets are carried out with only healthy data in training stage that is more suitable for practical industrial applications. The results show that the proposed method is superior to GANomaly and other advanced methods. Furthermore, the 2-D visualization results can indicate the level of fault while the last feature space of the two discriminators is combined and embedded into the visualization, and the fault-attention abnormal state indictor constructed on these features can indicate abnormalities well.

Introduction

With the rapid development of industrial technology, some mechanical equipment has become more developed in precision and automation. Due to the extreme high temperature, heavy load and other harsh operating environmental problems, the maintenance and repair of the close contact components becomes more difficult. In particular, the operation of the entire mechanical equipment will be affected once the key component fails, such as a gear or bearing, which may cause lower production quality and even catastrophic damages. Therefore, to avoid the serious accident, especially in the early stages of machine performance degradation, a predictive maintenance on the key components of these machines is of vital importance in prognosis and health management (PHM) [1]. As the first step of predictive maintenance, anomaly detection is the basic of PHM while it is still one of the most neglected aspects. Accurate abnormalities detection will help promote PHM work, improve the efficiency and quality of machine maintenance.

Abnormal detection has been applied in various fields, such as gearbox temperature abnormal detection [2], abnormal frames detection in CCTV video [3], soft-tissue lesions detection in clinical medicine [4], and neuropathological changes of Alzheimer's disease [5]. Anomaly is usually defined as the point where the system behavior is significantly different from the previous normal state in certain time steps [6]. In order to detect anomalies, the time step of these possible anomalies needs to be identify. Generally speaking, the state of mechanical equipment at the previous moment will affect the next moment, which is always closely related to the time factor. Therefore, it is challenging for anomaly detection model to capture different features of time series and detect anomalies effectively. Currently, mechanical equipment anomaly detection methods are mainly divided into three categories, namely, the boundary-based method, the probability density estimation-based method and the deep learning-based method.

For boundary-based anomaly detection methods, a structure for the training dataset need to be set up to create the boundary, thus determine the class membership of the unknown by its position relative to the boundary. In general, this type of method separates feature extraction and model construction. In terms of feature extraction, some classical features have been extracted, including time–frequency feature [7], wavelet feature [8] and envelope spectrum feature [9]. However, these features have a common weakness that they a lot of noise and fluctuations are brought in, which affect the detection accuracy. In order to improve the performance, Cheng et al. [10] decomposed the vibration signal into a finite number of intrinsic-mode functions and established an AR model for each IMF component. The corresponding autoregressive parameters and residual variance were taken as fault feature vectors, after which these vectors were picked as the input of support vector machine (SVM) classifier to classify the working state of gears. Saari et al. [11] utilized one-class of support vector machines (OCSVM) to detect wind turbine bearing faults. Subsequently, Liu et al. [12] proposed a new semi-supervised support vector data description with negative sample (NSVDD) fault detection. Afrooz et al. [13] extracted kurtosis and Non-Gaussianity Score (NGS) to develop SVM anomaly detection algorithm, which makes the detection converge to the optimal value with high accuracy. Moreover, as one of the most representative methods, fuzzy C-means (FCM) was also applied to anomaly detection [14], [15], [16], [17]. Liu et al. [18]combined it with correlation function fusion algorithm to solve the abnormal detection of rail vehicle gearbox. Xi et al. [19] introduced the artificial fish-swarm algorithm (AAFSA) into FCM to reduce the sensitivity of initial value selection and avoid the results in a local extreme. However, boundary-based anomaly detection methods are usually difficult to extract features adaptively, while the manual creation of useful features requires a lot of engineering work and expert experience. In addition, all types of data need to be equally provided during the training. The accuracy of evaluation results still needs to be further studied while failure data is underrepresented.

Probability density estimation-based anomaly detection methods are modeled according to the density of valid data in the training process. The detected data that falls into the low-density area are likely to be rejected when input into the model. Yu et al. [20]carried out the nonlinear dimensionality reduction of the extracted features by the local preserving projection method, and realized the performance degradation evaluation of the bearing by using Gaussian mixture models (GMM). Angle-based outlier detection (ABOD) method [21] and feature bagging (FB) method [22] processed data by considering the correlation of variables. Unlike boundary-based methods that only use small sample training, the limitation of probability density estimation-based methods requires the dataset for training to be large enough. Besides, the data with time series characteristics usually has non-stationary particularities, and its fluctuation range changes with time [23]. Probability density estimation-based methods failed to detect outliers on the deep features of time series signals since the time correlation is not taken into account.

In recent years, deep learning has been successfully applied to the automatic extraction and recognition of early fault features [24], [25]. The deep feature extraction capability and the powerful discriminant ability with rich information make them stand out in the detection of sequence anomalies. For instance, the convolutional neural network (CNN) has been applied to fault diagnosis due to its powerful feature extraction ability [26]. Piekarski et al. [27] presented a CNN architecture with a determined pre-trained model and fitted densely connected classifier for anomaly detection. Li et al. [28] provided a CNN with atrous convolution for the adaptive fusion of multi-source data, and applied it to fault diagnosis of an industrial fan system and a centrifugal pump. In addition, the autoencoder (AE) learns the feature representation of high-dimensional input through the encoder, then the decoder reconstructs low-dimensional features. A higher reconstruction error can be utilized to indicate anomalies, when non-anomalous data only made use of AE training [29]. Subsequently, variational autoencoder (VAE) is proposed to use variational inference to model the observed latent probability distribution [30]. Taking into account the correlation of sequence data before and after, sequence modeling methods such as recurrent neural network (RNN) [31] and long short-term memory (LSTM) [32] networks are often combined with AE and VAE to capture the machines behavior that change according to usage and external factors. Although effective in some cases, only the comparison of simple reconstruction errors is used to detect anomalies, which may not be sufficient considering the highly dynamic characteristics of the system. How to capture the dynamic information of monitoring data over time remains to be further explored.

Recently, a generative adversarial network (GAN) [33] has been put forward to provide a new perspective for anomaly detection, which was initially applied to the image generation. Given a training set, this model learns to generate new data with the same statistics as the training set. However, the framework of GAN has been improved for the purpose of meeting higher application requirements. AnoGAN [34], EGBAD [35], GANomaly [36] appeared one after another, in which AnoGAN is the first GAN framework for anomaly detection. Then, in order to improve the computational efficiency of the network, EGBAD introduced BiGAN structure into the field of anomaly detection, and tried to overcome the drawbacks of the AnoGAN according to the research of Donahue et al. [37]and Dumoulin et al. [38]. The lately proposed GANomaly combines AE with GAN, which not only trains the generator network to learn the manifold of normal samples, but also trains the AE to learn how to effectively encode images with its latent representation. Its anomaly evaluation index and execution speed exceeded the performance of EGBAD. These GAN-based methods only use normal data for training, and then identify anomalies by the distribution difference between normal data and abnormal data. Although GAN-based methods have achieved initial breakthrough in the field of anomaly detection, the detection precision is not satisfactory, and the time characteristics of the mechanical signal are not considered either. Up to now there is still no effective solution to enhance model robustness to signal noise and maintain the GAN training stability. The above research inspires us to further explore the effectiveness of GAN in industrial fault detection, especially when there is less abnormal data in the actual industrial field.

The purpose of this study is to propose an anomaly detection method that is able to overcome the shortcomings mentioned above and provide a system with higher sensitivity in anomaly detection at the same time. The proposed method detects anomalies from the normal model in the observed data. The encoder module and decoder module of VAE are used as generator network, whose original feedforward neural network layer is replaced by LSTM units. Due to the mapping ability of encoder and the identification ability of discriminator, the joint training of encoder, generator and discriminator are helpful to distinguish the health and fault state of machines. The contributions of this paper can be summarized as follows:

• A DSMDA anomaly detection method in industrial field is proposed to create normal data model based on anomaly detection methods, since training strategies that only use normal data training are more suitable for practical applications.

• The proposed model utilizes the advantage of time series data processing of LSTM network and the latent distribution mapping ability of VAE network to improve adaptive feature extraction capability and detection accuracy.

• Two discriminators are used to learn the latent feature distribution and real-time spatial distribution of monitoring data, and then the PCA method is used to find out the low-dimensional manifold of these two parts to construct an anomaly index, which makes the model more sensitive to early fault identification.

• In the industrial monitoring system, the acquired data might be noised, while the proposed method can be proved of sufficient robustness for anomalies detection of low signal-to-noise ratio (SNR) signals.

The remaining chapters are arranged as follows. The second section introduces the basic principle of the proposed method. The third section describes the basic framework of DSMDA model. Experimental results of the proposed method on two real datasets are shown and analyzed in the fourth section. Finally, the fifth section summarizes the whole paper and puts forward the future prospects.