Elsevier

Information Sciences

Volume 562, July 2021, Pages 13-27
Information Sciences

Deep model based on mode elimination and Fisher criterion combined with self-organizing map for visual multimodal chemical process monitoring

https://doi.org/10.1016/j.ins.2021.01.036Get rights and content

Abstract

Multimode feature is widely adopted in complex continuous chemical processes to meet changes in market demand. However, conducting effective process monitoring in a multimode chemical process is challenging because data usually have multimodal distribution. In this study, DMF, a novel model based on a deep network, is proposed for learning new feature spaces; the multimodality of the mix data is eliminated, the same faults from all operating modes are assembled, and different faults are separated from each other. The proposed model uses a three-layer stacked autoencoder to extract features from the mixed data, uses a mode elimination term to remove data multimodality, and adopts a Fisher criterion term to separate the normal and fault states. Subsequently, DMF is combined with a self-organizing map (DMF–SOM) for visual process monitoring. DMF extracts discriminative features from the original data, and SOM visualizes these features such that the normal and fault states are distinguished on the two-dimensional output plane. The effectiveness of the DMF–SOM in process monitoring is verified by a study on the multimodal Tennessee Eastman process.

Introduction

With the increasing complexity and scale of modern chemical devices, using process monitoring technology to ensure the safe and stable operation of such devices has become a focus of engineering personnel. Practical continuous chemical processes often have multiple operating modes due to factors, such as raw material changes, market demand, and seasonal changes.

Many process monitoring methods have been proposed for multimode continuous chemical processes. In general, they can be divided into the multiple-model scheme and the single-model scheme. In the multiple-model scheme, mixed data are first classified into several modes, and one monitoring model is established for each mode. The decision function is defined simultaneously for the testing data to determine the mode to which they belong. Then, the corresponding monitoring model is applied for process monitoring. Ha et al. [1] used the k-nearest neighbor algorithm to classify raw data into several modes and then adopted multiple principal component analysis (PCA) models to monitor the process. Xu et al. [2] proposed the dynamic Bayesian independent component analysis to classify operating modes and monitor multimode processes with non-Gaussian distribution. Guo et al. [3] used k-means and local outlier factor (LOF) for mode separation and then built a so-called multiway PCA model of each mode. Song et al. [4] proposed the multi-subspace PCA with LOF to monitor multimode processes, where a new clustering strategy was developed. Lou et al. [5] proposed the hidden semi-Markov model (HSMM) for mode division and identification and then used PCA for process monitoring. The hidden Markov model is widely used in mode identification [6], [7], [8], [9]. For the multiple-model scheme, the accuracy of mode identification considerably affects subsequent procedures. Moreover, some common correlation exists between modes, and this common information is not fully used in the multiple-model approach. The other method to monitor multimode processes is to create a single model that does not require any mode division and identification. Most single-model monitoring methods seek to remove the multimodality feature from the data distribution by using a transformation function. Guo et al. [10] proposed an improved local entropy method to eliminate the multimodality and non-Gaussianity of input data and then applied locality-preserving projection to the preprocessed data for process monitoring. To monitor multimode processes via one single model, Song and Shi [11] proposed an algorithm named temporal-spatial global locality projections (TSGLP) for feature extraction of multimode processes; then, they used LOF to construct the monitoring statistic based on the extracted features. Wang et al. [12] proposed a double-level local information-based LOF to eliminate the multimodality of raw data and isolate faults. [13], [14], [15] used the local standardization method to eliminate differences between modes, thereby converting multimodal data into unimodal data. In addition, some single-model monitoring solutions are based on adaptive monitoring methods. Ge and Song [16] introduced a just-in-time-learning strategy to the modeling procedure of local least squares support vector regression and the residuals between the real output and the predicted one by a two-step information extraction strategy. Ma et al. [17] proposed a numerically efficient moving window local outlier probability algorithm for monitoring multimode industrial processes. Zhu et al. [18] proposed a novel recursive mixture factor analyzer method for monitoring multimode industrial processes. Another approach proposed is the decomposition of multimode data into common and individual subspaces for developing a unique monitoring model [19], [20].

The focus of this paper is to develop a single-model approach for multimode continuous chemical process monitoring. As mentioned previously, an effective solution of the single-model scheme is to seek to remove the multimodality feature from the data distribution by using a transformation function. However, most of the existing solutions to solve transformation functions are developed based on shallow learning methods. In general, a deep structure composed of multiple nonlinear mapping layers can represent the complex function more effectively than a shallow structure. The auto-encoder (AE) is one of the frameworks of the deep neural network (DNN). AE-based methods have been widely used to learn features from complex data for fault diagnosis and process monitoring in industrial processes. Yan and Yan [21] designed a teacher and supervise dual stacked AE to obtain features representing both process variables and quality indicators for fault detection. Yu and Zhao [22] proposed a monitoring strategy based on a denoising AE and elastic net to monitor industrial processes and isolate faulty variables. Yu and Zhang [23] proposed a manifold regularized stacked AE to describe distribution of the nonlinear process data and learn effective features for fault detection. Cheng et al. [24] proposed a new process monitoring method based on the variational recurrent AE. Luo et al. [25] proposed a stacked discriminant AE by adding a distance penalty into the loss function, which can extract more suitable representations from industrial data for fault diagnosis.

However, these improved AE-based methods are designed for unimodal processes. For multimode chemical processes, we can design a mode elimination term as the loss function of a stacked AE (SAE), so that the original data can be mapped to a new feature space in which the multimodality of the mix input data is eliminated. In addition, samples from different classes should be separated as much as possible and samples from the same class should be gathered as much as possible, which will undoubtedly improve the accuracy of process monitoring. Fisher discriminant analysis (FDA) is a well-known linear technique for dimensionality reduction and feature extraction. It can maximize the between-class scatter while minimizing the within-class scatter by maximizing Fisher criterion [26]. FDA has been widely employed for fault diagnosis and process monitoring [27], [28], [29], [30]. However, FDA is a linear method and cannot effectively handle industrial processes with strong nonlinearity. In addition, FDA assumes that data in the same class are subject to the Gaussian distribution, whereas in multimode processes, such data have multicentered distribution. Therefore, we can combine the designed mode elimination term and Fisher criterion as the loss function of SAE to cope with multimode process monitoring. In this paper, a feature extraction method called DMF for multimode continuous chemical processes is proposed. DMF uses a three-layer SAE as a mapping function and utilizes a proposed mode elimination term and a Fisher criterion term as the loss function to adjust the parameters of SAE. DMF can learn a new feature space in which the multimodality of the input data is eliminated; furthermore, the same faults from all modes are classified together, and different faults are separated from each other. In order to intuitively understand and monitor high-dimensional multimode processes, a self-organizing map (SOM) is used to visualize the features extracted by DMF and then monitor processes. The SOM can map the high-dimensional input data onto a two-dimensional (2D) space while preserving the topological structure of the input data [31]. The SOM has been successfully applied in various engineering applications covering areas like data visualization [32], [33] and classification [34], [35], fault diagnosis [36], [37], and process monitoring [38], [39]. Due to the excellent properties of the SOM, we use the SOM to visualize and monitor multimode processes.

The process monitoring strategy proposed in this paper for multimode continuous chemical processes can be summarized as follows. First, DMF is used to extract the discriminative features from the original data, which can effectively eliminate the multimodality of the original data and maximize the separation between faults. It should be noted that the establishment of the DMF model requires the collection of historical data of all modes. Second, the SOM is used to map the discriminative features extracted by DMF onto a 2D space in which different faults are clustered in separate areas. Therefore, we can visually and quickly observe the state to which the new test data belong and track their motion trajectory in real time. The highlighted advantages of the proposed method, which combines DMF and SOM (DMF–SOM), are that instead of developing a large number of local models, DMF–SOM builds only one single model based on the DNN. DMF–SOM can project high-dimensional complex multimode chemical process onto 2D space in which the multimodality of the data is eliminated and different faults can be maximally separated. Presentation of diverse information is provided to operators in a meaningful, comprehensive and intuitive manner by this method. Experiments on the multimodal Tennessee Eastman (TE) process verify that DMF–SOM can perform effective process monitoring.

The contributions of this research are as follows.

  • (1)

    A deep model called DMF is proposed. DMF uses a three-layer SAE as a mapping function and then utilizes a proposed mode elimination term and a Fisher criterion term as the loss function to adjust the parameters of SAE. DMF can efficiently eliminate the multimodality of the data and maximize the separation between faults.

  • (2)

    SOM is used for fault diagnosis and process monitoring, thereby enabling complex industrial data to be understood intuitively and deeply.

  • (3)

    A novel multimodal process monitoring method combining DMF and SOM is proposed. The effectiveness of DMF–SOM is verified by a study on the multimodal TE process.

The rest of the article is arranged as follows: Section 2 presents the preliminaries of SOM and SAE. Section 3 explains the proposed DMF and DMF–SOM. Section 4 discusses experiment implementation and analysis. Finally, Section 5 summarizes this paper.

Section snippets

SOm

The SOM as an unsupervised neural network was first proposed by Kohonen in 1982 [40]. The SOM can map input data onto a 2D space while maintaining such data’s topology and density distribution; that is, points that are near each other in the input space are grouped to nearby map units in the SOM output space, and vice versa. The SOM consists of an input layer and an output layer. Each input neuron is connected to all output neurons through weights, and the output neurons are arranged in a 2D

DMf

In this section, the DMF model is proposed to learn a new feature space; in this space, the multimodality of mix data is eliminated, the same faults of all modes are clustered together, and different faults are separated from each other. As shown in Fig. 3, the data of modes 1, 2, and 3 are represented by the circles, triangles, and squares, respectively. The red and blue colors indicate faults 1 and 2, respectively. Data from the three modes are mixed in the input space. After feature

TE process

The TE process has been widely used to evaluate and compare the effectiveness of process monitoring methods [42], [43], [44]. In this paper, the simulator is based on the revised version [44], [45], which is available at http://depts.washington.edu/control/LARRY/TE/download.html. According to the ratio of products G and H, the TE process has six operating modes. In this paper, modes 1, 2, and 3 are chosen for multimode simulation. There are 12 manipulated variables XMV(1) to XMV(12), 30

Conclusion

In this work, we propose the DMF–SOM for visual multimodal continuous chemical process monitoring. DMF can effectively eliminate the multimodality of data and separate different faults. Meanwhile, SOM can project the discriminative features extracted by DMF onto a 2D space in which different faults are divided into different areas, thereby enabling process monitoring. Experiments on the TE process demonstrate that DMF–SOM is effective in multimodal process monitoring.

However, DMF can lead to an

CRediT authorship contribution statement

Weipeng Lu: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Visualization, Data curation, Investigation. Xuefeng Yan: Project administration, Resources, Funding acquisition, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The authors are grateful for the support of National key research and development program of China (2020YFA0908303), and National Natural Science Foundation of China (21878081).

References (47)

  • F. Cheng et al.

    A novel process monitoring approach based on variational recurrent autoencoder

    Comput Chem Eng

    (2019)
  • X. Luo et al.

    Discriminant autoencoder for feature extraction in fault diagnosis

    Chemometr Intell Lab

    (2019)
  • L.H. Chiang et al.

    Fault diagnosis in chemical processes using Fisher discriminant analysis, discriminant partial least squares, and principal component analysis

    Chemometr Intell Lab

    (2000)
  • L.H. Chiang et al.

    Fault diagnosis based on Fisher discriminant analysis and support vector machines

    Comput Chem Eng

    (2004)
  • J. Nikkila et al.

    Analysis and visualization of gene expression data using Self-Organizing Maps

    Neural Networks

    (2002)
  • X.Y. You et al.

    Fault diagnosis of frequency converter in wind power system based on SOM neural network

    Int. Workshop Inform. Electron. Eng.

    (2012)
  • S.L. Jamsa-Jounela et al.

    A process monitoring system based on the Kohonen self-organizing maps

    Control Eng. Pract.

    (2003)
  • G. Abaei et al.

    An empirical study based on semi-supervised hybrid self-organizing map for software fault prediction

    Knowl.-Based Syst.

    (2015)
  • H. Wu et al.

    Deep convolutional neural network model based chemical process fault diagnosis

    Comput. Chem. Eng.

    (2018)
  • Y. Wang et al.

    A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network

    Isa T

    (2020)
  • A. Bathelt et al.

    Revision of the Tennessee Eastman process model

    Ifac Papersonline

    (2015)
  • M. Quiñones-Grueiro et al.

    Data-driven monitoring of multimode continuous processes: a review

    Chemometr Intell Lab

    (2019)
  • B. Song et al.

    Multisubspace principal component analysis with local outlier factor for multimode process monitoring

    Ind Eng Chem Res

    (2014)
  • Cited by (8)

    • A review on autoencoder based representation learning for fault detection and diagnosis in industrial processes

      2022, Chemometrics and Intelligent Laboratory Systems
      Citation Excerpt :

      The most commonly used strategy is to incorporate sparse regularization into the loss function, as it can restrict the activation degree of nodes during the multimode training so that hidden layer nodes can be selectively activated [54,55]. Besides, Lu et al. [56] designed a regularization for mode elimination to reduce the influence of mode change, and they further utilized the Fisher criterion to increase the sensitivity of features to fault information. Wang et al. [57] designed a network structure based on two SAEs for extracting mode-common features and mode-specific features.

    • An enhanced kernel learning data-driven method for multiple fault detection and identification in industrial systems

      2022, Information Sciences
      Citation Excerpt :

      Therefore, detecting abnormal conditions at an early stage can reduce maintenance costs and maintain the functionality of equipment [4–6]. As an effective data-driven fault detection approach, the kernel-based learning (KBL) technique can directly gain information from data sets saved in the sensors without the mathematical model [6–8]. The KBL fault detection method represents the original data in a low dimensional form to be free of the curse of dimensionality to a significant extent [9–12].

    • A distributed principal component regression method for quality-related fault detection and diagnosis

      2022, Information Sciences
      Citation Excerpt :

      With equipment becoming more sophisticated and large-scale, the requirement for the safety and stability of factory production processes has also increased [1–4]. The multivariate statistical analysis method (MSAM), a common data-driven technique, is developed to control some chemistry processes or complicated mechanical systems [4–6]. However, at the early stages, the assumed conditions of the systems are linear in the MSAM, which are quite different from the real operating environment [7].

    View all citing articles on Scopus
    View full text