A batch-wise LSTM-encoder decoder network for batch process monitoring

Highlights

• Proposed a multi-layer recurrent neural network in the encoder–decoder structure and the corresponding monitoring method for batch process monitoring.

• The proposed network can simultaneously capture the between and within batch dynamic features in the nonlinear batch process.

• The proposed model outperforms the AutoEncoder and Sub-PCA model in fault detection.

Abstract

Process monitoring is essential to keep quality consistency and operation safety in the batch process. However, the existence of multiphase, nonlinearity and dynamic features in the batch process makes the batch process monitoring a complicated task. In this work, a multi-layer recurrent neural network in the encoder–decoder structure called batch-wise LSTM-encoder decoder network is proposed to solve the difficulties mentioned above in batch process monitoring. The LSTM-encoder extracts the nonlinear dynamic features in both between and within batch direction, then projects the high dimensional input space to a low dimensional hidden state space. The decoder part regenerates the samples from hidden states. Control statistics H² and SPE are designed for process monitoring, and the corresponding control limits are estimated by kernel density estimation. A case study on an extensive reference penicillin fermentation dataset suggests that the proposed method can detect the fault samples more effectively than previous methods while keeping the same robustness in normal conditions.

Introduction

Batch processes play a significant role in modern industries such as pharmaceutical, semiconductor and biotechnology (Wang et al., 2019). It is defined as a type of process that repeatably produces a finite quantity of products once a time following a sequence of process steps (Mehta and Reddy, 2015). Because of the flexibility in production, it is important to keep quality consistency and operation safety in the batch process. Process monitoring, or in other words, online fault detection, is vital to meet the demand of quality and safety in the batch process, so it has become a hot field in the past decades (Qin, 2012).

Owing to the abundant sensors linking to the network in the modern batch process, statistical process control (SPC) methods have been widely applied to monitor and control the process (Yao and Gao, 2009). Univariate control chart is a feasible and popular method to monitor the process. However, it suffers from “blind spots” in a multivariate setting, which would loosen the control limits (Wang et al., 2017). In order to overcome these difficulties, many multivariate process monitoring methods have been developed. Compared to the continuous processes, the dynamic features within and between batches make the monitoring of the batch process more complex and challenging. In all the methods suitable for batch processes, multiway principal component analysis (MPCA) is the most widely used one (Nomikos and MacGregor, 1994). The multiway methods firstly unfold the three-dimensional (Batch × Time × Variable) historical batch process data cube to a two-dimensional matrix, such as batch-wise unfolding, Batch × (Time × Variable) and variable-wise unfolding, Variable × (Time × Batch). Then, the PCA method is applied to these two-dimensional matrices to project the high-dimensional data into a low-dimensional space and the statistics such as Hotelling's T² and square prediction error(SPE) are adopted to measure the main space and residual information.

However, MPCA methods are typical linear methods which may fail to extract the nonlinear information in the batch process. In order to improve the nonlinear process monitoring efficiency, many modified methods based on MPCA are proposed in the past decades. Among them, multiphase methods (Lu et al., 2004, Zhao and Sun, 2013) and kernel methods (Lee et al., 2004, Jia et al., 2010) are two mainstream methods. Multiphase methods are similar to piecewise linearization. They first divide the batch into several phases containing relatively similar dynamic characteristics. Then local MPCA models are built for each phase separately. The ability of extracting nonlinear information is improved by reducing nonlinearity in one phase (Lu et al., 2004). Kernel PCA uses kernel functions to map data from low-dimensional input space to high-dimensional feature space, then apply PCA in the high-dimensional feature space (Lee et al., 2004).

Nowadays, artificial neural networks (ANN) are getting more attention for the ability of effectively extracting nonlinear features (Adamowski et al., 2012). Particularly, autoencoders (AEs) and recurrent neural networks(RNN) are two widely studied networks in the field of process monitoring (Yan et al., 2016, Yu and Zhao, 2019, Shahnazari, 2020, Zhang et al., 2019, Loy-Benitez et al., 2020). AEs is a type of multi-layer neural network with a low-dimensional central layer and is effective in dimension reducing and variance capturing in the nonlinear continuous processes according to (Yan et al., 2016, Yu and Zhao, 2019). In Yan et al. (2016), variant autoencoders including denoising autoencoders and contractive autoencoders are applied to the Tennessee Eastman process (TE process) and monitors the process by statistics H². In Yu and Zhao (2019), denoising autoencoders along with elastic net is proposed to generate a sparse and robust network. Statistics A², SPE and C are used to monitor the main hidden space, residual information, and overall measurement in the continuous processes. Considering the nature of time sequences, RNN is extensively studied for the ability of capturing sequential and dynamic information in the continuous process (Shahnazari, 2020). In Shahnazari (2020), a bank of RNNs is used to build the predictive model from the input sequence, and process monitoring is done by comparing the predictions and realistic samples. Moreover, the networks combining AEs and RNN are carried out in the field of continuous process monitoring to utilize the ability of dimension reducing and sequential information capturing (Zhang et al., 2019, Loy-Benitez et al., 2020). In Zhang et al. (2019), UNet based model, which is a variant of AEs with LSTM linking the corresponding encoder and decoder layer, is proposed to extract the nonlinear features along with time sequence information and is applied to monitor a power plant. In Loy-Benitez et al. (2020), a multi-input multi-output RNN encoder called MG-RNN-AE is designed to reduce dimension and reconstruct the original input sequence, and process monitoring is realized by monitoring the statistics H² and SPE.

However, all the above mentioned neural networks can only extract the nonlinear feature between samples. As a characteristic of the batch process, the dynamic features not only exist between samples within a batch but also between batches. This concern indicates that some specific designs could be carried out to improve the monitoring performance in the nonlinear batch process. In Jiang et al. (2020), the three-way batch process data cube is firstly unfolded to a two-dimensional matrix. An AE model is constructed to extract the relations between variables. Finally the dynamic features between batches are captured by canonical correlation analysis (CCA) of 2-D matrices, including samples from different batches. In this way, the synthetic hidden state of AEs and CCA is more suitable to monitor the batch processes.

In order to address the above concerns, a batch-wise LSTM-encoder decoder network, along with the monitoring scheme for the nonlinear batch process, is proposed. The main contribution of the proposed method is that the LSTM-encoder can capture the between and within batch dynamic features in the nonlinear batch process simultaneously by sliding through the batch-wise input sequence. Specifically, the batch-wise LSTM-encoder is a multi-layer recurrent neural network with a shrinking layer dimension. The decoder part is a symmetric multi-layer neural network. The input sequence of LSTM-encoder is a series of kth samples from different batches. The encoder slides through these input sequences to obtain the between batch features and shrinks layer by layer to obtain the within batch features. After encoding, the last hidden state of the input sequence is passed to the decoder to reconstruct the input sample. Batch process monitoring is achieved by monitoring the proposed H² and SPE statistics. A case study of an extensive reference data set associated with the PenSim benchmark model proposed by Van Impe and Gins (2015) shows an outstanding process-monitoring performance, especially for the barely detectable faults comparing to the AEs and MPCA monitoring model.