A data-driven Bayesian network learning method for process fault diagnosis

Abstract

This paper presents a data-driven methodology for fault detection and diagnosis (FDD) by integrating the principal component analysis (PCA) with the Bayesian network (BN). Though the integration of PCA-BN for FDD purposes has been studied in the past, the present work makes two contributions for process systems. First, the application of correlation dimension (CD) to select principal components (PCs) automatically. Second, the use of Kullback-Leibler divergence (KLD) and copula theory to develop a data-based BN learning technique. The proposed method uses a combination of vine copula and Bayes’ theorem (BT) to capture nonlinear dependence of high-dimensional process data which eliminates the need for discretization of continuous data. The data-driven integrated PCA-BN framework has been applied to two processing systems. Performance of the proposed methodology is compared with the independent component analysis (ICA), kernel principal component analysis (KPCA), kernel independent component analysis (KICA), and their integrated frameworks with the BN. The comparative study suggests that the proposed framework provides superior performance.

Introduction

Fault detection and diagnosis (FDD) has an utmost importance in increasing the profitability of a process plant by ensuring safety, reliability, and product quality. By nature, process industries are a source of high-dimensional correlated data due to multivariate process operations and digitalization. The success of an FDD tool largely depends on an accurate analysis of these data for predicting the process state (i.e. faulty or normal) and decoding the underneath causal relationships and correlation structure among the process variables (Jia and Li, 2020; Zhou and Li, 2018). Data-based process FDD tools play a pivotal role in preventing a fault being propagated to an accident by providing an early indication of fault and information about the root cause. The multivariate statistical process monitoring (MSPM) tools are continually drawing researchers’ attention due to their ease of implementation, reliable performance, and relatively lower historical data requirement compared to machine learning techniques (i.e. support vector machine).

The principal component analysis (PCA) (Wise et al., 1988), partial least squares (PLS) (Kresta et al., 1991), independent component analysis (ICA) (Kano et al., 2003), and their derivatives are the major MSPM tools used in FDD. An appropriate number of principal component (PC) or independent component (IC) selection greatly affects the performance of these MSPM tools. The cumulative percentile variation (CPV) (Malinowski and Howery, 1980) and SCREE (Cattell, 1966) procedures are the two most common and reliable means in this context. Unlike PCA, the ICA cannot distinguish variations captured by the ICs, and therefore, the number of ICs is often set equal to the number of PCs from PCA. The modified ICA (MICA) is another alternative formulation of ICA to capture the significant process variations with the required ICs (Lee et al., 2006). Like PCA, the MICA utilizes CPV or SCREE approach to pick the significant ICs to build the monitoring model.

Due to SCREE’s graphical nature, its output is prone to error when the dimension is higher. On the other hand, the CPV uses a simple mathematical formulation to estimate the required number of PCs. Usually, the first few PCs are selected based on the percentage variation that an MSPM designer wants to capture. Although the CPV uses a straightforward equation, it is a user-perspective approach, as different researchers have used a wide range of values (i.e. 65–99%) to construct the monitoring models, with a view to reducing false alarms, yet securing an early fault detectability.

The correlation dimension (CD) finds a set of linear or nonlinear axes that represents the multidimensional data in a reduced dimension retaining the vital information of original data (Grassberger and Procaccia, 1983). This exactly matches to the definition of PC or IC, and the existing techniques to measure the required number of PCs or ICs to build the FDD model can be improved and made automated by including it in model development (i.e. the CD is an assumption-free approach, and no user preference is required).

Along with early fault detectability, the MSPM tools can provide diagnostic information in terms of multivariate contribution plots. However, this diagnosis is often inaccurate and misleading due to the smearing effect (Westerhuis et al., 2000). The Bayesian networks (BNs) are becoming increasingly popular in FDD, specially in fault diagnosis. In addition to FDD, BNs are widely used in process safety and risk analysis (Adedigba et al., 2017; Amin et al., 2020; Barua et al., 2016; Ghosh et al., 2020; Guo et al., 2019; Ping et al., 2018; Rostamabadi et al., 2020; Yazdi and Kabir, 2017).

Rojas-Guzman and Kramer (1993) showed the suitability of a BN over the then rule-based expert systems in fault diagnosis. Process knowledge was utilized as the basic building block of the network. A BN-based single sensor fault detection and identification technique was proposed by Mehranbod et al. (2003). The probability absolute difference was utilized for fault detection. Nonetheless, the authors did not address how continuous data were converted into probabilities that were further used to update the BN.

A combination of T² statistics and BN was proposed by Verron et al. (2010). The authors utilized the causal decomposition of T² statistics for fault diagnosis. However, the application to a hot forming process suggested that the proposed method could not ensure accurate diagnosis in all the studied cases. The conventional BN is static, and hence, it cannot capture the dynamic nature of process operations. To address this issue, Yu and Rashid (2013) proposed a dynamic BN (DBN)-based process monitoring model. The kernel distribution was utilized to learn the parameters while the network topology was developed from prior knowledge and process flow diagram. The other applications of DBN in process monitoring can be found in the works by Zhang and Dong (2014) and Amin et al. (2019a). The major advantage of DBN-based methods is their capability to detect and diagnose a fault and identify its propagation pathway by a solitary tool. However, these cannot provide an early detection in case of subtle faults.

Many integrated frameworks have been developed using the early fault detectability of the MSPM tools and accurate diagnosis capacity of a BN. A fault is first detected by an MSPM tool; then, diagnosis is completed by the BN using contribution plots. As a result, both the early fault detection and accurate diagnosis features are captured. Some of the hybrid methods adopting this philosophy are PCA-BN by Mallick and Imtiaz (2013), MICA-BN by Yu et al. (2015), KPCA-BN by Gharahbagheri et al. (2017), and PCA-BN with likelihood evidence by Amin et al. (2018). Strong prior knowledge of BN structure and conditional probability tables (CPTs) was the prerequisite for the first PCA-BN work. Yu et al. (2015) and Amin et al. (2018) utilized prior knowledge and data for learning structure and CPTs, respectively.

Developing BNs from process data is still an existing challenge. Yang et al. (2014) demonstrated the applications of Granger causality and transfer entropy for capturing causality and connectivity from process data. Later, Gharahbagheri et al. (2017) utilized these tools to build BNs from process data. Selecting the optimal lag for Granger causality requires significant efforts. Besides, the outputs from transfer entropy are often error-prone in the context of process data (Yu and Rashid, 2013). One of the salient features of Gharahbagheri’s work was the use of residuals for estimating priors and CPTs. However, continuous data were discretized at the cost of information loss. Zhu et al. (2019) proposed a multiblock transfer entropy (MBTE)-based BN learning technique for root cause diagnosis. The authors segregated the process into different sub-systems based on prior knowledge and subsequently, used the MBTE to find causal relations. This technique cannot be considered as purely data-driven since prior knowledge is used to divide the entire process.

Meng et al. (2019) applied a technique called the family transfer entropy (FTE) to alarm repository and proposed a score-based BN learning method. Process variables were classified based on alarm history. Suppose fault A causes a total of five variables exceeding the pre-defined thresholds. These variables are included in the developed BN for diagnosing fault A. The causal structure of these five variables are then determined using FTE. This procedure is carried out for each fault type. Wang et al. (2018) proposed a BN-based fault diagnosis methodology. Knowledge of faults and Pearson’s correlation coefficient were used to develop the causal structure for each fault type, and the parameters were learnt using the expectation maximization algorithm. Amin et al. (2019b) combined the multivariate exponentially weighted moving average PCA (MEWMA-PCA) with the BN to detect and diagnose some faults in the Tennessee Eastman chemical process that have been described as unobservable or difficult to detect. The authors proposed a data-based BN structure learning algorithm that utilized historical fault signatures. However, in-depth fault information is required to develop the BNs in the above mentioned works which may not be obtainable in many cases.

The Kullback-Leibler divergence (KLD) is an approach that is used in information theory to measure the distance between two probability distributions (Kullback and Leibler, 1951). Also, it can be utilized to determine the amount and direction of mutual information (MI) transfer between two variables. Several studies are conducted to use MI by KLD to learn BN topologies (Friedman et al., 2013; Wu et al., 2001). These works mainly consider discrete data. The KLD is a lag selection free technique, and its calculation is straightforward compared to Granger causality and transfer entropy. Therefore, it can be utilized to develop a BN topology learning method to overcome the limitations faced by Granger causality and transfer entropy. Besides, it will eliminate the necessity of process knowledge or detail fault information to build the BN and enable developing a data-driven BN structure.

Copula functions provide the estimate of joint density among multi-dimensional variables without discretizing data. The correlation estimates provided by the copula functions in terms of the Kendall’s rank correlation coefficient and Spearman's rank correlation coefficient are often described as a measure of nonlinear dependence, as well. Although the conventional bivariate copulas can be used to model dependency between two variables, these cannot be utilized flexibly to measure the dependence in high-dimensional cases (Joe, 1996).

Elidan (2010) developed a technique to estimate the joint densities in a flexible manner using bivariate copulas and proposed a copula Bayesian network (CBN) model. It overcomes the limitation of traditional discrete BNs, as the use of copulas allowed to model continuous variables. Additionally, the CBN preserves the strengths of traditional BNs. Conditional independence among variables was assumed to estimate high-dimensional joint densities. However, this may not be a valid assumption, especially in chemical systems, as process variables exhibit strong nonlinear dependence.

The vine copula models such as the R-vine, C-vine, and D-vine are pertinent in this context, as these can capture the joint dependence in high-dimensional cases using the bivariate copula decompositions. Furthermore, no conditional independence assumption is required. Although several process monitoring schemes are available in existing literature that have utilized the vine copula models (Ren et al., 2015; Zhou and Li, 2018), none of them utilized the copula-based BN for fault diagnosis. The vine copula-based models can provide a good detection performance like the MSPM tools. However, the diagnostic task is not straightforward like the BN-based methods, as each pair of variables needs to be analyzed that may introduce a huge computational burden for large-scale processes. On the contrary, the BNs only need to be updated, and the percentage change in each node can be used for fault diagnosis. Therefore, a vine copula aided BN model can be utilized for fault diagnosis that ensures high-dimensional continuous process data are utilized in building the CPTs, avoiding a considerable amount of computational efforts.

The current research first examines the efficacy of CD-based PCA, ICA, KPCA, and KICA over the CPV-based counterparts. It then integrates the data-based BN structure with PCA-CD, as PCA-CD-BN is found to be the most effective method for FDD based on four fault cases studied in the continuous stirred tank heater (CSTH) and binary distillation column.

This work proposes a CD-based PC or IC selection technique since it can provide an unbiased measurement of required dimensions to compress the original dataset. A data-driven BN learning technique is also proposed using the KLD and copula theory. Although bivariate copulas are easier to apply for estimation of the joint density between two variables, process variables often have higher dimensional dependencies where the bivariate copulas may not be suitable. Therefore, a combination of the vine copula and the Bayes’ theorem (BT) is used to overcome this problem.

The remainder of this article is organized as follows: Section 2 describes the distinct steps of this methodology. Applications of the proposed framework to two process systems are displayed in Section 3. Detailed comparative performance analysis with the ICA and KPCA-based methods are discussed in Section 4. The concluding remarks, advantages, limitations, and future work scopes are summarized in Section 5.