Skip to main content
Log in

Fault Detection of Complex Processes Using nonlinear Mean Function Based Gaussian Process Regression: Application to the Tennessee Eastman Process

  • Research Article-Chemical Engineering
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Process monitoring or fault detection and diagnosis have gained tremendous attention over the past decade in order to achieve better product quality, minimise downtime and maximise profit in process industries. Among various process monitoring techniques, data-based machine learning approaches have become immensely popular in the past decade. However, a promising machine learning technique Gaussian process regression has not yet received adequate attention for process monitoring. In this work, Gaussian process regression (GPR)-based process monitoring approach is applied to the benchmark Tennessee Eastman challenge problem. Effect of various GPR hyper-parameters on monitoring efficiency is also thoroughly investigated. The results of GPR model is found to be better than many other techniques which is reported in a comparative study in this work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Abbreviations

abs:

Absolute value

a, b, B, c, d :

Weights assigned

cov:

Predicted covariance

\({\mathcal{D}}\) :

Dataset

D :

Number of variables

\({\mathbb{E}}\left( . \right)\) :

Expectations

\(f_{*} ,g_{*}\) :

Gaussian process posterior prediction

\(\bar{f}_{*} ,\bar{g}\) :

Predicted mean

\({\mathcal{G}\mathcal{P}}\) :

Gaussian process

\(I\) :

Identity matrix

\(k\) :

Covariance function for a single case

\(k_{*}\) :

Covariance function for single test case

K :

Covariance (gram) matrix

l :

Length scale

m :

Number of samples

\(m\left( x \right)\) :

Mean function

\({\mathcal{N}}\) :

Normal distribution function

o :

Value associated with \(\left( {1 - \alpha } \right)\%\) confidence interval

\(p\) :

Probability density

\({\mathbb{V}}\) :

Variance of a sample

x :

Sample vector

X :

Matrix of training input

\(X_{*}\) :

Matrix of testing input

\(y\) :

Output training vector

\(\beta\) :

Gaussian prior

\(\bar{\beta }\) :

Constant

\(\delta_{pq}\) :

Kronecker delta

\(\epsilon\) :

Gaussian noise

\(\sigma\) :

Variance associated with the predicted output

\(\sigma_{f}^{2}\) :

Signal variance

\(\sigma_{n}^{2}\) :

Noise variance

\({\text{BIC}}_{\text{SPE}}\) :

Bayesian Inference-Based Criterion - Squared Prediction Error

\({\text{BIP}}_{\text{S}}\) :

Bayesian inference-based probability in state space

BIP:

Bayesian inference-based probability

BSPCA:

Bayesian strategy PCA

CCA-ERD:

Canonical correlation analysis-based explicit relation discovery

D:

Dissimilarity index

DiPCA:

Distributed PCA

DR:

Distance from the hypersphere

DTL:

Determining whether it is tight or loose

EICA:

Ensemble ICA

FA:

Factor analysis

FA-ICA:

Factor analysis-ICA

\(GT^{2}\) :

Statistic to monitor the factor space

GLSA:

Global local structure analysis

GMM:

Gaussian mixture models

JIR-PCA-SVDD:

Just in time response-PCA-SVDD

KGLPP:

Kernel global local preserving projections

KICA:

Kernel independent component analysis

KLPP:

Kernel local preserving projections

LDS:

Linear dynamic system

LGPCA:

Local global principal component analysis

LGSS:

Linear Gaussian state Space

LPP:

Local preserving projections

Modified ICA:

Modified independent component analysis

MCVA:

Mixture canonical variate analysis

NGS:

Non Gaussian static

OCSVM:

One class support vector machines

OMMP:

Orthogonal multi manifold projections

PCA-SVDD PCA:

Support vector data description

PM-WKPCA:

Probability density estimation and moving weighted PCA

PPCA:

Probabilistic PCA

P-WKPCA:

Probability density estimation and weighted PCA

RPCA:

Recursive PCA

Semi NMU:

Semi nonnegative matrix approximations

SFA:

Sensitive factor analysis

SLDS:

Supervised linear dynamic system

SPE:

Squared prediction error

SVM:

Support vector machines

SpPCA:

Sparce PCA

\(T^{2}\) :

Hotelling \(T^{2}\)

\(T_{f}^{2}\) :

Hotelling \(T^{2}\) in feature subspace

\({\text{WGT}}^{2}\) :

Modified hotelling \(T^{2}\)

WFA:

Weighted factor analysis

wOCSVM:

Weighted one class support vector machines

WPCA:

Weighted PCA

References

  1. Russell, E.; Chiang, L.H.: Data-Driven Methods for Fault Detection and Diagnosis in Chemical Processes, 2001st ed., Springer, London (n.d.)

  2. Ge, Z.; Song, Z.; Ding, S.X.; Huang, B.: Data mining and analytics in the process industry: the role of machine learning. IEEE Access. 5, 20590–20616 (2017). https://doi.org/10.1109/ACCESS.2017.2756872

    Article  Google Scholar 

  3. Jackson, J.E.: Principal components and factor analysis: part I—principal components. J. Qual. Technol. 12, 201–213 (1980). https://doi.org/10.1080/00224065.1980.11980967

    Article  Google Scholar 

  4. Russell, E.L.; Chiang, L.H.; Braatz, R.D.: Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis. Chemom. Intell. Lab. Syst. 51, 81–93 (2000). https://doi.org/10.1016/S0169-7439(00)00058-7

    Article  Google Scholar 

  5. Shao, J.D.; Rong, G.; Lee, J.M.: Learning a data-dependent kernel function for KPCA-based nonlinear process monitoring. Chem. Eng. Res. Des. 87, 1471–1480 (2009). https://doi.org/10.1016/j.cherd.2009.04.011

    Article  Google Scholar 

  6. Rato, T.J.; Reis, M.S.: Fault detection in the Tennessee Eastman benchmark process using dynamic principal components analysis based on decorrelated residuals (DPCA-DR). Chemom. Intell. Lab. Syst. 125, 101–108 (2013). https://doi.org/10.1016/J.CHEMOLAB.2013.04.002

    Article  Google Scholar 

  7. Lee, J.M.; Yoo, C.K.; Lee, I.B.: Statistical monitoring of dynamic processes based on dynamic independent component analysis. Chem. Eng. Sci. 59, 2995–3006 (2004). https://doi.org/10.1016/j.ces.2004.04.031

    Article  Google Scholar 

  8. Yin, L.; Wang, H.; Fan, W.: Active learning-based support vector data description method for robust novelty detection. Knowl. Based Syst. 153, 40–52 (2018). https://doi.org/10.1016/j.knosys.2018.04.020

    Article  Google Scholar 

  9. Karami, M.; Wang, L.: Fault detection and diagnosis for nonlinear systems: a new adaptive Gaussian mixture modeling approach. Energy Build. 166, 477–488 (2018). https://doi.org/10.1016/j.enbuild.2018.02.032

    Article  Google Scholar 

  10. Jia, Q.; Zhang, Y.: Quality-related fault detection approach based on dynamic kernel partial least squares. Chem. Eng. Res. Des. 106, 242–252 (2016). https://doi.org/10.1016/j.cherd.2015.12.015

    Article  Google Scholar 

  11. Xiao, Y.; Wang, H.; Zhang, L.; Xu, W.: Two methods of selecting Gaussian kernel parameters for one-class SVM and their application to fault detection. Knowl. Based Syst. 59, 75–84 (2014). https://doi.org/10.1016/j.knosys.2014.01.020

    Article  Google Scholar 

  12. Yang, Q.; Li, J.; Le Blond, S.; Wang, C.: Artificial neural network based fault detection and fault location in the DC microgrid. Energy Procedia 103, 129–134 (2016). https://doi.org/10.1016/j.egypro.2016.11.261

    Article  Google Scholar 

  13. Chiang, L.H.; Kotanchek, M.E.; Kordon, A.K.: Fault diagnosis based on Fisher discriminant analysis and support vector machines. Comput. Chem. Eng. 28, 1389–1401 (2004). https://doi.org/10.1016/J.COMPCHEMENG.2003.10.002

    Article  Google Scholar 

  14. Boškoski, P.; Gašperin, M.; Petelin, D.; Juričić, D.: Bearing fault prognostics using Rényi entropy-based features and Gaussian process models. Mech. Syst. Signal Process. 52–53, 327–337 (2015). https://doi.org/10.1016/j.ymssp.2014.07.011

    Article  Google Scholar 

  15. Deng, H.; Liu, Y.; Li, P.; Zhang, S.: Active learning for modeling and prediction of dynamical fluid processes. Chemom. Intell. Lab. Syst. 183, 11–22 (2018). https://doi.org/10.1016/j.chemolab.2018.10.005

    Article  Google Scholar 

  16. Liu, Y.; Chen, T.; Chen, J.: Auto-switch gaussian process regression-based probabilistic soft sensors for industrial multigrade processes with transitions. Ind. Eng. Chem. Res. 54, 5037–5047 (2015). https://doi.org/10.1021/ie504185j

    Article  Google Scholar 

  17. Liu, Y.; Wu, Q.Y.; Chen, J.: Active selection of informative data for sequential quality enhancement of soft sensor models with latent variables. Ind. Eng. Chem. Res. 56, 4804–4817 (2017). https://doi.org/10.1021/acs.iecr.6b04620

    Article  Google Scholar 

  18. Samuelsson, O.; Björk, A.; Zambrano, J.; Carlsson, B.: Gaussian process regression for monitoring and fault detection of wastewater treatment processes. Water Sci. Technol. 75, 2952–2963 (2017). https://doi.org/10.2166/wst.2017.162

    Article  Google Scholar 

  19. Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) Advanced Lectures on Machine Learning. ML 2003. Lecture Notes in Computer Science, vol. 3176. Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28650-9_4

  20. Downs, J.J.; Vogel, E.F.: A plant-wide industrial process control problem. Comput. Chem. Eng. 17, 245–255 (1993). https://doi.org/10.1016/0098-1354(93)80018-I

    Article  Google Scholar 

  21. Chen, J.; Liao, C.M.: Dynamic process fault monitoring based on neural network and PCA. J. Process Control 12, 277–289 (2002). https://doi.org/10.1016/S0959-1524(01)00027-0

    Article  Google Scholar 

  22. Zhang, Y.: Enhanced statistical analysis of nonlinear processes using KPCA, KICA and SVM. Chem. Eng. Sci. 64, 801–811 (2009). https://doi.org/10.1016/j.ces.2008.10.012

    Article  Google Scholar 

  23. Ge, Z.; Yang, C.; Song, Z.: Improved kernel PCA-based monitoring approach for nonlinear processes. Chem. Eng. Sci. 64, 2245–2255 (2009). https://doi.org/10.1016/j.ces.2009.01.050

    Article  Google Scholar 

  24. Ben Khediri, I.; Limam, M.; Weihs, C.: Variable window adaptive Kernel Principal Component Analysis for nonlinear nonstationary process monitoring. Comput. Ind. Eng. 61, 437–446 (2011). https://doi.org/10.1016/j.cie.2011.02.014

    Article  Google Scholar 

  25. Ma, H.; Hu, Y.; Shi, H.: A novel local neighborhood standardization strategy and its application in fault detection of multimode processes. Chemom. Intell. Lab. Syst. 118, 287–300 (2012). https://doi.org/10.1016/j.chemolab.2012.05.010

    Article  Google Scholar 

  26. Jiang, Q.; Yan, X.: Chemical processes monitoring based on weighted principal component analysis and its application. Chemom. Intell. Lab. Syst. 119, 11–20 (2012). https://doi.org/10.1016/j.chemolab.2012.09.002

    Article  Google Scholar 

  27. Fan, J.; Wang, Y.: Fault detection and diagnosis of nonlinear non-Gaussian dynamic processes using kernel dynamic independent component analysis. Inf. Sci. (Ny) 259, 369–379 (2014). https://doi.org/10.1016/J.INS.2013.06.021

    Article  MATH  Google Scholar 

  28. Jing, C.; Hou, J.: SVM and PCA-based fault classification approaches for complicated industrial process. Neurocomputing 167, 636–642 (2015). https://doi.org/10.1016/j.neucom.2015.03.082

    Article  Google Scholar 

  29. Hu, Y.; Ma, H.; Shi, H.: Robust online monitoring based on spherical-kernel partial least squares for nonlinear processes with contaminated modeling data. Ind. Eng. Chem. Res. 52, 9155–9164 (2013). https://doi.org/10.1021/ie4008776

    Article  Google Scholar 

  30. Yin, S.; Zhu, X.; Kaynak, O.: Improved PLS focused on key-performance-indicator-related fault diagnosis. IEEE Trans. Ind. Electron. 62, 1651–1658 (2015). https://doi.org/10.1109/TIE.2014.2345331

    Article  Google Scholar 

  31. Zhou, D.; Li, G.; Qin, S.J.: Total projection to latent structures for process monitoring. AIChE J. (2009). https://doi.org/10.1002/aic.11977

    Article  Google Scholar 

  32. Odiowei, P.-E.P.; Cao, Yi: Nonlinear dynamic process monitoring using canonical variate analysis and kernel density estimations. IEEE Trans. Ind. Inform. 6, 36–45 (2010). https://doi.org/10.1109/TII.2009.2032654

    Article  Google Scholar 

  33. Yin, S.; Wang, G.; Gao, H.: Data-driven process monitoring based on modified orthogonal projections to latent structures. IEEE Trans. Control Syst. Technol. 24, 1480–1487 (2016). https://doi.org/10.1109/TCST.2015.2481318

    Article  Google Scholar 

  34. Xie, L.; Kruger, U.; Lieftucht, D.; Littler, T.; Chen, Q.; Wang, S.Q.: Statistical monitoring of dynamic multivariate processes: part 1. Modeling autocorrelation and cross-correlation. Ind. Eng. Chem. Res. 45, 1659–1676 (2006). https://doi.org/10.1021/ie050583r

    Article  Google Scholar 

  35. Lee, J.M.; Qin, S.J.; Lee, I.B.: Fault detection and diagnosis based on modified independent component analysis. AIChE J. 52, 3501–3514 (2006). https://doi.org/10.1002/aic.10978

    Article  Google Scholar 

  36. Zhang, Y.; Zhang, Y.: Fault detection of non-Gaussian processes based on modified independent component analysis. Chem. Eng. Sci. 65, 4630–4639 (2010). https://doi.org/10.1016/j.ces.2010.05.010

    Article  Google Scholar 

  37. Wang, L.; Shi, H.: Multivariate statistical process monitoring using an improved independent component analysis. Chem. Eng. Res. Des. 88, 403–414 (2010). https://doi.org/10.1016/j.cherd.2009.09.002

    Article  Google Scholar 

  38. Odiowei, P.P.; Cao, Y.: State-space independent component analysis for nonlinear dynamic process monitoring. Chemom. Intell. Lab. Syst. 103, 59–65 (2010). https://doi.org/10.1016/j.chemolab.2010.05.014

    Article  Google Scholar 

  39. Jiang, Q.; Yan, X.: Non-Gaussian chemical process monitoring with adaptively weighted independent component analysis and its applications. J. Process Control 23, 1320–1331 (2013). https://doi.org/10.1016/j.jprocont.2013.09.008

    Article  Google Scholar 

  40. Ding, S.X.; Yin, S.; Hao, H.; Zhang, P.; Haghani, A.: A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J. Process Control 22, 1567–1581 (2012). https://doi.org/10.1016/j.jprocont.2012.06.009

    Article  Google Scholar 

  41. Yu, J.: Hidden Markov models combining local and global information for nonlinear and multimodal process monitoring. J. Process Control 20, 344–359 (2010). https://doi.org/10.1016/j.jprocont.2009.12.002

    Article  Google Scholar 

  42. Tao, Y.; Shi, H.; Song, B.; Tan, S.: Parallel supervised additive and multiplicative faults detection for nonlinear process. J. Frankl. Inst. 356, 11716–11740 (2019). https://doi.org/10.1016/j.jfranklin.2019.06.020

    Article  MathSciNet  MATH  Google Scholar 

  43. Jiang, Q.; Yan, X.: Probabilistic monitoring of chemical processes using adaptively weighted factor analysis and its application. Chem. Eng. Res. Des. 92, 127–138 (2014). https://doi.org/10.1016/j.cherd.2013.06.031

    Article  Google Scholar 

  44. Jiang, Q.; Yan, X.: Probabilistic Weighted NPE-SVDD for chemical process monitoring. Control Eng. Pract. 28, 74–89 (2014). https://doi.org/10.1016/j.conengprac.2014.03.008

    Article  Google Scholar 

  45. Jiang, Q.; Yan, X.; Huang, B.: Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and Bayesian inference. IEEE Trans. Ind. Electron. 63, 377–386 (2016). https://doi.org/10.1109/TIE.2015.2466557

    Article  Google Scholar 

  46. Wang, B.; Li, Z.; Yan, X.: Multi-subspace factor analysis integrated with support vector data description for multimode process monitoring. J. Frankl. Inst. 355, 7664–7690 (2018). https://doi.org/10.1016/j.jfranklin.2018.07.044

    Article  MathSciNet  MATH  Google Scholar 

  47. Wen, Q.; Ge, Z.; Song, Z.: Multimode dynamic process monitoring based on mixture canonical variate analysis model. Ind. Eng. Chem. Res. 54, 1605–1614 (2015). https://doi.org/10.1021/ie503324g

    Article  Google Scholar 

  48. Liu, Y.; Xie, M.: Rebooting data-driven soft-sensors in process industries: a review of kernel methods. J. Process Control 89, 58–73 (2020). https://doi.org/10.1016/j.jprocont.2020.03.012

    Article  Google Scholar 

  49. Apsemidis, A.; Psarakis, S.; Moguerza, J.M.: A review of machine learning kernel methods in statistical process monitoring. Comput. Ind. Eng. 142, 106376 (2020). https://doi.org/10.1016/j.cie.2020.106376

    Article  Google Scholar 

  50. Rasmussen, C.E.; Nickisch, H.: Gaussian processes for machine learning (GPML) toolbox. J. Mach. Learn. Res. 11, 3011–3015 (2010)

    MathSciNet  MATH  Google Scholar 

  51. Ge, Z.; Song, Z.: Distributed PCA model for plant-wide process monitoring. Ind. Eng. Chem. Res. 52, 1947–1957 (2013). https://doi.org/10.1021/ie301945s

    Article  Google Scholar 

  52. Ge, Z.; Chen, X.: Supervised linear dynamic system model for quality related fault detection in dynamic processes. J. Process Control. 44, 224–235 (2016). https://doi.org/10.1016/j.jprocont.2016.06.003

    Article  Google Scholar 

  53. Rato, T.; Reis, M.; Schmitt, E.; Hubert, M.; De Ketelaere, B.: A systematic comparison of PCA-based Statistical Process Monitoring methods for high-dimensional, time-dependent Processes. AIChE J. 62, 1478–1493 (2016). https://doi.org/10.1002/aic.15062

    Article  Google Scholar 

  54. Luo, L.; Bao, S.; Mao, J.; Tang, D.: Nonlinear process monitoring based on kernel global-local preserving projections. J. Process Control. 38, 11–21 (2016). https://doi.org/10.1016/j.jprocont.2015.12.005

    Article  Google Scholar 

  55. Luo, L.; Bao, S.; Mao, J.; Tang, D.: Fault detection and diagnosis based on sparse PCA and two-level contribution plots. Ind. Eng. Chem. Res. 56, 225–240 (2017). https://doi.org/10.1021/acs.iecr.6b01500

    Article  Google Scholar 

  56. Meng, S.; Tong, C.; Lan, T.; Yu, H.: Canonical correlation analysis-based explicit relation discovery for statistical process monitoring. J. Franklin Inst. (2020). https://doi.org/10.1016/j.jfranklin.2020.01.049s

    Article  MATH  Google Scholar 

  57. Ge, Z.; Song, Z.: Performance-driven ensemble learning ICA model for improved non-Gaussian process monitoring. Chemom. Intell. Lab. Syst. 123, 1–8 (2013). https://doi.org/10.1016/j.chemolab.2013.02.001

    Article  Google Scholar 

  58. Li, N.; Yang, Y.: Using semi-nonnegative matrix underapproximation for statistical process monitoring. Chemom. Intell. Lab. Syst. 153, 126–139 (2016). https://doi.org/10.1016/j.chemolab.2016.03.006

    Article  Google Scholar 

  59. Ge, Z.; Zhang, M.; Song, Z.: Nonlinear process monitoring based on linear subspace and Bayesian inference. J. Process Control. 20, 676–688 (2010). https://doi.org/10.1016/j.jprocont.2010.03.003

    Article  Google Scholar 

  60. Jiang, Q.; Yan, X.: Just-in-time reorganized PCA integrated with SVDD for chemical process monitoring. AIChE J. 60, 949–965 (2014). https://doi.org/10.1002/aic.14335

    Article  Google Scholar 

  61. Xiao, Y.; Wang, H.; Xu, W.; Zhou, J.: Robust one-class SVM for fault detection. Chemom. Intell. Lab. Syst. 151, 15–25 (2016). https://doi.org/10.1016/j.chemolab.2015.11.010

    Article  Google Scholar 

  62. Tong, C.; Shi, X.; Lan, T.: Statistical process monitoring based on orthogonal multi-manifold projections and a novel variable contribution analysis. ISA Trans. 65, 407–417 (2016). https://doi.org/10.1016/j.isatra.2016.06.017

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ajaya Kumar Pani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maran Beena, A., Pani, A.K. Fault Detection of Complex Processes Using nonlinear Mean Function Based Gaussian Process Regression: Application to the Tennessee Eastman Process. Arab J Sci Eng 46, 6369–6390 (2021). https://doi.org/10.1007/s13369-020-05052-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-020-05052-x

Keywords

Navigation