Abstract
Fault detection and isolation (FDI) is considered as one of the most critical problems in biological processes. Therefore, in this paper, we consider a new FDI framework that aims to improve the monitoring of biological processes. To do that, a machine learning-based statistical hypothesis approach, which can identify the model, detect and isolate the faults, will be developed. In the developed approach, so-called partial Gaussian process regression (PGPR)-based generalized likelihood ratio test (GLRT), first, the GPR model that can accurately model biological processes is presented. Then, the fault detection phase is performed using the GLRT chart. Finally, the PGPR-based GLRT, which can effectively isolate the faults, is developed. The FDI performances of the developed PGPR-based GLRT approach are compared with partial support vector regression (SVR), extreme learning machines (ELM), Kernel ridge regression (KRR) and relevance vector machines (RVM)-based GLRT methods in terms of missed detection rate (MDR), false alarm rate (FAR), root mean square error (RMSE), execution time (ET) and isolation accuracy. The obtained results show that the proposed technique can reliably detect and isolate various faults using two examples: a synthetic data and a biological process representing a Cad System in E. coli (CSEC) model.
Similar content being viewed by others
References
Mansouri M, Nounou MN, Nounou HN (2017) Improved statistical fault detection technique and application to biological phenomena modeled by s-systems. IEEE Trans Nanobiosci 16(6):504–512
Mansouri M, Nounou MN, Nounou HN (2017) Multiscale kernel PLS-based exponentially weighted-GLRT and its application to fault detection. IEEE Trans Emerg Top Comput Intell 3(1):49–58
Stetco A, Dinmohammadi F, Zhao X, Robu V, Flynn D, Barnes M, Keane J, Nenadic G (2019) Machine learning methods for wind turbine condition monitoring: a review. Renew Energy 133:620–635
Tax DM, Ypma A, Duin RP (1999) Pump failure detection using support vector data descriptions. In: International symposium on intelligent data analysis. Springer, New York, pp 415–425
Yin Z, Hou J (2016) Recent advances on svm based fault diagnosis and process monitoring in complicated industrial processes. Neurocomputing 174:643–650
Matić D, Kulić F, Pineda-Sánchez M, Kamenko I (2012) Support vector machine classifier for diagnosis in electrical machines: application to broken bar. Expert Syst Appl 39(10):8681–8689
Widodo A, Yang B-S (2007) Support vector machine in machine condition monitoring and fault diagnosis. Mech Syst Signal Process 21(6):2560–2574
Liu R, Yang B, Zio E, Chen X (2018) Artificial intelligence for fault diagnosis of rotating machinery: a review. Mech Syst Signal Process 108:33–47
Heo S, Lee JH (2018) Fault detection and classification using artificial neural networks. IFAC-PapersOnLine 51(18):470–475
Yang Q, Li J, Le Blond S, Wang C (2016) Artificial neural network based fault detection and fault location in the DC microgrid. Energy Proc 103:129–134
Zhao H, Lai Z (2019) Neighborhood preserving neural network for fault detection. Neural Netw 109:6–18
Koppen-Seliger B, Frank P (1995) Fault detection and isolation in technical processes with neural networks. In: Proceedings of 1995 34th IEEE conference on decision and control, vol 3, IEEE, pp 2414–2419
Pan T-H, Wong DS-H, Jang S-S (2010) Development of a novel soft sensor using a local model network with an adaptive subtractive clustering approach. Ind Eng Chem Res 49(10):4738–4747
Pang J, Liu D, Liao H, Peng Y, Peng X (2014) Anomaly detection based on data stream monitoring and prediction with improved gaussian process regression algorithm. In: 2014 International conference on prognostics and health management, IEEE, pp 1–7
Fazai R, Abodayeh K, Mansouri M, Trabelsi M, Nounou H, Nounou M, Georghiou G (2019) Machine learning-based statistical testing hypothesis for fault detection in photovoltaic systems. Sol Energy 190:405–413
Harkat M-F, Mansouri M, Nounou M, Nounou H (2018) Enhanced data validation strategy of air quality monitoring network. Environ Res 160:183–194
Tharatipyakul A, Numnark S, Wichadakul D, Ingsriswang S (2012) Chemex: information extraction system for chemical data curation. In: BMC bioinformatics, vol 13, BioMed Central, p S9
Park J, Rosania GR, Shedden KA, Nguyen M, Lyu N, Saitou K (2009) Automated extraction of chemical structure information from digital raster images. Chem Cent J 3(1):4
Muñoz CA, Telen D, Nimmegeers P, Van Impe J (2018) Feature extraction for batch process monitoring and fault detection via simultaneous data scaling and training of tensor based models. IFAC-PapersOnLine 51(24):433–440
Fiannaca A, La Rosa M, La Paglia L, Rizzo R, Urso A (2017) nrc: non-coding RNA classifier based on structural features. BioData Mining 10(1):27
Hira ZM, Gillies DF (2015) A review of feature selection and feature extraction methods applied on microarray data. Adv Bioinform 2015:13
Björne J, Heimonen J, Ginter F, Airola A, Pahikkala T, Salakoski T (2011) Extracting contextualized complex biological events with rich graph-based feature sets. Comput Intell 27(4):541–557
Gamboa-Medina MM, Reis LR, Guido RC (2014) Feature extraction in pressure signals for leak detection in water networks. Proc Eng 70:688–697
Wu H, Liu C, Zhang Y, Sun W, Li W (2013) Building a water feature extraction model by integrating aerial image and lidar point clouds. Int J Remote Sens 34(21):7691–7705
Liu J, Wang J, Liu S, Qian X (2018) Feature extraction and identification of leak acoustic signal in water supply pipelines using correlation analysis and lyapunov exponent. Vibroeng Proc 19:182–187
Dutta S, Overbye TJ (2013) Feature extraction and visualization of power system transient stability results. IEEE Trans Power Syst 29(2):966–973
Erişti H, Uçar A, Demir Y (2010) Wavelet-based feature extraction and selection for classification of power system disturbances using support vector machines. Electr Power Syst Res 80(7):743–752
Almeida A, Almeida O, Junior B, Barreto L, Barros A (2017) Ica feature extraction for the location and classification of faults in high-voltage transmission lines. Electr Power Syst Res 148:254–263
Blaszczyk P, Stapor K (2009) A new feature extraction method based on the partial least squares algorithm and its applications. In: Computer recognition systems, vol 3. Springer, New York, pp 179–186
Segreto T, Simeone A, Teti R (2014) Principal component analysis for feature extraction and nn pattern recognition in sensor monitoring of chip form during turning. CIRP J Manuf Sci Technol 7(3):202–209
Wang W, Zhang M, Wang D, Jiang Y (2017) Kernel PCA feature extraction and the svm classification algorithm for multiple-status, through-wall, human being detection. EURASIP J Wirel Commun Netw 2017(1):1–7
Rosipal R, Trejo LJ (2001) Kernel partial least squares regression in reproducing kernel hilbert space. J Mach Learn Res 2:97–123
Dhanjal C, Gunn SR, Shawe-Taylor J (2008) Efficient sparse kernel feature extraction based on partial least squares. IEEE Trans Pattern Anal Mach Intell 31(8):1347–1361
Li S, Liao C, Kwok JT (2006) Gene feature extraction using t-test statistics and kernel partial least squares. In: International conference on neural information processing, Springer, pp 11–20
Pilario KE, Shafiee M, Cao Y, Lao L, Yang S-H (2020) A review of kernel methods for feature extraction in nonlinear process monitoring. Processes 8(1):24
Silva AF, Vercruysse J, Vervaet C, Remon JP, Lopes JA, De Beer T, Sarraguça MC (2019) In-depth evaluation of data collected during a continuous pharmaceutical manufacturing process: a multivariate statistical process monitoring approach. J Pharm Sci 108(1):439–450
Bersimis S, Sgora A, Psarakis S (2018) The application of multivariate statistical process monitoring in non-industrial processes. Qual Technol Quant Manag 15(4):526–549
Guo W, Shao C, Kim TH, Hu SJ, Jin JJ, Spicer JP, Wang H (2016) Online process monitoring with near-zero misdetection for ultrasonic welding of lithium-ion batteries: an integration of univariate and multivariate methods. J Manuf Syst 38:141–150
Antunes ACL, Dorea F, Halasa T, Toft N (2016) Monitoring endemic livestock diseases using laboratory diagnostic data: a simulation study to evaluate the performance of univariate process monitoring control algorithms. Prev Vet Med 127:15–20
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Jiang L, Zhang L, Yu L, Wang D (2019) Class-specific attribute weighted naive bayes. Pattern Recogn 88:321–330
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58(1):267–288
Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Stat Methodol) 67(2):301–320
Quinlan JR (1987) Simplifying decision trees. Int J Man Mach Stud 27(3):221–234
Steinberg D, Colla P (2009) Cart: classification and regression trees. Top Ten Algorithms Data Mining 9:179
Hamed MM, Khalafallah MG, Hassanien EA (2004) Prediction of wastewater treatment plant performance using artificial neural networks. Environ Model Softw 19(10):919–928
Nasr MS, Moustafa MA, Seif HA, El Kobrosy G (2012) Application of artificial neural network (ANN) for the prediction of el-agamy wastewater treatment plant performance-egypt. Alex Eng J 51(1):37–43
Djebbar Y, Narbaitz R (2002) Neural network prediction of air stripping KLA. J Environ Eng 128(5):451–460
Moreno-Alfonso N, Redondo C (2001) Intelligent waste-water treatment with neural-networks. Water Policy 3(3):267–271
Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Huang G, Huang G-B, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48
Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recogn 38(10):1759–1763
Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529
Suguna N, Thanushkodi K (2010) An improved k-nearest neighbor classification using genetic algorithm. Int J Comput Sci Issues 7(2):18–21
Wang Y, Pan Z, Pan Y (2019) A training data set cleaning method by classification ability ranking for the k-nearest neighbor classifier. IEEE Trans Neural Netw Learn Syst 31(5):1544–1556
Thanh Noi P, Kappas M (2018) Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using sentinel-2 imagery. Sensors 18(1):18
Jia W, Deng Y, Xin C, Liu X, Pedrycz W (2019) A classification algorithm with linear discriminant analysis and axiomatic fuzzy sets. Math Found Comput 2(1):73–81
Ullah A, Zinde-Walsh V (1984) On the robustness of LM, LR, and W tests in regression models. Econ J Econ Soc 52(4):1055–1066
Hahne JM, Biessmann F, Jiang N, Rehbaum H, Farina D, Meinecke F, Müller K-R, Parra L (2014) Linear and nonlinear regression techniques for simultaneous and proportional myoelectric control. IEEE Trans Neural Syst Rehabil Eng 22(2):269–279
Bishop CM, Tipping ME (2000) Variational relevance vector machines. In: Proceedings of the sixteenth conference on uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., pp 46–53
Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260
Ogutu JO, Schulz-Streeck, T, Piepho H-P (2012) Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. In: BMC proceedings, vol 6, BioMed Central, p S10
Nguyen-Tuong D, Seeger M, Peters J (2009) Model learning with local gaussian process regression. Adv Robot 23(15):2015–2034
Smola AJ, Bartlett PL (2001) Sparse greedy gaussian process regression. In: Advances in neural information processing systems, pp 619–625
Sheng H, Xiao J, Cheng Y, Ni Q, Wang S (2017) Short-term solar power forecasting based on weighted gaussian process regression. IEEE Trans Ind Electron 65(1):300–308
Rohani A, Taki M, Abdollahpour M (2018) A novel soft computing model (gaussian process regression with k-fold cross validation) for daily and monthly solar radiation forecasting (part: I). Renew Energy 115:411–422
Verrelst J, Alonso L, Camps-Valls G, Delegido J, Moreno J (2012) Retrieval of vegetation biophysical parameters using gaussian process techniques. IEEE Trans Geosci Remote Sens 50(5):1832–1843
Lázaro-Gredilla M, Titsias MK, Verrelst J, Camps-Valls G (2014) Retrieval of biophysical parameters with heteroscedastic gaussian processes. IEEE Geosci Remote Sens Lett 11(4):838–842
Chihi I, Benrejeb M (2018) Online fault detection approach of unpredictable inputs: application to handwriting system. Complexity 2018:12–24
Mansouri M, Baklouti R, Harkat MF, Nounou M, Nounou H, Hamida AB (2018) Kernel generalized likelihood ratio test for fault detection of biological systems. IEEE Trans Nanobiosci 17(4):498–506
Lall P, Gupta P, Kulkarni M, Panchagade D, Suhling J, Hofmeister J (2008) Prognostication and health monitoring of electronics in implantable biological systems. In: ASME 2008 international mechanical engineering congress and exposition, American Society of Mechanical Engineers Digital Collection, pp 657–671
Mansouri M, Harkat MF, Teh SY, Al-khazraji A, Nounou H, Nounou M (2018) Model-based and data-driven with multiscale sum of squares double EWMA control chart for fault detection in biological systems. J Chemom 32(12):e3068
Williams CK, Rasmussen CE (1996) Gaussian processes for regression. In: Advances in neural information processing systems, pp 514–520
Rasmussen CE (2003) Gaussian processes in machine learning. In: Summer school on machine learning, Springer, pp 63–71
Yuan F, Xia X, Shi J, Li H, Li G (2017) Non-linear dimensionality reduction and gaussian process based classification method for smoke detection. IEEE Access 5:6833–6841
Fazai R, Mansouri M, Abodayeh K, Nounou H, Nounou M (2019) Online reduced kernel PLS combined with GLRT for fault detection in chemical systems. Process Saf Environ Prot 128:228–243
Fazai R, Mansouri M, Abodayeh K, Puig V, Raouf M-IN, Nounou H, Nounou M (2019) Multiscale gaussian process regression-based generalized likelihood ratio test for fault detection in water distribution networks. Eng Appl Artif Intell 85:474–491
Gustafsson F (1996) The marginalized likelihood ratio test for detecting abrupt changes. IEEE Trans Autom Control 41(1):66–78
Willsky A, Chow E, Gershwin S, Greene C, Houpt P, Kurkjian A (1980) Dynamic model-based techniques for the detection of incidents on freeways. IEEE Trans Autom Control 25(3):347–360
Wei X, Liu H, Qin Y (2011) Fault diagnosis of rail vehicle suspension systems by using GLRT. In: 2011 Chinese control and decision conference (CCDC). IEEE, pp 1932–1936
Harkat M-F, Mansouri M, Nounou MN, Nounou HN (2019) Fault detection of uncertain chemical processes using interval partial least squares-based generalized likelihood ratio test. Inf Sci 490:265–284
Fezai R, Mansouri M, Trabelsi M, Hajji M, Nounou H, Nounou M (2019) Online reduced kernel GLRT technique for improved fault detection in photovoltaic systems. Energy 179:1133–1154
Gertler J, Cao J (2005) Design of optimal structured residuals from partial principal component models for fault diagnosis in linear systems. J Process Control 15(5):585–603
Gertler J, Li W, Huang Y, McAvoy T (1999) Isolation enhanced principal component analysis. AIChE J 45(2):323–334. https://doi.org/10.1002/aic.690450213
Huang Y, Gertler J, McAvoy TJ (2000) Sensor and actuator fault isolation by structured partial PCA with nonlinear extensions. J Process Control 10(5):459–469
Gonzalez OR et al (2007) Parameter estimation using simulated annealing for s-system models of biochemical networks. Bioinformatics 23(4):480–486
Acknowledgements
This work was made possible by NPRP Grant NPRP9-330-2-140 from the Qatar National Research Fund (a member of Qatar Foundation).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fezai, R., Abodayeh, K., Mansouri, M. et al. Fault diagnosis of biological systems using improved machine learning technique. Int. J. Mach. Learn. & Cyber. 12, 515–528 (2021). https://doi.org/10.1007/s13042-020-01184-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-020-01184-6