Skip to main content
Log in

Fault diagnosis of biological systems using improved machine learning technique

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Fault detection and isolation (FDI) is considered as one of the most critical problems in biological processes. Therefore, in this paper, we consider a new FDI framework that aims to improve the monitoring of biological processes. To do that, a machine learning-based statistical hypothesis approach, which can identify the model, detect and isolate the faults, will be developed. In the developed approach, so-called partial Gaussian process regression (PGPR)-based generalized likelihood ratio test (GLRT), first, the GPR model that can accurately model biological processes is presented. Then, the fault detection phase is performed using the GLRT chart. Finally, the PGPR-based GLRT, which can effectively isolate the faults, is developed. The FDI performances of the developed PGPR-based GLRT approach are compared with partial support vector regression (SVR), extreme learning machines (ELM), Kernel ridge regression (KRR) and relevance vector machines (RVM)-based GLRT methods in terms of missed detection rate (MDR), false alarm rate (FAR), root mean square error (RMSE), execution time (ET) and isolation accuracy. The obtained results show that the proposed technique can reliably detect and isolate various faults using two examples: a synthetic data and a biological process representing a Cad System in E. coli (CSEC) model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Mansouri M, Nounou MN, Nounou HN (2017) Improved statistical fault detection technique and application to biological phenomena modeled by s-systems. IEEE Trans Nanobiosci 16(6):504–512

    Google Scholar 

  2. Mansouri M, Nounou MN, Nounou HN (2017) Multiscale kernel PLS-based exponentially weighted-GLRT and its application to fault detection. IEEE Trans Emerg Top Comput Intell 3(1):49–58

    Google Scholar 

  3. Stetco A, Dinmohammadi F, Zhao X, Robu V, Flynn D, Barnes M, Keane J, Nenadic G (2019) Machine learning methods for wind turbine condition monitoring: a review. Renew Energy 133:620–635

    Google Scholar 

  4. Tax DM, Ypma A, Duin RP (1999) Pump failure detection using support vector data descriptions. In: International symposium on intelligent data analysis. Springer, New York, pp 415–425

  5. Yin Z, Hou J (2016) Recent advances on svm based fault diagnosis and process monitoring in complicated industrial processes. Neurocomputing 174:643–650

    Google Scholar 

  6. Matić D, Kulić F, Pineda-Sánchez M, Kamenko I (2012) Support vector machine classifier for diagnosis in electrical machines: application to broken bar. Expert Syst Appl 39(10):8681–8689

    Google Scholar 

  7. Widodo A, Yang B-S (2007) Support vector machine in machine condition monitoring and fault diagnosis. Mech Syst Signal Process 21(6):2560–2574

    Google Scholar 

  8. Liu R, Yang B, Zio E, Chen X (2018) Artificial intelligence for fault diagnosis of rotating machinery: a review. Mech Syst Signal Process 108:33–47

    Google Scholar 

  9. Heo S, Lee JH (2018) Fault detection and classification using artificial neural networks. IFAC-PapersOnLine 51(18):470–475

    Google Scholar 

  10. Yang Q, Li J, Le Blond S, Wang C (2016) Artificial neural network based fault detection and fault location in the DC microgrid. Energy Proc 103:129–134

    Google Scholar 

  11. Zhao H, Lai Z (2019) Neighborhood preserving neural network for fault detection. Neural Netw 109:6–18

    Google Scholar 

  12. Koppen-Seliger B, Frank P (1995) Fault detection and isolation in technical processes with neural networks. In: Proceedings of 1995 34th IEEE conference on decision and control, vol 3, IEEE, pp 2414–2419

  13. Pan T-H, Wong DS-H, Jang S-S (2010) Development of a novel soft sensor using a local model network with an adaptive subtractive clustering approach. Ind Eng Chem Res 49(10):4738–4747

    Google Scholar 

  14. Pang J, Liu D, Liao H, Peng Y, Peng X (2014) Anomaly detection based on data stream monitoring and prediction with improved gaussian process regression algorithm. In: 2014 International conference on prognostics and health management, IEEE, pp 1–7

  15. Fazai R, Abodayeh K, Mansouri M, Trabelsi M, Nounou H, Nounou M, Georghiou G (2019) Machine learning-based statistical testing hypothesis for fault detection in photovoltaic systems. Sol Energy 190:405–413

    Google Scholar 

  16. Harkat M-F, Mansouri M, Nounou M, Nounou H (2018) Enhanced data validation strategy of air quality monitoring network. Environ Res 160:183–194

    Google Scholar 

  17. Tharatipyakul A, Numnark S, Wichadakul D, Ingsriswang S (2012) Chemex: information extraction system for chemical data curation. In: BMC bioinformatics, vol 13, BioMed Central, p S9

  18. Park J, Rosania GR, Shedden KA, Nguyen M, Lyu N, Saitou K (2009) Automated extraction of chemical structure information from digital raster images. Chem Cent J 3(1):4

    Google Scholar 

  19. Muñoz CA, Telen D, Nimmegeers P, Van Impe J (2018) Feature extraction for batch process monitoring and fault detection via simultaneous data scaling and training of tensor based models. IFAC-PapersOnLine 51(24):433–440

    Google Scholar 

  20. Fiannaca A, La Rosa M, La Paglia L, Rizzo R, Urso A (2017) nrc: non-coding RNA classifier based on structural features. BioData Mining 10(1):27

    Google Scholar 

  21. Hira ZM, Gillies DF (2015) A review of feature selection and feature extraction methods applied on microarray data. Adv Bioinform 2015:13

    Google Scholar 

  22. Björne J, Heimonen J, Ginter F, Airola A, Pahikkala T, Salakoski T (2011) Extracting contextualized complex biological events with rich graph-based feature sets. Comput Intell 27(4):541–557

    MathSciNet  Google Scholar 

  23. Gamboa-Medina MM, Reis LR, Guido RC (2014) Feature extraction in pressure signals for leak detection in water networks. Proc Eng 70:688–697

    Google Scholar 

  24. Wu H, Liu C, Zhang Y, Sun W, Li W (2013) Building a water feature extraction model by integrating aerial image and lidar point clouds. Int J Remote Sens 34(21):7691–7705

    Google Scholar 

  25. Liu J, Wang J, Liu S, Qian X (2018) Feature extraction and identification of leak acoustic signal in water supply pipelines using correlation analysis and lyapunov exponent. Vibroeng Proc 19:182–187

    Google Scholar 

  26. Dutta S, Overbye TJ (2013) Feature extraction and visualization of power system transient stability results. IEEE Trans Power Syst 29(2):966–973

    Google Scholar 

  27. Erişti H, Uçar A, Demir Y (2010) Wavelet-based feature extraction and selection for classification of power system disturbances using support vector machines. Electr Power Syst Res 80(7):743–752

    Google Scholar 

  28. Almeida A, Almeida O, Junior B, Barreto L, Barros A (2017) Ica feature extraction for the location and classification of faults in high-voltage transmission lines. Electr Power Syst Res 148:254–263

    Google Scholar 

  29. Blaszczyk P, Stapor K (2009) A new feature extraction method based on the partial least squares algorithm and its applications. In: Computer recognition systems, vol 3. Springer, New York, pp 179–186

  30. Segreto T, Simeone A, Teti R (2014) Principal component analysis for feature extraction and nn pattern recognition in sensor monitoring of chip form during turning. CIRP J Manuf Sci Technol 7(3):202–209

    Google Scholar 

  31. Wang W, Zhang M, Wang D, Jiang Y (2017) Kernel PCA feature extraction and the svm classification algorithm for multiple-status, through-wall, human being detection. EURASIP J Wirel Commun Netw 2017(1):1–7

    Google Scholar 

  32. Rosipal R, Trejo LJ (2001) Kernel partial least squares regression in reproducing kernel hilbert space. J Mach Learn Res 2:97–123

    MATH  Google Scholar 

  33. Dhanjal C, Gunn SR, Shawe-Taylor J (2008) Efficient sparse kernel feature extraction based on partial least squares. IEEE Trans Pattern Anal Mach Intell 31(8):1347–1361

    Google Scholar 

  34. Li S, Liao C, Kwok JT (2006) Gene feature extraction using t-test statistics and kernel partial least squares. In: International conference on neural information processing, Springer, pp 11–20

  35. Pilario KE, Shafiee M, Cao Y, Lao L, Yang S-H (2020) A review of kernel methods for feature extraction in nonlinear process monitoring. Processes 8(1):24

    Google Scholar 

  36. Silva AF, Vercruysse J, Vervaet C, Remon JP, Lopes JA, De Beer T, Sarraguça MC (2019) In-depth evaluation of data collected during a continuous pharmaceutical manufacturing process: a multivariate statistical process monitoring approach. J Pharm Sci 108(1):439–450

    Google Scholar 

  37. Bersimis S, Sgora A, Psarakis S (2018) The application of multivariate statistical process monitoring in non-industrial processes. Qual Technol Quant Manag 15(4):526–549

    Google Scholar 

  38. Guo W, Shao C, Kim TH, Hu SJ, Jin JJ, Spicer JP, Wang H (2016) Online process monitoring with near-zero misdetection for ultrasonic welding of lithium-ion batteries: an integration of univariate and multivariate methods. J Manuf Syst 38:141–150

    Google Scholar 

  39. Antunes ACL, Dorea F, Halasa T, Toft N (2016) Monitoring endemic livestock diseases using laboratory diagnostic data: a simulation study to evaluate the performance of univariate process monitoring control algorithms. Prev Vet Med 127:15–20

    Google Scholar 

  40. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  41. Jiang L, Zhang L, Yu L, Wang D (2019) Class-specific attribute weighted naive bayes. Pattern Recogn 88:321–330

    Google Scholar 

  42. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58(1):267–288

    MathSciNet  MATH  Google Scholar 

  43. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Stat Methodol) 67(2):301–320

    MathSciNet  MATH  Google Scholar 

  44. Quinlan JR (1987) Simplifying decision trees. Int J Man Mach Stud 27(3):221–234

    Google Scholar 

  45. Steinberg D, Colla P (2009) Cart: classification and regression trees. Top Ten Algorithms Data Mining 9:179

    Google Scholar 

  46. Hamed MM, Khalafallah MG, Hassanien EA (2004) Prediction of wastewater treatment plant performance using artificial neural networks. Environ Model Softw 19(10):919–928

    Google Scholar 

  47. Nasr MS, Moustafa MA, Seif HA, El Kobrosy G (2012) Application of artificial neural network (ANN) for the prediction of el-agamy wastewater treatment plant performance-egypt. Alex Eng J 51(1):37–43

    Google Scholar 

  48. Djebbar Y, Narbaitz R (2002) Neural network prediction of air stripping KLA. J Environ Eng 128(5):451–460

    Google Scholar 

  49. Moreno-Alfonso N, Redondo C (2001) Intelligent waste-water treatment with neural-networks. Water Policy 3(3):267–271

    Google Scholar 

  50. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Google Scholar 

  51. Huang G, Huang G-B, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    MATH  Google Scholar 

  52. Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recogn 38(10):1759–1763

    MATH  Google Scholar 

  53. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529

    Google Scholar 

  54. Suguna N, Thanushkodi K (2010) An improved k-nearest neighbor classification using genetic algorithm. Int J Comput Sci Issues 7(2):18–21

    Google Scholar 

  55. Wang Y, Pan Z, Pan Y (2019) A training data set cleaning method by classification ability ranking for the k-nearest neighbor classifier. IEEE Trans Neural Netw Learn Syst 31(5):1544–1556

    Google Scholar 

  56. Thanh Noi P, Kappas M (2018) Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using sentinel-2 imagery. Sensors 18(1):18

    Google Scholar 

  57. Jia W, Deng Y, Xin C, Liu X, Pedrycz W (2019) A classification algorithm with linear discriminant analysis and axiomatic fuzzy sets. Math Found Comput 2(1):73–81

    Google Scholar 

  58. Ullah A, Zinde-Walsh V (1984) On the robustness of LM, LR, and W tests in regression models. Econ J Econ Soc 52(4):1055–1066

    MathSciNet  MATH  Google Scholar 

  59. Hahne JM, Biessmann F, Jiang N, Rehbaum H, Farina D, Meinecke F, Müller K-R, Parra L (2014) Linear and nonlinear regression techniques for simultaneous and proportional myoelectric control. IEEE Trans Neural Syst Rehabil Eng 22(2):269–279

    Google Scholar 

  60. Bishop CM, Tipping ME (2000) Variational relevance vector machines. In: Proceedings of the sixteenth conference on uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., pp 46–53

  61. Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260

    MathSciNet  MATH  Google Scholar 

  62. Ogutu JO, Schulz-Streeck, T, Piepho H-P (2012) Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. In: BMC proceedings, vol 6, BioMed Central, p S10

  63. Nguyen-Tuong D, Seeger M, Peters J (2009) Model learning with local gaussian process regression. Adv Robot 23(15):2015–2034

    Google Scholar 

  64. Smola AJ, Bartlett PL (2001) Sparse greedy gaussian process regression. In: Advances in neural information processing systems, pp 619–625

  65. Sheng H, Xiao J, Cheng Y, Ni Q, Wang S (2017) Short-term solar power forecasting based on weighted gaussian process regression. IEEE Trans Ind Electron 65(1):300–308

    Google Scholar 

  66. Rohani A, Taki M, Abdollahpour M (2018) A novel soft computing model (gaussian process regression with k-fold cross validation) for daily and monthly solar radiation forecasting (part: I). Renew Energy 115:411–422

    Google Scholar 

  67. Verrelst J, Alonso L, Camps-Valls G, Delegido J, Moreno J (2012) Retrieval of vegetation biophysical parameters using gaussian process techniques. IEEE Trans Geosci Remote Sens 50(5):1832–1843

    Google Scholar 

  68. Lázaro-Gredilla M, Titsias MK, Verrelst J, Camps-Valls G (2014) Retrieval of biophysical parameters with heteroscedastic gaussian processes. IEEE Geosci Remote Sens Lett 11(4):838–842

    Google Scholar 

  69. Chihi I, Benrejeb M (2018) Online fault detection approach of unpredictable inputs: application to handwriting system. Complexity 2018:12–24

    Google Scholar 

  70. Mansouri M, Baklouti R, Harkat MF, Nounou M, Nounou H, Hamida AB (2018) Kernel generalized likelihood ratio test for fault detection of biological systems. IEEE Trans Nanobiosci 17(4):498–506

    Google Scholar 

  71. Lall P, Gupta P, Kulkarni M, Panchagade D, Suhling J, Hofmeister J (2008) Prognostication and health monitoring of electronics in implantable biological systems. In: ASME 2008 international mechanical engineering congress and exposition, American Society of Mechanical Engineers Digital Collection, pp 657–671

  72. Mansouri M, Harkat MF, Teh SY, Al-khazraji A, Nounou H, Nounou M (2018) Model-based and data-driven with multiscale sum of squares double EWMA control chart for fault detection in biological systems. J Chemom 32(12):e3068

    Google Scholar 

  73. Williams CK, Rasmussen CE (1996) Gaussian processes for regression. In: Advances in neural information processing systems, pp 514–520

  74. Rasmussen CE (2003) Gaussian processes in machine learning. In: Summer school on machine learning, Springer, pp 63–71

  75. Yuan F, Xia X, Shi J, Li H, Li G (2017) Non-linear dimensionality reduction and gaussian process based classification method for smoke detection. IEEE Access 5:6833–6841

    Google Scholar 

  76. Fazai R, Mansouri M, Abodayeh K, Nounou H, Nounou M (2019) Online reduced kernel PLS combined with GLRT for fault detection in chemical systems. Process Saf Environ Prot 128:228–243

    Google Scholar 

  77. Fazai R, Mansouri M, Abodayeh K, Puig V, Raouf M-IN, Nounou H, Nounou M (2019) Multiscale gaussian process regression-based generalized likelihood ratio test for fault detection in water distribution networks. Eng Appl Artif Intell 85:474–491

    Google Scholar 

  78. Gustafsson F (1996) The marginalized likelihood ratio test for detecting abrupt changes. IEEE Trans Autom Control 41(1):66–78

    MathSciNet  MATH  Google Scholar 

  79. Willsky A, Chow E, Gershwin S, Greene C, Houpt P, Kurkjian A (1980) Dynamic model-based techniques for the detection of incidents on freeways. IEEE Trans Autom Control 25(3):347–360

    MATH  Google Scholar 

  80. Wei X, Liu H, Qin Y (2011) Fault diagnosis of rail vehicle suspension systems by using GLRT. In: 2011 Chinese control and decision conference (CCDC). IEEE, pp 1932–1936

  81. Harkat M-F, Mansouri M, Nounou MN, Nounou HN (2019) Fault detection of uncertain chemical processes using interval partial least squares-based generalized likelihood ratio test. Inf Sci 490:265–284

    MathSciNet  MATH  Google Scholar 

  82. Fezai R, Mansouri M, Trabelsi M, Hajji M, Nounou H, Nounou M (2019) Online reduced kernel GLRT technique for improved fault detection in photovoltaic systems. Energy 179:1133–1154

    Google Scholar 

  83. Gertler J, Cao J (2005) Design of optimal structured residuals from partial principal component models for fault diagnosis in linear systems. J Process Control 15(5):585–603

    Google Scholar 

  84. Gertler J, Li W, Huang Y, McAvoy T (1999) Isolation enhanced principal component analysis. AIChE J 45(2):323–334. https://doi.org/10.1002/aic.690450213

    Article  Google Scholar 

  85. Huang Y, Gertler J, McAvoy TJ (2000) Sensor and actuator fault isolation by structured partial PCA with nonlinear extensions. J Process Control 10(5):459–469

    Google Scholar 

  86. Gonzalez OR et al (2007) Parameter estimation using simulated annealing for s-system models of biochemical networks. Bioinformatics 23(4):480–486

    Google Scholar 

Download references

Acknowledgements

This work was made possible by NPRP Grant NPRP9-330-2-140 from the Qatar National Research Fund (a member of Qatar Foundation).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Majdi Mansouri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fezai, R., Abodayeh, K., Mansouri, M. et al. Fault diagnosis of biological systems using improved machine learning technique. Int. J. Mach. Learn. & Cyber. 12, 515–528 (2021). https://doi.org/10.1007/s13042-020-01184-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01184-6

Keywords

Navigation