Skip to main content
SpringerLink
Log in

Deep learning for machine health prognostics using Kernel-based feature transformation

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Prognostic health management minimizes system downtime and improves overall equipment effectiveness. Accurate prediction of remaining useful life (RUL) is key to prognostics. Prominent machine learning algorithms implement handcrafted feature extraction to improve RUL prediction. Deep learning automates feature extraction from raw data but requires large datasets and computationally expensive fine-tuning. Data-specific handcrafting and fine-tuning limit the generalization capability of existing models. Proposed framework addresses these challenges using Temporal Multivariate 3D Convolutional Network (TM3C) and Kernel-based Transformation (KT) of features. KT generates 3D features that incorporate trendable degradation patterns from multivariate temporal relationship among sensor data. TM3C implements 3D convolutional layers with temporal filters for RUL prediction. KT is generalizable and improves feature relevance. Full-width filters in TM3C reduce number of tunable parameters and convolution operations. Proposed TM3C-KT capitalizes on the strength of deep learning while lowering the cost for feature discovery, parameter learning, and model fine-tuning. TM3C-KT is evaluated on three prognostics applications, (1) RUL prediction for turbofan engines, (2) Failure state estimation for hydraulic pumps, and (3) Component wear prediction for milling machines. Performance of the framework is comparable and better than benchmark methods in literature. Characteristics of the framework are reviewed on generalizability, prognosability and versatility metrics. Results and corresponding analysis demonstrate suitability of TM3C-KT for industrial applications of machine health prognostics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Agogino A, Goebel K (2007) Milling data set. http://ti.arc.nasa.gov/project/prognostic-data-repository, (visited on 2019-07-15)

  • Babu, G. S., Zhao, P., & Li, X. L. (2016). Deep convolutional neural network based regression approach for estimation of remaining useful life. Lecture Notes in Computer Science (including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9642, 214–228. https://doi.org/10.1007/978-3-319-32025-0_14.

    Article  Google Scholar 

  • Bengio, Y. (2012). Practical recommendations for gradient-based training of deep architectures. In Neural networks: Tricks of the trade (pp. 437–478). Germany: Springer.

    Book  Google Scholar 

  • Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers and Electrical Engineering, 40(1), 16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024.

    Article  Google Scholar 

  • Coble JB (2010) Merging data sources to predict remaining useful life - an automated method to identify prognostic parameters. PhD thesis, University of Tennessee

  • Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition

  • Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675–701. https://doi.org/10.1080/01621459.1937.10503522.

    Article  Google Scholar 

  • Gamboa JCB (2017) Deep learning for time-series analysis. http://arxiv.org/abs/170101887

  • Genton, M. G. (2002). Classes of kernels for machine learning: a statistics perspective. Journal of Machine Learning Research, 2, 299–312.

    Google Scholar 

  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Germany: Springer Science & Business Media.

    Book  Google Scholar 

  • Helwig N, Pignanelli E, Schutze A (2015) Condition monitoring of a complex hydraulic system using multivariate statistics. Conference Record - IEEE Instrumentation and Measurement Technology Conference 2015-July:210–215, https://doi.org/10.1109/I2MTC.2015.7151267

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.

    Article  Google Scholar 

  • Huang B, Di Y, Jin C, Lee J (2017) Review of data-driven prognostics and health management techniques: Lessons learned from PHM data challenge competitions. MFPT 2017 Annual Conference: 50 Years of Failure Prevention Technology Innovation pp 1–17

  • Huang, Z., Zhu, J., Lei, J., Li, X., & Tian, F. (2020). Tool wear predicting based on multi-domain feature fusion by deep convolutional neural network in milling operations. Journal of Intelligent Manufacturing, 31(4), 953–966. https://doi.org/10.1007/s10845-019-01488-7.

    Article  Google Scholar 

  • Jayasinghe L, Samarasinghe T, Yuen C, Low JCN, Ge SS (2018) Temporal convolutional memory networks for remaining useful life estimation of industrial machinery. arXiv:181005644

  • Ji, S., Xu, W., Yang, M., & Yu, K. (2013). 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 221–231. https://doi.org/10.1109/TPAMI.2012.59.

    Article  Google Scholar 

  • Khalid S, Khalil T, Nasreen S (2014) A survey of feature selection and feature extraction techniques in machine learning. In: Science and Information Conference, pp 1–8

  • Khan, S., & Yairi, T. (2018). A review on the application of deep learning in system health management. Mechanical Systems and Signal Processing, 107, 241–265. https://doi.org/10.1016/j.ymssp.2017.11.024.

    Article  Google Scholar 

  • Kim, T. S., & Sohn, S. Y. (2020). Multitask learning for health condition identification and remaining useful life prediction: Deep convolutional neural network approach. Journal of Intelligent Manufacturing,. https://doi.org/10.1007/s10845-020-01630-w.

    Article  Google Scholar 

  • Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images. Citeseer: Tech. rep.

    Google Scholar 

  • Kumar S, Torres M, Chan YC, Pecht M (2008) A hybrid prognostics methodology for electronic products. Proceedings of the International Joint Conference on Neural Networks pp 3479–3485, https://doi.org/10.1109/IJCNN.2008.4634294

  • Laredo, D., Chen, Z., Schütze, O., & Sun, J. Q. (2019). A neural network-evolutionary computational framework for remaining useful life estimation of mechanical systems. Neural Networks, 116, 178–187. https://doi.org/10.1016/j.neunet.2019.04.016.

    Article  Google Scholar 

  • Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018). Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing, 104, 799–834. https://doi.org/10.1016/j.ymssp.2017.11.016.

    Article  Google Scholar 

  • Li, X., Ding, Q., & Sun, J. Q. (2017). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering and System Safety, 172, 1–11. https://doi.org/10.1016/j.ress.2017.11.021.

    Article  Google Scholar 

  • Li, X., Zhang, W., & Ding, Q. (2018). Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction. Reliability Engineering and System Safety, 182, 208–218. https://doi.org/10.1016/j.ress.2018.11.011.

    Article  Google Scholar 

  • Liao, L., Jin, W., & Pavel, R. (2016). Prognosability regularization for prognostics and health assessment. IEEE Transactions on Industrial Electronics, 63, 7076–7083.

    Article  Google Scholar 

  • Listou Ellefsen, A., Bjørlykhaug, E., Æsøy, V., Ushakov, S., & Zhang, H. (2018). Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture. Reliability Engineering and System Safety, 183, 240–251. https://doi.org/10.1016/j.ress.2018.11.027.

    Article  Google Scholar 

  • Liu, Q., Basu, S., Ganguly, S., Mukhopadhyay, S., DiBiano, R., Karki, M., et al. (2020). DeepSat V2: Feature augmented convolutional neural nets for satellite image classification. Remote Sensing Letters, 11(2), 156–165. https://doi.org/10.1080/2150704X.2019.1693071.

    Article  Google Scholar 

  • Malhotra P, TV V, Ramakrishnan A, Anand G, Vig L, Agarwal P, Shroff G (2016) Multi-sensor prognostics using an unsupervised health index based on LSTM encoder-decoder. arXiv:160806154

  • Marwan, N., Carmen Romano, M., Thiel, M., & Kurths, J. (2007). Recurrence plots for the analysis of complex systems. Physics Reports, 438(5–6), 237–329. https://doi.org/10.1016/j.physrep.2006.11.001.

    Article  Google Scholar 

  • Naduvil-Vadukootu S, Angryk RA, Riley P (2017) Evaluating preprocessing strategies for time series prediction using deep learning architectures. FLAIRS 2017 - Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference pp 520–525

  • Nemenyi, P. B. (1963). Distribution-free multiple comparisons (doctoral dissertation, princeton university, 1963). Dissertation Abstracts International, 25(2), 1233.

  • Poggio, T., Mhaskar, H., Rosasco, L., Miranda, B., & Liao, Q. (2017). Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review. International Journal of Automation and Computing, 14(5), 503–519. https://doi.org/10.1007/s11633-017-1054-2.

    Article  Google Scholar 

  • Ramasso E, Saxena A (2014) Review and analysis of algorithmic approaches developed for prognostics on CMAPSS dataset. PHM 2014 - Proceedings of the Annual Conference of the Prognostics and Health Management Society 2014 pp 612–622

  • Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252. https://doi.org/10.1007/s11263-015-0816-y.

    Article  Google Scholar 

  • Sadouk, L. (2018). CNN approaches for time-series classification. In: Time series analysis-data, methods, and applications. London: IntechOpen.

    Google Scholar 

  • Saxena A, Goebel K, Simon D, Eklund N (2008) Damage propagation modeling for aircraft engine run-to-failure simulation. In: 2008 International Conference on Prognostics and Health Management, https://doi.org/10.1109/PHM.2008.4711414

  • Schneider, T., Helwig, N., & Schütze, A. (2017). Automatic feature extraction and selection for classification of cyclical time series data. Technisches Messen, 84(3), 198–206. https://doi.org/10.1515/teme-2016-0072.

    Article  Google Scholar 

  • Sharma, A., Vans, E., Shigemizu, D., Boroevich, K. A., & Tsunoda, T. (2019). DeepInsight: A methodology to transform a non-image data to an image for convolution neural network architecture. Scientific Reports, 9(1), 1–7. https://doi.org/10.1038/s41598-019-47765-6.

    Article  Google Scholar 

  • Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 2015-January:802–810

  • Souza C (2010) Kernel Functions for Machine Learning Applications. http://crsouza.com/2010/03/17/kernel-functions-for-machine-learning-applications/, (visited on 2020-01-25)

  • Sun L, Jia K, Yeung DY, Shi BE (2015) Human action recognition using factorized spatio-temporal convolutional networks. Proceedings of the IEEE International Conference on Computer Vision pp 4597–4605, https://doi.org/10.1109/ICCV.2015.522

  • Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 4489–4497

  • Vogl, G. W., Weiss, B. A., & Helu, M. (2019). A review of diagnostic and prognostic capabilities and best practices for manufacturing. Journal of Intelligent Manufacturing, 30(1), 79–95. https://doi.org/10.1007/s10845-016-1228-8.

    Article  Google Scholar 

  • Wienberger K (2018) Lecture 12: bias-variance tradeoff. http://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/lecturenote12.html, (visited on 2020-03-12)

  • Wu G, Chang EY, Panda N (2005) Formulating distance functions via the kernel trick. In: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp 703–709

  • Wu, Q., Ding, K., & Huang, B. (2020). Approach for fault prognosis using recurrent neural network. Journal of Intelligent Manufacturing, 31(7), 1621–1633. https://doi.org/10.1007/s10845-018-1428-5.

    Article  Google Scholar 

  • Yoon AS, Lee T, Lim Y, Jung D, Kang P, Kim D, Park K, Choi Y (2017) Semi-supervised Learning with Deep Generative Models for Asset Failure Prediction.

  • Yuan M, Wu Y, Lin L (2016) Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network. AUS 2016 - 2016 IEEE/CSAA International Conference on Aircraft Utility Systems pp 135–140, https://doi.org/10.1109/AUS.2016.7748035

  • Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2017a). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2306–2318. https://doi.org/10.1109/TNNLS.2016.2582798.

    Article  Google Scholar 

  • Zhang, W., Min-Ping Jia, B., Lin Zhu, B., & Xiao-An Yan, B. (2017b). Comprehensive overview on computational intelligence techniques for machinery condition monitoring and fault diagnosis. Chinese Journal of Mechanical Engineering,. https://doi.org/10.1007/s10033-017-0150-0.

  • Zhang, X., Xiao, P., Yang, Y., Cheng, Y., Chen, B., Gao, D., et al. (2019). Remaining useful life estimation using CNN-XGB with extended time window. IEEE Access, 7, 154386–154397. https://doi.org/10.1109/ACCESS.2019.2942991.

    Article  Google Scholar 

  • Zhao, R., Yan, R., Wang, J., & Mao, K. (2017). Learning to monitor machine health with convolutional Bi-directional LSTM networks. Sensors (Switzerland), 17(2), 1–18. https://doi.org/10.3390/s17020273.

    Article  Google Scholar 

  • Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237. https://doi.org/10.1016/j.ymssp.2018.05.050.

    Article  Google Scholar 

  • Zheng S, Ristovski K, Farahat A, Gupta C (2017) Long short-term memory network for remaining useful life estimation. 2017 IEEE International Conference on Prognostics and Health Management, ICPHM 2017 pp 88–95, https://doi.org/10.1109/ICPHM.2017.7998311

Download references

Author information

Authors and Affiliations

  1. National University of Singapore, 21 Lower Kent Ridge Rd, Singapore, 119077, Singapore

    Shanmugasivam Pillai & Prahlad Vadakkepat

Authors
  1. Shanmugasivam Pillai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prahlad Vadakkepat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shanmugasivam Pillai.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest in the research work presented.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

A TM3C model parameter configuration for different datasets

Data-specific fine-tuning of TM3C model parameters aids in improvement of prediction performance. Table 6 provides list of key parameter values for C-MAPSS subsets, UCI Hydraulics, and milling machine data.

B An example illustrating the effect of model complexity and number of features on prediction performance

Example below demonstrates the effect of model complexity and feature transformation on the prediction performance of a neural network based model. A bivariate regression problem is defined where the output is a sinusoidal function of the sum of inputs (Fig. 10a, c). An area of the input data range is selected from which 200 samples are randomly picked for training the model. Different model variants are generated by modifying the number of neurons and layers. New features are generated by applying transformation functions on input variables (Fig. 10b). It is observed that, increasing model complexity and number of features help improve model performance (M2-F2 in Fig. 10). However, overly complex models or too many features lead to overfitting (M3-F3 in Fig. 10). (Table 6)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pillai, S., Vadakkepat, P. Deep learning for machine health prognostics using Kernel-based feature transformation. J Intell Manuf 33, 1665–1680 (2022). https://doi.org/10.1007/s10845-021-01747-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01747-6

Keywords

Search

Navigation