Skip to main content
SpringerLink
Log in

End-to-end multivariate time series classification via hybrid deep learning architectures

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Deep learning has revolutionized many areas, including time series data mining. Multivariate time series classification (MTSC) remained to be a well-known problem in the time series data mining community, due to its availability in various practical applications such as healthcare, finance, geoscience, and bioinformatics. Recently, multivariate long short-term memory with fully convolutional network (MLSTM-FCN) and multivariate attention long short-term memory with fully convolutional network (MALSTM-FCN) have shown superior results over various state-of-the-art methods. So, in this paper, we explore the usage of recurrent neural network (RNN), and its variants, such as bidirectional recurrent neural network (BiRNN), bidirectional long short-term memory (BiLSTM), gated recurrent unit (GRU), and bidirectional gated recurrent unit (BiGRU). We augment these RNN variants separately by replacing long short-term memory (LSTM) in MLSTM-FCN, which is the combination of LSTM, squeeze-and-excitation (SE) block, and fully convolutional network (FCN). Moreover, we integrate the SE block within FCN to leverage its high performance for the MTSC task. The resulting algorithms do not require heavy pre-processing or feature crafting. Thus, they could be easily deployed on real-time systems. We conduct a comprehensive evaluation with a large number of standard datasets and demonstrate that our approaches achieve notable results over the current best MTSC approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Fawaz HI et al (2019) Deep learning for time series classification: a review. Data Min Knowl Disc 33(4):917–963

    Article  MathSciNet  MATH  Google Scholar 

  2. De Gooijer JG, Hyndman RJ (2006) 25 years of time series forecasting. Int J Forecast 22(3):443–473

    Article  Google Scholar 

  3. Schäfer P, Leser U (2017) Multivariate time series classification with WEASEL+ MUSE. arXiv preprint arXiv:1711.11343

  4. Jaakkola T, Diekhans M, Haussler D (2000) A discriminative framework for detecting remote protein homologies. J Comput Biol 7(1–2):95–114

    Article  Google Scholar 

  5. Van Der Maaten L (2011) Learning discriminative fisher kernels. in ICML

  6. Orsenigo C, Vercellis C (2010) Combining discrete SVM and fixed cardinality warping distances for multivariate time series classification. Pattern Recogn 43(11):3787–3794

    Article  MATH  Google Scholar 

  7. Pei W, Dibeklioglu H, Tax DMJ, van der Maaten L (2017) Multivariate time-series classification using the hidden-unit logistic model. IEEE Transactions on Neural Networks and Learning Systems 29(4):920–931

    Article  Google Scholar 

  8. Baydogan MG, Runger G (2015) Learning a symbolic representation for multivariate time series classification. Data Min Knowl Disc 29(2):400–422

    Article  MathSciNet  MATH  Google Scholar 

  9. Wistuba M, Grabocka J, Schmidt-Thieme L (2015) Ultra-fast shapelets for time series classification. arXiv preprint arXiv:1503.05018

  10. Tuncel KS, Baydogan MG (2018) Autoregressive forests for multivariate time series modeling. Pattern Recogn 73:202–215

    Article  Google Scholar 

  11. Zheng Y, et al. (2014) Time series classification using multi-channels deep convolutional neural networks. in International Conference on Web-Age Information Management. 2014. Springer

  12. Karim F, Majumdar S, Darabi H, Harford S (2019) Multivariate lstm-fcns for time series classification. Neural Netw 116:237–245

    Article  Google Scholar 

  13. Wang Z, Yan W, Oates T (2017) Time series classification from scratch with deep neural networks: a strong baseline. in 2017 International Joint Conference on Neural Networks (IJCNN). IEEE

  14. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  15. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. in Proceedings of the 27th international conference on machine learning (ICML-10)

  16. Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400

  17. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. in Proceedings of the IEEE conference on computer vision and pattern recognition

  18. He K, et al. (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. in Proceedings of the IEEE international conference on computer vision

  19. Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019

  20. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Article  Google Scholar 

  21. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  22. Zhao Y, Yang R, Chevalier G, Shah RC, Romijnders R (2018) Applying deep bidirectional LSTM and mixture density network for basketball trajectory prediction. Optik 158:266–272

    Article  Google Scholar 

  23. Graves A, Schmidhuber J (2005) Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw 18(5–6):602–610

    Article  Google Scholar 

  24. Chung J, et al. (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

  25. Srivastava N et al (2014) Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  26. Dua, DAG, Casey (2017) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences

  27. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  28. Chollet F (2015) Keras. Available from: https://github.com/fchollet/keras

  29. Abadi M, et al. (2016) Tensorflow: a system for large-scale machine learning. in 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16)

Download references

Funding

This paper was partially supported by the NSFC grant U1509216, U1866602, 61602129, and Microsoft Research Asia.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China

    Mehak Khan, Hongzhi Wang, Alladoumbaye Ngueilbaye & Aya Elfatyany

Authors
  1. Mehak Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongzhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alladoumbaye Ngueilbaye
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aya Elfatyany
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehak Khan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, M., Wang, H., Ngueilbaye, A. et al. End-to-end multivariate time series classification via hybrid deep learning architectures. Pers Ubiquit Comput 27, 177–191 (2023). https://doi.org/10.1007/s00779-020-01447-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01447-7

Keywords

Search

Navigation