Skip to main content
SpringerLink
Log in

ExtriDeNet: an intensive feature extrication deep network for hand gesture recognition

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, a lightweighted Intensive Feature Extrication Deep Network (ExtriDeNet) is proposed for precise hand gesture recognition (HGR). ExtriDeNet primarily consists of two blocks: Intensive Feature Fusion Block (IFFB) and Intensive Feature Assimilation Block (IFAB). IFFB incorporates two different scaled filters \(3\times 3\) and \(5 \times 5\) to capture contextual features of hands, while IFAB is designed by embedding influential features of IFFB with two extreme minute and high-level feature responses from two receptive fields generated by employing \(1\times 1\) and \(7\times 7\) sized filters, respectively. The combination of multiscaled filters enriches the network with the most significant features and enhances the learnability of the network. Thus, the proposed ExtriDeNet efficiently defines the distinctive features of different hand gesture classes and achieves high performance as compared to state-of-the-art HGR approaches. The performance of the proposed network is evaluated on the standard datasets: MUGD, Finger Spelling, OUHands, NUS-I, NUS-II and HGR1 for both subject-dependent and subject-independent scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: European Conference on Computer Vision, pp. 404–417. Springer, Berlin (2006)

  2. Lindeberg, T.: Scale invariant feature transform. Scholarpedia 7(5), 10491 (2012)

    Article  Google Scholar 

  3. Bhaumik, G., Verma, M., Govil, M.C., Vipparthi, S.K.: EXTRA: an extended radial mean response pattern for hand gesture recognition. In: 2020 International Conference on Communication and Signal Processing (ICCSP) (2020)

  4. Pedersoli, F., Benini, S., Adami, N., Leonardi, R.: XKin: an open source framework for hand pose and gesture recognition using kinect. Vis. Comput. 30(10), 1107–1122 (2014)

    Article  Google Scholar 

  5. Tamiru, N.K., Tekeba, M., Salau, A.O.: Recognition of Amharic sign language with Amharic alphabet signs using ANN and SVM. Vis. Comput. (2021)

  6. Islam, M., Hossain, M.S., Ul Islam, R., Andersson, K.: Static hand gesture recognition using convolutional neural network with data augmentation. In: Joint 2019 8th International Conference on Informatics. IEEE, Electronics and Vision (ICIEV) (2019)

  7. Imran, J., Raman, B.: Deep motion templates and extreme learning machine for sign language recognition. Vis. Comput. 36(6), 1233–1246 (2020)

    Article  Google Scholar 

  8. Pinto, R.F., Borges, C.D., Almeida, A., Paula, I.C.: Static hand gesture recognition based on convolutional neural networks. J. Electric. Comput. Eng. (2019)

  9. Nguyen, T.N., Huynh, H.H., Meunier, J.: Static hand gesture recognition using principal component analysis combined with artificial neural network. J. Autom. Control Eng. 3(1), 40–45 (2015)

    Article  Google Scholar 

  10. Zhang, W., Zeyi, L., Jian, C., Cuixia, M., Xiaoming, D., Hongan, W.: STA-GCN: two-stream graph convolutional network with spatial-temporal attention for hand gesture recognition. Vis. Comput. 36(10), 2433–2444 (2020)

    Article  Google Scholar 

  11. Bhuvaneshwari, C., Manjunathan, A.: Advanced gesture recognition system using long-term recurrent convolution network. In: Materials Today: Proceedings (2019)

  12. Tang, J., Shu, X., Li, Z., Qi, G.J., Wang, J.: Generalized deep transfer networks for knowledge propagation in heterogeneous domains. In: ACM Transactions on Multimedia Computing, Communications, and Applications, pp. 1–22 (2016)

  13. Shu, X., Qi, G.J., Tang, J., Wang, J.: Weakly-shared deep transfer networks for heterogeneous-domain knowledge propagation. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 35–44 (2015)

  14. Ozcan, T., Basturk, A.: Transfer learning-based convolutional neural networks with heuristic optimization for hand gesture recognition. Neural Comput. Appl. 31(12), 8955–8970 (2019)

    Article  Google Scholar 

  15. Liu, J., Furusawa, K., Tateyama, T., Iwamoto, Y., Chen, Y. W.: An improved hand gesture recognition with two-stage convolution neural networks using a hand color image and its pseudo-depth image. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 375–379. IEEE (2019)

  16. Côté-Allard, U., Fall, C.L., Drouin, A., Campeau-Lecours, A., Gosselin, C., Glette, K., Gosselin, B.: Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans. Neural Syst. Rehabil. Eng. 27(4), 760–771 (2019)

    Article  Google Scholar 

  17. Shu, X., Zhang, L., Qi, G.J., Liu, W., Tang, J.: Spatiotemporal co-attention recurrent neural networks for human-skeleton motion prediction. IEEE Trans. Pattern Anal. Mach. Intell. (2021)

  18. Shu, X., Tang, J., Qi, G.J., Liu, W., Yang, J.: Hierarchical long short-term concurrent memory for human interaction recognition. IEEE Trans. Pattern Anal. Mach. Intell. 43(3), 1110–1118 (2021)

    Article  Google Scholar 

  19. Shu, X., Zhang, L., Sun, Y., Tang, J.: Host-parasite: graph LSTM-In-LSTM for group activity recognition. IEEE Trans. Neural Netw. Learn. Syst. 32(2), 663–674 (2021)

    Article  Google Scholar 

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  21. Szegedy, C., et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  22. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Adam, H.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  23. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  24. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

  25. Varun, K., Sai, I., Puneeth, T., Prem, J.: Hand gesture recognition and implementation for disables using CNN’S. In: 2019 International Conference on Communication and Signal Processing (ICCSP), pp. 0592–0595. IEEE (2019)

  26. Zhan, F.: Hand gesture recognition with convolution neural networks. In: 2019 IEEE 20th International Conference on Information Reuse and Integration for Data Science (IRI), pp. 295–298. IEEE (2019)

  27. Barczak, A.L.C., Reyes, N.H., Abastillas, M., Piccio, A., Susnjak, T.: A new 2D static hand gesture colour image dataset for ASL gestures (2011)

  28. Nicolas Pugeault, R.B.: ASL finger spelling dataset. http://personal.ee.surrey.ac.uk/Personal/N.Pugeault/index.php

  29. Matti, M., Pekka, S., Jukka, H., Olli, S.: Ouhands database for hand detection and pose recognition. In: 6th International Conference on Image Processing Theory Tools and Applications, p. 1–5. IEEE (2016)

  30. Prahlad, V., Loh, A.P.: Attention based detection and recognition of hand postures against complex backgrounds. Int. J. Comput. Vis. 101(3), 403–419 (2013)

    Article  Google Scholar 

  31. HGR1. http://sun.aei.polsl.pl/ mkawulok/gestures/

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. CSE Department, National Institute of Technology, Sikkim, India

    Gopa Bhaumik & Mahesh Chandra Govil

  2. CSE Dept, Malaviya National Institute of Technology, Jaipur, India

    Monu Verma & Santosh Kumar Vipparthi

Authors
  1. Gopa Bhaumik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monu Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahesh Chandra Govil
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Santosh Kumar Vipparthi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gopa Bhaumik.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhaumik, G., Verma, M., Govil, M.C. et al. ExtriDeNet: an intensive feature extrication deep network for hand gesture recognition. Vis Comput 38, 3853–3866 (2022). https://doi.org/10.1007/s00371-021-02225-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02225-z

Keywords

Search

Navigation