Skip to main content
SpringerLink
Log in

American Sign Language recognition using Support Vector Machine and Convolutional Neural Network

  • Original Research
  • Published:
International Journal of Information Technology Aims and scope Submit manuscript

Abstract

A sign language recognition system is an attempt to help the speech and the hearing-impaired community. The biggest challenge is to recognize a sign accurately. This can be achieved by training the computers to identify the signs. The accuracy depends on the methods used for classification and prediction which is achieved through machine learning. This research proposes the recognition of American Sign Language by using Support Vector Machine (SVM) and Convolutional Neural Network (CNN). In this work we have also calculated optimal filter size for single and double layer Convolutional Neural Network. In the first phase features from the dataset are extracted. After applying various preprocessing techniques, Support Vector Machine with four different kernels i.e., ‘poly’, ‘linear’, ‘rbf’ and ‘sigmoid’ and Convolutional Neural Networks with single and double layer are applied on training dataset to train the model. Finally, accuracy is calculated and compared for both the techniques. In CNN filters of different sizes have been used and optimal filter size has been found. The experimental results establish that the double layer Convolutional Neural Network achieve an accuracy of 98.58%. Optimal filter size is found out to be 8 × 8 for both single and double layer Convolutional Neural Network. From the experimental results we conclude that accuracy of CNN model can be improved by altering the filter size. This also helps in CNN to learn optimum values for variable sized parameters and tuning of different hyper parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Cheok MJ, Omar Z, Jaward MH (2019) A review of hand gesture and sign language recognition techniques. Int J Mach Learn Cybern 10(1):131–153

    Article  Google Scholar 

  2. Gu Y, Liu T, Jia X, Benediktsson JA, Chanussot J (2016) Nonlinear multiple kernel learning with multiple-structure-element extended morphological profiles for hyperspectral image classification. IEEE Trans Geosci Remote Sens 54(6):3235–3247

    Article  Google Scholar 

  3. Bellugi U, Fischer S (1972) A comparison of sign language and spoken language. Cognition 1:173–200

    Article  Google Scholar 

  4. Yang R, Sarkar S, Loeding B (2010) Handling movement epenthesis and hand segmentation ambiguities in continuous sign language recognition using nested dynamic programming. IEEE Trans Pattern Anal Mach Intell 32:462–477

    Article  Google Scholar 

  5. Wu Y, Huang TS (1999) Human hand modeling, analysis and animation in the context of HCI. In: Image processing, ICIP 99. Proceedings. 1999 international conference, IEEE, pp 6–10

  6. Wu Y, Huang TS (1999) Vision-based gesture recognition: a review. In: International gesture workshop, Springer, pp 103–115

  7. Lockton R (2002) Hand gesture recognition using computer vision 4th year project report, pp 1–69

  8. Starner TE (1995) Visual recognition of American sign language using hidden Markov models. Dept of Brain and Cognitive Sciences, Massachusetts Inst of Tech, Cambridge

  9. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43:1–54

    Article  Google Scholar 

  10. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54

    Article  Google Scholar 

  11. Chen Q, Georganas ND, Petriu EM (2007). Real-time vision-based hand gesture recognition using haar-like features. In: 2007 IEEE instrumentation and measurement technology conference IMTC 2007, pp 1–6

  12. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R et al (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 675–678

  13. Starner T, Weaver J, Pentland A (1998) Real-time American sign language recognition using desk and wearable computer-based video. IEEE Trans Pattern Anal Mach Intell 20:1371–1375

    Article  Google Scholar 

  14. Grobel K, Assan M (1997) Isolated sign language recognition using hidden Markov models. In: Systems, man, and cybernetics, 1997. Computational cybernetics and simulation. 1997 IEEE international conference, IEEE, pp 162–167

  15. Vogler C, Metaxas D (1998) ASL recognition based on a coupling between HMMs and 3D motion analysis. In: Computer vision, 1998. Sixth international conference, IEEE, pp 363–369

  16. Vogler C, Metaxas D (1997) Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods. In: Systems, man, and cybernetics, computational cybernetics and simulation. 1997 IEEE international conference, IEEE, pp 156–161

  17. Ahuja R, Jain D, Sachdeva D, Garg A, Rajput C (2019) Convolutional neural network based American sign language static hand gesture recognition. Int J Ambient Comput Intell IJACI 10(3):60–73

    Article  Google Scholar 

  18. Ko SK, Kim CJ, Jung H, Cho C (2019) Neural sign language translation based on human keypoint estimation. Appl Sci 9(13):2683

    Article  Google Scholar 

  19. Koller O, Camgoz C, Ney H, Bowden R (2019) Weakly supervised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos. IEEE Trans Pattern Mach Intell 42:2306–2320

    Article  Google Scholar 

  20. Wang H, Chai X, Chen X (2019) A novel sign language recognition framework using hierarchical Grassmann covariance matrix. IEEE Trans Multimed 21:2806–2814

    Article  Google Scholar 

  21. Mittal A, Kumar P, Roy PP, Balasubramanian R, Chaudhuri BB (2019) A modified-LSTM model for continuous sign language recognition using leap motion. IEEE Sens J 19:7056–7063

    Article  Google Scholar 

  22. Masood S, Thuwal HC, Srivastava A (2018) American sign language character recognition using convolution neural network. In: Satapathy S, Bhateja V, Das S (eds) Smart computing and informatics. Smart innovation, systems and technologie, vol 78. Springer, Singapore

    Google Scholar 

  23. Trigueiros P, Ribeiro F, Reis LP (2014) Vision-based Portuguese sign language recognition system. In: New perspectives in information systems and technologies, vol 1, pp 605–617. Springer International Publishing

  24. Tavari NV, Deorankar AV (2014) Indian sign language recognition based on histograms of oriented gradient. Int J Comput Sci Inf Technol 5:3657–3660

    Google Scholar 

  25. Hasan H, Abdul-Kareem S (2014) Static hand gesture recognition using neural networks. Artif Intell Rev 41(2):147–181

    Article  Google Scholar 

  26. Pigou L, Dieleman S, Kindermans PJ, Schrauwen B (2014) Sign language recognition using convolutional neural networks. In: Workshop at the European conference on computer vision 2014, pp 572–578. Springer International Publishing

  27. Gupta B, Shukla P, Mittal A (2016) K-nearest correlated neighbor classification for Indian sign language gesture recognition using feature fusion. In: 2016 International conference on computer communication and informatics (ICCCI), pp 1–5

  28. Nagarajan S, Subashini TS (2013) Static hand gesture recognition for sign language alphabets using edge-oriented histogram and multi class SVM. Int J Comput Appl 82(4):28–35

    Google Scholar 

  29. Singh K, Kumar S, Kaur P (2019) Support vector machine classifier based detection of fungal rust disease in Pea Plant (Pisam sativam). Int J Inf Technol 11:485–492. https://doi.org/10.1007/s41870-018-0134-z

    Article  Google Scholar 

  30. Solanki A, Pandey S (2019) Music instrument recognition using deep convolutional neural networks. Int J Inf Technol. https://doi.org/10.1007/s41870-019-00285-y

    Article  Google Scholar 

  31. Kaggle (2017) Sign language MNIST: drop-in replacement for MNIST for hand gesture recognition tasks. https://www.kaggle.com/datamunge/sign-language-mnist. Accessed Jan 2018

Download references

Author information

Authors and Affiliations

  1. Bharati Vidyapeeth’s College of Engineering, New Delhi, India

    Vanita Jain, Achin Jain, Abhinav Chauhan, Srinivasu Soma Kotla & Ashish Gautam

Authors
  1. Vanita Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Achin Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abhinav Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Srinivasu Soma Kotla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ashish Gautam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanita Jain.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jain, V., Jain, A., Chauhan, A. et al. American Sign Language recognition using Support Vector Machine and Convolutional Neural Network. Int. j. inf. tecnol. 13, 1193–1200 (2021). https://doi.org/10.1007/s41870-021-00617-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41870-021-00617-x

Keywords

Search

Navigation