Skip to main content
SpringerLink
Log in

SGRNN-AM and HRF-DBN: a hybrid machine learning model for cricket video summarization

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Summarization is important in sports video analysis; it gives a more compact and interesting representation of content. The automatic cricket video summarization is more challenging as it contains several rules and longer match duration. In this research, a hybrid machine learning approach is proposed to summarize cricket video. It analyzes the excitement, object, and event-based features for the detection of key events from the cricket video. First, the audio is analyzed for the extraction of the exciting clips by using an adaptive threshold, speech-to-text framework, and Stacked Gated Recurrent Neural Network with Attention Module (SGRNN-AM). Then, the scenes of each exciting clip are classified with a new Hybrid Rotation Forest Deep Belief Network (HRF-DBN). Next, the characters and action features are extracted from the scorecard region of each key frame and umpire frames of exciting clips. Finally, SGRNN-AM model is used to detect key events including fours, sixes, and wickets. The accuracy of the proposed SGRNN-AM video summarization model is increased with an attention module in the hidden outputs of Gated Recurrent Unit (GRU) for selecting the significant features. The performance of the suggested technique has been improved on various collections of cricket videos. It achieved a precision of \(96.82\ \%\) and an accuracy of \(96.32\%\) that proves its effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ji, Z., Ma, Y., Pang, Y., Li, X.: Query-aware sparse coding for web multi-video summarization. Inf. Sci. 478, 152–166 (2019)

    Article  Google Scholar 

  2. Panagiotakis, C., Papadakis, H., Fragopoulou, P.: Personalized video summarization based exclusively on user preferences. In: European Conference on Information Retrieval, Springer, pp. 305–311 (2020)

  3. Shukla, P., Sadana, H., Bansal, A., Verma, D., Elmadjian, C., Raman, B. Turk, M.: Automatic cricket highlight generation using event-driven and excitement-based features, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1800–1808 (2018)

  4. Merler, M., Mac, K.-N.C., Joshi, D., Nguyen, Q.-B., Hammer, S., Kent, J., Xiong, J., Do, M.N., Smith, J.R., Feris, R.S.: Automatic curation of sports highlights using multimodal excitement features. IEEE Trans. Multimedia 21(5), 1147–1160 (2018)

    Article  Google Scholar 

  5. Javed, A., Bajwa, K.B., Malik, H., Irtaza, A.: An efficient framework for automatic highlights generation from sports videos. IEEE Signal Process. Lett. 23(7), 954–958 (2016)

    Article  Google Scholar 

  6. Nandyal, S., Kattimani, S.L.: Bird swarm optimization-based stacked autoencoder deep learning for umpire detection and classification. Scalable Comput. Practice Exp. 21(2), 173–188 (2020)

    Article  Google Scholar 

  7. Choroś, K.: Highlights extraction in sports videos based on automatic posture and gesture recognition, In: Asian Conference on Intelligent Information and Database Systems, Springer, pp. 619–628 (2017)

  8. Javed, A., Bajwa, K.B., Malik, H., Irtaza, A., Mahmood, M.T.: A hybrid approach for summarization of cricket videos, In: 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), IEEE, pp. 1–4 (2016)

  9. Kastrati, Z., Imran, A.S., Yayilgan, S.Y.: The impact of deep learning on document classification using semantically rich representations. Inf. Process. Manag. 56(5), 1618–1632 (2019)

    Article  Google Scholar 

  10. Ståhl, N., Falkman, G., Karlsson, A., Mathiason, G.: Evaluation of uncertainty quantification in deep learning. Inf. Process. Manag. Uncertain. Knowl. Based Syst. 1237, 556–568 (2020)

    Google Scholar 

  11. O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G.V., Krpalkova, L., Riordan, D., Walsh, J.: Deep learning vs. traditional computer vision, In: Science and Information Conference, Springer, pp. 128–144 (2019)

  12. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. (2018)

  13. Hassan, M.M., Alam, M.G.R., Uddin, M.Z., Huda, S., Almogren, A., Fortino, G.: Human emotion recognition using deep belief network architecture. Inf. Fusion 51, 10–18 (2019)

    Article  Google Scholar 

  14. Abdel-Zaher, A.M., Eldeib, A.M.: Breast cancer classification using deep belief networks. Expert Syst. Appl. 46, 139–144 (2016)

    Article  Google Scholar 

  15. Rani, S., Kumar, M.: Social media video summarization using multi-visual features and kohnen’s self organizing map. Inf. Process. Manag. 57(3), 102190 (2020)

    Article  Google Scholar 

  16. Ravi, A., Venugopal, H., Paul, S., Tizhoosh, H.R.: A dataset and preliminary results for umpire pose detection using svm classification of deep features, In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE, pp. 1396–1402 (2018)

  17. Hari, R., Wilscy, M.: Event detection in cricket videos using intensity projection profile of umpire gestures, In: 2014 Annual IEEE India Conference (INDICON), IEEE, pp. 1–6 (2014)

  18. Nasir, M., Javed, A., Irtaza, A., Malik, H., Mahmood, M.: Event detection and summarization of cricket videos. J. Image Gr. 6(1)

  19. Javed, A., Irtaza, A., Malik, H., Mahmood, M.T., Adnan, S.: Multimodal framework based on audio-visual features for summarisation of cricket videos. IET Image Proc. 13(4), 615–622 (2019)

    Article  Google Scholar 

  20. Khan, A.A., Shao, J., Ali, W., Tumrani, S.: Content-aware summarization of broadcast sports videos: An audio-visual feature extraction approach. Neural Process. Lett. 1–24 (2020)

  21. Javed, A., Irtaza, A., Khaliq, Y., Malik, H., Mahmood, M.T.: Replay and key-events detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Appl. Intell. 49(8), 2899–2917 (2019)

    Article  Google Scholar 

  22. Moodley, T., van der Haar, D.: Cricket stroke recognition using computer vision methods, In: Information Science and Applications, Springer, pp. 171–181 (2020)

  23. Minhas, R.A., Javed, A., Irtaza, A., Mahmood, M.T., Joo, Y.B.: Shot classification of field sports videos using alexnet convolutional neural network. Appl. Sci. 9(3), 483 (2019)

    Article  Google Scholar 

  24. Rafiq, M., Rafiq, G., Agyeman, R., Choi, G.S., Jin, S.-I.: Scene classification for sports video summarization using transfer learning. Sensors 20(6), 1702 (2020)

    Article  Google Scholar 

  25. Javed, A., Malik, K.M., Irtaza, A., Malik, H.: A decision tree framework for shot classification of field sports videos. J. Supercomput. pp. 1–26 (2020)

  26. Taherkhani, A., Cosma, G., Alani, A.A., McGinnity, T.: Activity recognition from multi-modal sensor data using a deep convolutional neural network, In: Science and Information Conference, Springer, pp. 203–218 (2018)

  27. Shingrakhia, H., Patel, H.: Emperor penguin optimized event recognition and summarization for cricket highlight generation. Multimedia Syst. pp. 1–15 (2020)

  28. Kolekar, M.H., Sengupta, S.: Bayesian network-based customized highlight generation for broadcast soccer videos. IEEE Trans. Broadcast. 61(2), 195–209 (2015)

    Article  Google Scholar 

  29. Yang, F., Enzner, G., Yang, J.: Frequency-domain adaptive kalman filter with fast recovery of abrupt echo-path changes. IEEE Signal Process. Lett. 24(12), 1778–1782 (2017)

    Article  Google Scholar 

  30. Sheena, C.V., Narayanan, N.: Key-frame extraction by analysis of histograms of video frames using statistical methods. Procedia Comput. Sci. 70, 36–40 (2015)

    Article  Google Scholar 

  31. Naghibi, S.A., Dolatkordestani, M., Rezaei, A., Amouzegari, P., Heravi, M.T., Kalantar, B., Pradhan, B.: Application of rotation forest with decision trees as base classifier and a novel ensemble model in spatial modeling of groundwater potential. Environ. Monit. Assess. 191(4), 248 (2019)

    Article  Google Scholar 

  32. Zhang, N., Ding, S., Zhang, J., Xue, Y.: An overview on restricted boltzmann machines. Neurocomputing 275, 1186–1199 (2018)

    Article  Google Scholar 

  33. Lin, P., Fu, S.-W., Wang, S.-S., Lai, Y.-H., Tsao, Y.: Maximum entropy learning with deep belief networks. Entropy 18(7), 251 (2016)

    Article  MathSciNet  Google Scholar 

  34. Lu, W., Sun, H., Chu, J., Huang, X., Yu, J.: A novel approach for video text detection and recognition based on a corner response feature map and transferred deep convolutional neural network. IEEE Access 6, 40198–40211 (2018)

    Article  Google Scholar 

  35. Kolekar, M.H., Sengupta, S.: Semantic concept mining in cricket videos for automated highlight generation. Multimedia Tools Appl. 47(3), 545–579 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

This paper would not have been possible without the exceptional support of Prof. R N Mutagi. His enthusiasm, knowledge, and exacting attention to detail have been an inspiration and kept our work on track since the first version to the final draft of this paper.

Author information

Authors and Affiliations

  1. Gujarat Technological University, Ahmedabad, Gujarat, India

    Hansa Shingrakhia & Hetal Patel

  2. ECE Department, Indus University, Ahmedabad, Gujarat, India

    Hansa Shingrakhia

  3. ECE Department, A.D. Patel Institute of Technology and Engineering, New Vallabh Vidyanagar, Gujarat, India

    Hetal Patel

Authors
  1. Hansa Shingrakhia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hetal Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hansa Shingrakhia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shingrakhia, H., Patel, H. SGRNN-AM and HRF-DBN: a hybrid machine learning model for cricket video summarization. Vis Comput 38, 2285–2301 (2022). https://doi.org/10.1007/s00371-021-02111-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02111-8

Keywords

Search

Navigation