Skip to main content
SpringerLink
Log in

A novel bifold-stage shot boundary detection algorithm: invariant to motion and illumination

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Shot boundary detection is mainly considered as a stepping stone in the broad arena of content-based video retrieval. Ample systematic investigation has been carried out in the terrain of shot boundary detection. The attainment of shot boundary detection procedures is greatly hindered due to the presence of unforeseen illumination change and motion effects in a video. This work proposes a novel bifold-stage technique to recognize abrupt transition in videos, invariant to motion, and illumination effects. In the first stage, the local ternary patterns feature is used to extract information from each frame in a video. Then, a set of novel adaptive thresholds such as \({\gamma }\) and \({{{\beta }}}\) are used to find the possible transition frames. In the confirmation stage, Lab color difference along with an adaptive threshold \({\delta }\) is used to extract actual transition frames. The experimental result depicts that the motion effect is also scaled down in the initial stage. The Lab color difference passed down in the second stage also handles the illumination and motion effects which are not managed in the initial stage. Experimentation is done using TRECVid 2001 and 2007 standard datasets and palpable that the proposed technique outperforms most of the contemporary shot boundary detection approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abdulhussain, S.H., Ramli, A.R., Saripan, M.I., Mahmmod, B.M., Al-Haddad, S.A.R.: Methods and challenges in shot boundary detection: a review. Entropy 20(4), 214 (2018)

    Article  Google Scholar 

  2. Waghmare, M.S.P., Bhide, A.S.: Shot boundary detection using histogram differences. Int. J. Adv. Res. Electron. Commun. Eng. 3, 1460–1464 (2014)

    Google Scholar 

  3. Liu, T., Chan, S.: Automatic shot boundary detection algorithm using structure-aware histogram metric. In: International Conference on Digital Signal Processing, pp. 541–546 (2014)

  4. Kaabneh, K., Alia, O., Suleiman, A., Abuirbaleh, A.A.A.A.: Video segmentation via dual shot boundary detection (DSBD). In: 2006 2nd International Conference on Information and Communication Technologies, vol. 1, pp. 1530–1533. IEEE (2006)

  5. Warhade, K.K., Merchant, S.N., Desai, U.B.: Avoiding false positive due to flashlights in shot detection using illumination suppression algorithm. In: International Conference on Visual Information Engineering, pp. 377–381 (2008)

  6. Warhade, K.K., Merchant, S.N., Desai, U.B.: Shot boundary detection in the presence of fire flicker and explosion using stationary wavelet transform. Signal Image Video Process. 5(4), 507–515 (2011)

    Article  Google Scholar 

  7. Warhade, K.K., Merchant, S.N., Desai, U.B.: Shot boundary detection in the presence of illumination and motion. Signal Image Video Process. 7(3), 581–592 (2013)

    Article  Google Scholar 

  8. Kanungo, P., Kar, T.: Cut detection using block based center symmetric local binary pattern. In: International Conference on Man and Machine Interfacing, pp. 1–5 (2015)

  9. Li, J., Yao, T., Ling, Q., Mei, T.: Detecting shot boundary with sparse coding for video summarization. Neurocomputing 266(C), 66–78 (2017)

    Article  Google Scholar 

  10. Pingping, C., Guan, Y., Ding, X., Yu, Z.: Shot boundary detection with sparse presentation. In: 2016 IEEE 13th International Conference on Signal Processing (ICSP), pp. 900–904. IEEE (2016)

  11. Huan, Z., Xiuhuan, L., Lilei, Y.: Shot boundary detection based on mutual information and canny edge detector. Int. Conf. Comput. Sci. Softw. Eng. 2, 1124–1128 (2008)

    Google Scholar 

  12. Fu, Q., Zhang, Y., Xu, L., Li, H.: A method of shot-boundary detection based on HSV space. In: Ninth International Conference on Computational Intelligence and Security, pp. 219–223 (2013)

  13. Lakshmi Priya, G.G., Domnic, S.: Walsh-Hadamard transform kernel-based feature vector for shot boundary detection. IEEE Trans. Image Process. 23(12), 5187–5197 (2014)

    Article  MathSciNet  Google Scholar 

  14. Li, Y., Lu, Z., Niu, X.: Fast video shot boundary detection framework employing pre-processing techniques. IET Image Process. 3(3), 121–134 (2009)

    Article  Google Scholar 

  15. Thounaojam, D.M., Khelchandra, T., Singh, K.M., Roy, S.: A genetic algorithm and fuzzy logic approach for video shot boundary detection. Comput. Intell. Neurosci. 2016, 14 (2016)

    Article  Google Scholar 

  16. Hassanien, A., Elgharib, M.A., Selim, A., Hefeeda, M., Matusik, W.: Large-scale, fast and accurate shot boundary detection through spatio-temporal convolutional neural networks. CoRR, arXiv: 1705.03281 (2017)

  17. Tong, W., Song, L., Yang, X., Qu, H., Xie, R.: CNN-based shot boundary detection and video annotation. In: IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1–5 (2015)

  18. Xu, J., Song, L., Xie, R.: Shot boundary detection using convolutional neural networks. In: Visual Communications and Image Processing, pp. 1–4 (2016)

  19. Tang, S., Feng, L., Kuang, Z., Chen, Y., Zhang, W.: Fast video shot transition localization with deep structured models. CoRR, arXiv:1808.04234 (2018)

  20. Srilakshmi, B., Sandeep, R.: Shot boundary detection using structural similarity index. In: Fifth International Conference on Advances in Computing and Communications (ICACC), pp. 439–442 (2015)

  21. Chen, L.-H., Hsu, B.-C., Chih-Wen, S.: A supervised learning approach to flashlight detection. Cybern. Syst. 48(1), 1–12 (2017)

    Article  Google Scholar 

  22. Kar, T., Kanungo, P.: A motion and illumination resilient framework for automatic shot boundary detection. Signal Image Video Process. 11(7), 1237–1244 (2017)

    Article  Google Scholar 

  23. Heng, W.J., Ngan, K.N.: The implementation of object-based shot boundary detection using edge tracing and tracking. IEEE Int. Symp. Circuits Syst. VLSI 4, 439–442 (1999)

    Google Scholar 

  24. Heng, W.J., Ngan, K.N.: An object-based shot boundary detection using edge tracing and tracking. J. Vis. Commun. Image Represent. 12(3), 217–239 (2001)

    Article  Google Scholar 

  25. Lan, X., Zhang, S., Yuen, P.C., Chellappa, R.: Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans. Image Process. 27(4), 2022–2037 (2018)

    Article  MathSciNet  Google Scholar 

  26. Lan, X., Ye, M., Shao, R., Zhong, B., Yuen, P.C., Zhou, H.: Learning modality-consistency feature templates: a robust RGB-infrared tracking system. IEEE Trans. Ind. Electron. 66(12), 9887–9897 (2019)

    Article  Google Scholar 

  27. Lan, X., Ye, M., Shao, R., Zhong, B., Jain, D.K., Zhou, H.: Online non-negative multi-modality feature template learning for RGB-assisted infrared tracking. IEEE Access 7, 67761–67771 (2019)

    Article  Google Scholar 

  28. Rashmi, B.S., Nagendraswamy, H.S.: Video shot boundary detection using midrange local binary pattern. In: International Conference on Advances in Computing, Communications and Informatics, pp. 201–206 (2016)

  29. Kar, T., Kanungo, P.: A texture based method for scene change detection. In: International Conference on Power, Communication and Information Technology Conference, pp. 72–77 (2015)

  30. Singh, A., Thounaojam, D.M., Chakraborty, S.: A novel automatic shot boundary detection algorithm: robust to illumination and motion effect. Signal Image Video Process. 14, 1–9 (2019)

    Google Scholar 

  31. Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)

    Article  MathSciNet  Google Scholar 

  32. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  33. Sharma, G., Wencheng, W., Dalal, E.N.: The ciede2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations. Color Res. Appl. 30, 21–30 (2005)

    Article  Google Scholar 

  34. Pele, O., Werman, M.: The quadratic-chi histogram distance family. In: European Conference on Computer Cision, pp. 749–762. Springer (2010)

  35. Chakraborty, S., Thounaojam, D.M.: A novel shot boundary detection system using hybrid optimization technique. Appl. Intell. 49, 1–14 (2019)

    Article  Google Scholar 

  36. Youssef, B., Fedwa, E., Driss, A., Ahmed, S.: Shot boundary detection via adaptive low rank and SVD-updating. Comput. Vis. Image Underst. 161, 20–28 (2017)

    Article  Google Scholar 

  37. Thounaojam, D.M., Thongam, K., Jayshree, T., Roy, S., Singh, K.M.: Colour histogram and modified multi-layer perceptron neural network based video shot boundary detection. Int. Arab J. Inf. Technol. 16, 686–693 (2019)

    Google Scholar 

  38. Krishnakumar, K., Gandhi, S.I.: Video stitching based on multi-view spatiotemporal feature points and grid-based matching. Vis. Computer. 36, 1–10 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Laboratory, Department of Computer Science and Engineering, National Institute of Technology Silchar, Assam, Silchar, India

    Saptarshi Chakraborty, Alok Singh & Dalton Meitei Thounaojam

Authors
  1. Saptarshi Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alok Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dalton Meitei Thounaojam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saptarshi Chakraborty.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, S., Singh, A. & Thounaojam, D.M. A novel bifold-stage shot boundary detection algorithm: invariant to motion and illumination. Vis Comput 38, 445–456 (2022). https://doi.org/10.1007/s00371-020-02027-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-02027-9

Keywords

Search

Navigation