Skip to main content
Log in

Instance-vote-based motion detection using spatially extended hybrid feature space

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Motion recognition, a trivial step employed in several video-based applications, is still a challenging task in real-world complex scenarios containing dynamic noise, varying backgrounds, shadows, improper illuminations, camouflages, etc. Numerous pixel-based change detection techniques employing varied combinations of different feature spaces have been proposed to efficiently overcome many real-world challenges. But ideally, handling all the possible real-world challenges simultaneously is yet to be achieved. Hence, this paper proposes a memory-efficient unique combination of multi-colour feature space with a light-weight intensity-based texture descriptor. The proposed spatially enlarged extended centre-symmetric local binary pattern is combined with YCbCr and RGB colour features for robust pixel representation. The proposed feature space is fed to an extended instance-vote technique for pixel classification. The random and time-subsampled update is employed conditionally for model update, followed by a feedback network that continuously optimizes the local threshold and learning rate parameters of the proposed model. The proposed feature space and model have been evaluated on whole 2014 Change Detection dataset, the largest known dataset. The outperforming performance and memory analysis strengthens its acceptability for real-time applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Porikli, F., Tuzel, O.: Human body tracking by adaptive background models and mean-shift analysis. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, pp. 1–9. Citeseer (2003)

  2. Wang, Y., Wei, X., Ding, L., Tang, X., Zhang, H.: A robust visual tracking method via local feature extraction and saliency detection. Vis. Comput. 36, 1–18 (2019)

    Google Scholar 

  3. Elsayed, R.A., Sayed, M.S., Abdalla, M.I.: Kin-based adaptive background subtraction for hand gesture segmentation. In: 2015 IEEE International Conference on Electronics, Circuits, and Systems (ICECS), pp. 33–36. IEEE (2015)

  4. Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J.: Fall detection from human shape and motion history using video surveillance. In: 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW’07), vol. 2, pp. 875–880. IEEE (2007)

  5. Reinders, F., Post, F.H., Spoelder, H.J.W.: Visualization of time-dependent data with feature tracking and event detection. Vis. Comput. 17(1), 55–71 (2001)

    MATH  Google Scholar 

  6. Shen, J., Peng, J., Dong, X., Shao, L., Porikli, F.: Higher order energies for image segmentation. IEEE Trans. Image Process. 26(10), 4911–4922 (2017)

    MathSciNet  MATH  Google Scholar 

  7. Goyette, N., Jodoin, P.-M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection. net: a new change detection benchmark dataset. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8. IEEE (2012)

  8. Wang, Y., Jodoin, P.-M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 387–394 (2014)

  9. Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), vol. 2, pp. 246–252. IEEE (1999)

  10. KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Remagnino, P., Jones, G.A., Paragios, N., Regazzoni, C.S. (eds.) Video-Based Surveillance Systems, pp. 135–144. Springer, Berlin (2002)

    Google Scholar 

  11. Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol. 2, pp. 28–31. IEEE (2004)

  12. Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)

    Google Scholar 

  13. Lee, D.-S.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 5, 827–832 (2005)

    Google Scholar 

  14. Oliver, N.M., Rosario, B., Pentland, A.P.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)

    Google Scholar 

  15. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)

    MathSciNet  MATH  Google Scholar 

  16. Zhou, X., Yang, C., Weichuan, Y.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 597–610 (2013)

    Google Scholar 

  17. Gao, Z., Cheong, L.-F., Wang, Y.-X.: Block-sparse RPCA for salient motion detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 1975–1987 (2014)

    Google Scholar 

  18. Kawabata, S., Hiura, S., Sato, K.: Real-time detection of anomalous objects in dynamic scene. In: 18th International Conference on Pattern Recognition, 2006. ICPR 2006, vol. 3, pp. 1171–1174. IEEE (2006)

  19. He, J., Balzano, L., Szlam, A.: Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1568–1575. IEEE (2012)

  20. Feng, J., Xu, H., Yan, S.: Online robust PCA via stochastic optimization. In: Advances in Neural Information Processing Systems, pp. 404–412 (2013)

  21. Sun, Y., Tao, X., Li, Y., Jianhua, L.: Robust 2D principal component analysis: a structured sparsity regularized approach. IEEE Trans. Image Process. 24(8), 2515–2526 (2015)

    MathSciNet  MATH  Google Scholar 

  22. Wang, J., Bebis, G., Nicolescu, M., Nicolescu, M., Miller, R.: Improving target detection by coupling it with tracking. Mach. Vis. Appl. 20(4), 205–223 (2009)

    Google Scholar 

  23. Wang, J., Bebis, G., Miller, R.: Robust video-based surveillance by integrating target detection with tracking. In: Computer Vision and Pattern Recognition Workshop, 2006. CVPRW’06. Conference on, pp. 137–137. IEEE (2006)

  24. Lin, H.-H., Liu, T.-L., Chuang, J.-H.: A probabilistic SVM approach for background scene initialization. In: Proceedings. 2002 International Conference on Image Processing. 2002, vol. 3, pp. 893–896. IEEE (2002)

  25. Lin, H.-H., Liu, T.-L., Chuang, J.-H.: Learning a scene background model via classification. IEEE Trans. Signal Process. 57(5), 1641–1654 (2009)

    MathSciNet  MATH  Google Scholar 

  26. Tavakkoli, A., Nicolescu, M., Nicolescu, M., Bebis, G.: Incremental SVDD training: improving efficiency of background modeling in videos. In: International Conference on Signal and Image Processing, Kailua-Kona, HI, pp. 1–6 (2008)

  27. Kim, K., Chalidabhongse, T.H., David, H., Larry, D.: Real-time foreground–background segmentation using codebook model. Real Time Imaging 11(3), 172–185 (2005)

    Google Scholar 

  28. Ilyas, A., Scuturici, M., Miguet, S.: Real time foreground-background segmentation using a modified codebook model. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009. AVSS’09, pp. 454–459. IEEE (2009)

  29. Chacon-Murguia, M.I., Gonzalez-Duarte, S.: An adaptive neural-fuzzy approach for object detection in dynamic backgrounds for surveillance systems. IEEE Trans. Ind. Electron. 59(8), 3286–3298 (2011)

    Google Scholar 

  30. Zeng, J., Xie, L., Liu, Z.-Q.: Type-2 fuzzy Gaussian mixture models. Pattern Recognit. 41(12), 3636–3643 (2008)

    MATH  Google Scholar 

  31. El Baf, F., Bouwmans, T., Vachon, B.: Type-2 fuzzy mixture of Gaussians model: application to background modeling. In: International Symposium on Visual Computing, pp. 772–781. Springer (2008)

  32. Zhao, Z., Bouwmans, T., Zhang, X., Fang, Y.: A fuzzy background modeling approach for motion detection in dynamic backgrounds. In: International Conference on Multimedia and Signal Processing, pp. 177–185. Springer (2012)

  33. Luo, L., Zhao, Z.-Q., Li, X.-P.: A novel surveillance video processing using stochastic low-rank and generalized low-rank approximation techniques. In: 2018 International Conference on Machine Learning and Cybernetics (ICMLC), vol. 1, pp. 91–98. IEEE (2018)

  34. Elgammal, A., Harwood, D., Davis, L.: Non-parametric model for background subtraction. In: European Conference on Computer Vision, pp. 751–767. Springer (2000)

  35. Tavakkoli, A., Nicolescu, M., Bebis, G.: An adaptive recursive learning technique for robust foreground object detection. In: Proceedings of the International Workshop on Statistical Methods in Multi-image and Video Processing (in Conjunction with ECCV06) (2006)

  36. Tavakkoli, A., Nicolescu, M., Bebis, G.: Robust recursive learning for foreground region detection in videos with quasi-stationary backgrounds. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 315–318. IEEE (2006)

  37. Tanaka, T., Shimada, A., Arita, D., Taniguchi, R.-I.: Non-parametric background and shadow modeling for object detection. In: Asian Conference on Computer Vision, pp. 159–168. Springer (2007)

  38. Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis. 39(1), 41–56 (2000)

    MATH  Google Scholar 

  39. Chen, M., Wei, X., Yang, Q., Li, Q., Wang, G., Yang, M.-H.: Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1518–1525 (2018)

    Google Scholar 

  40. Shen, J., Peng, J., Shao, L.: Submodular trajectories for better motion segmentation in videos. IEEE Trans. Image Process. 27(6), 2688–2700 (2018)

    MathSciNet  MATH  Google Scholar 

  41. Shen, J., Dong, X., Peng, J., Jin, X., Shao, L., Porikli, F.: Submodular function optimization for motion clustering and image segmentation. IEEE Trans. Neural Netw. Learn. Syst. 30(9), 2637–2649 (2019)

    MathSciNet  Google Scholar 

  42. Liu, Y., Shen, J., Wang, W., Sun, H., Shao, L.: Better dense trajectories by motion in videos. IEEE Trans. Cybern. 49(1), 159–170 (2017)

    Google Scholar 

  43. Wang, W., Shen, J., Porikli, F., Yang, R.: Semi-supervised video object segmentation with super-trajectories. IEEE Trans Pattern Anal Mach Intell 41(4), 985–998 (2018)

    Google Scholar 

  44. Liu, C., Wang, W., Shen, J., Shao, L.: Stereo video object segmentation using stereoscopic foreground trajectories. IEEE Trans. Cybern. 49(10), 3665–3676 (2018)

    Google Scholar 

  45. Wang, H., Suter, D.: A consensus-based method for tracking: modelling background scenario and foreground appearance. Pattern Recognit. 40(3), 1091–1105 (2007)

    MATH  Google Scholar 

  46. Wang, H., Suter, D.: Background subtraction based on a robust consensus method. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 223–226. IEEE (2006)

  47. Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)

    MathSciNet  MATH  Google Scholar 

  48. Van Droogenbroeck, M., Barnich, O.: ViBe: a disruptive method for background subtraction. In: Background Modeling and Foreground Detection for Video Surveillance, pp. 7–1 (2014)

  49. Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–43. IEEE (2012)

  50. Yang, S., Hao, K., Ding, Y., Liu, J.: Improved visual background extractor with adaptive range change. Memet. Comput. 10(1), 53–61 (2018)

    Google Scholar 

  51. Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)

    Google Scholar 

  52. Heikkilä, M., Pietikäinen, M., Schmid, C.: Description of interest regions with local binary patterns. Pattern Recognit. 42(3), 425–436 (2009)

    MATH  Google Scholar 

  53. Xue, G., Sun, J., Song, L.: Dynamic background subtraction based on spatial extended center-symmetric local binary pattern. In: 2010 IEEE International Conference on Multimedia and Expo, pp. 1050–1054. IEEE (2010)

  54. Wang, L., Pan, C.: Fast and effective background subtraction based on \(\varepsilon \)lbp. In: International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2010 (2010)

  55. Wang, L., Wu, H., Pan, C.: Adaptive \(\varepsilon \) LBP for background subtraction. In: Asian Conference on Computer Vision, ACCV 2010, pp. 560–571. Springer (2010)

  56. Liao, S., Zhao, G., Kellokumpu, V., Pietikäinen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1301–1306. IEEE (2010)

  57. Noh, S., Jeon, M.: A new framework for background subtraction using multiple cues. In: Asian Conference on Computer Vision, pp. 493–506. Springer (2012)

  58. Hefeng, W., Liu, N., Luo, X., Jiawei, S., Chen, L.: Real-time background subtraction-based video surveillance of people by integrating local texture patterns. Signal Image Video Process. 8(4), 665–676 (2014)

    Google Scholar 

  59. Xue, G., Song, L., Sun, J., Wu, M.: Hybrid center-symmetric local pattern for dynamic background subtraction. In: 2011 IEEE International Conference on Multimedia and Expo, pp. 1–6. IEEE (2011)

  60. Lee, Y., Jung, J., Kweon, I.-S.: Hierarchical on-line boosting based background subtraction. In: 2011 17th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), pp. 1–5. IEEE (2011)

  61. Bilodeau, G.-A., Jodoin, J.-P., Saunier, N.: Change detection in feature space using local binary similarity patterns. In: International Conference on Computer and Robot Vision, CRV 2013, pp. 106–112. IEEE (2013)

  62. St-Charles, P.-L., Bilodeau, G.-A.: Improving background subtraction using local binary similarity patterns. In: 2014 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 509–515. IEEE (2014)

  63. Silva, C., Bouwmans, T., Frélicot, C.: An extended center-symmetric local binary pattern for background modeling and subtraction in videos. In: International Joint Conference on Computer Vision. VISAPP, Imaging and Computer Graphics Theory and Applications, p. 2015 (2015)

  64. Silva, C., Bouwmans, T., Frélicot, C.: Online weighted one-class ensemble for feature selection in background/foreground separation. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2216–2221. IEEE (2016)

  65. Bouwmans, T., Silva, C., Marghes, C., Zitouni, M.S., Bhaskar, H., Frelicot, C.: On the role and the importance of features for background modeling and foreground detection. Comput. Sci. Rev. 28, 26–91 (2018)

    MathSciNet  Google Scholar 

  66. St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)

    MathSciNet  MATH  Google Scholar 

  67. St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: Universal background subtraction using word consensus models. IEEE Trans. Image Process. 25(10), 4768–4781 (2016)

    MathSciNet  MATH  Google Scholar 

  68. Jiang, S., Xiaobo, L.: Wesambe: a weight-sample-based method for background subtraction. IEEE Trans. Circuits Syst. Video Technol. 28(9), 2105–2115 (2018)

    Google Scholar 

  69. Braham, M., Van Droogenbroeck, M.: Deep background subtraction with scene-specific convolutional neural networks. In: 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–4. IEEE (2016)

  70. Liao, J., Guo, G., Yan, Y., Wang, H.: Multiscale cascaded scene-specific convolutional neural networks for background subtraction. In: Pacific Rim Conference on Multimedia, pp. 524–533. Springer (2018)

  71. Liang, X., Liao, S., Wang, X., Liu, W., Chen, Y., Li, S.Z.: Deep background subtraction with guided learning. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2018)

  72. Dong, X., Shen, J., Dongming, W., Guo, K., Jin, X., Porikli, F.: Quadruplet network with one-shot learning for fast visual object tracking. IEEE Trans. Image Process. 28(7), 3516–3527 (2019)

    MathSciNet  MATH  Google Scholar 

  73. Liang, Z., Shen, J.: Local semantic siamese networks for fast tracking. IEEE Trans. Image Process. 29, 3351–3364 (2019)

    Google Scholar 

  74. Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 459–474 (2018)

  75. Lai, Q., Wang, W., Sun, H., Shen, J.: Video saliency prediction using spatiotemporal residual attentive networks. IEEE Trans. Image Process. 29, 1113–1126 (2019)

    MathSciNet  Google Scholar 

  76. Bouwmans, T., Javed, S., Sultana, M., Soon, K.J.: A systematic review and comparative evaluation. Neural Netw. Deep Neural Netw. Concepts Backgr. Subtraction 117, 8–66 (2019)

    Google Scholar 

  77. Singh, R.P., Sharma, P., Madarkar, J.: Compute-extensive background subtraction for efficient ghost suppression. IEEE Access 7, 130180–130196 (2019)

    Google Scholar 

  78. Singh RP, Sharma P: A light-weight change detection method using ycbcr-based texture consensus model. Int. J. Pattern Recognit. Artif. Intell (2019). https://doi.org/10.1142/S0218001420500238

  79. Kalsotra, R., Arora, S.: A comprehensive survey of video datasets for background subtraction. IEEE Access 7, 59143–59171 (2019)

    Google Scholar 

  80. Chan, Y.-T., Wang, S.-J., Tsai, C.-H.: Real-time foreground detection approach based on adaptive ensemble learning with arbitrary algorithms for changing environments. Inf. Fusion 39, 154–167 (2018)

    Google Scholar 

  81. Shi, G., Huang, T., Dong, W., Jinjian, W., Xie, X.: Robust foreground estimation via structured Gaussian scale mixture modeling. IEEE Trans. Image Process. 27(10), 4810–4824 (2018)

    MathSciNet  MATH  Google Scholar 

  82. Lee, S., Lee, G., Yoo, J., Kwon, S.: Wisenetmd: motion detection using dynamic background region analysis. Symmetry 11(5), 621 (2019)

    Google Scholar 

  83. Zeng, D., Zhu, M., Fang, X., Zhou, T.: Extended scale invariant local binary pattern for background subtraction. IET Image Process. 12(8), 1292–1302 (2018)

    Google Scholar 

  84. De Gregorio, M., Giordano, M.: Wisardrp for change detection in video sequences. In: ESANN (2017)

  85. Wang, K., Gou, C., Wang, F.-Y.: \( M^{4} CD \): a robust change detection method for intelligent visual surveillance. IEEE Access 6, 15505–15520 (2018)

    Google Scholar 

  86. Isik, S., Özkan, K., Günal, S., Gerek, Ö.N.: SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos. J. Electron. Imaging 27(2), 023002 (2018)

    Google Scholar 

  87. Ramírez-Alonso, G., Chacón-Murguía, M.I.: Auto-adaptive parallel SOM architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175, 990–1000 (2016)

    Google Scholar 

  88. Allebosch, G., Van Hamme, D., Deboeverie, F., Veelaert, P., Philips, W.: C-EFIC: color and edge based foreground background segmentation with interior classification. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics, p. 433–454. Springer (2015)

  89. Sedky, M., Moniri, M., Chibelushi, C.C.: Spectral-360: a physics-based technique for change detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 399–402 (2014)

  90. Maddalena, L., Petrosino, A.: The sobs algorithm: what are the limits? In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 21–26. IEEE (2012)

  91. Wang, B., Dudek, P.: A fast self-tuning background subtraction algorithm. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 395–398 (2014)

  92. Bianco, S., Ciocca, G., Schettini, R.: Combination of video change detection algorithms by genetic programming. IEEE Trans. Evol. Comput. 21(6), 914–928 (2017)

    Google Scholar 

  93. Sajid, H.: Cheung S-CS: Universal multimode background subtraction. IEEE Trans. Image Process. 26(7), 3249–3260 (2017)

    MathSciNet  MATH  Google Scholar 

  94. Lu, X.: A multiscale spatio-temporal background model for motion detection. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3268–3271. IEEE (2014)

  95. Varadarajan, S., Miller, P., Zhou, H.: Spatial mixture of Gaussians for dynamic background modelling. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 63–68. IEEE (2013)

Download references

Funding

This work has been conducted with the research grants received from Ministry of Human Resource Development, India (MHRD) and Ministry of Electronics and Information Technology, India (MEITY).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rimjhim Padam Singh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, R.P., Sharma, P. Instance-vote-based motion detection using spatially extended hybrid feature space. Vis Comput 37, 1527–1543 (2021). https://doi.org/10.1007/s00371-020-01890-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01890-w

Keywords

Navigation