Abstract
Motion recognition, a trivial step employed in several video-based applications, is still a challenging task in real-world complex scenarios containing dynamic noise, varying backgrounds, shadows, improper illuminations, camouflages, etc. Numerous pixel-based change detection techniques employing varied combinations of different feature spaces have been proposed to efficiently overcome many real-world challenges. But ideally, handling all the possible real-world challenges simultaneously is yet to be achieved. Hence, this paper proposes a memory-efficient unique combination of multi-colour feature space with a light-weight intensity-based texture descriptor. The proposed spatially enlarged extended centre-symmetric local binary pattern is combined with YCbCr and RGB colour features for robust pixel representation. The proposed feature space is fed to an extended instance-vote technique for pixel classification. The random and time-subsampled update is employed conditionally for model update, followed by a feedback network that continuously optimizes the local threshold and learning rate parameters of the proposed model. The proposed feature space and model have been evaluated on whole 2014 Change Detection dataset, the largest known dataset. The outperforming performance and memory analysis strengthens its acceptability for real-time applications.
Similar content being viewed by others
References
Porikli, F., Tuzel, O.: Human body tracking by adaptive background models and mean-shift analysis. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, pp. 1–9. Citeseer (2003)
Wang, Y., Wei, X., Ding, L., Tang, X., Zhang, H.: A robust visual tracking method via local feature extraction and saliency detection. Vis. Comput. 36, 1–18 (2019)
Elsayed, R.A., Sayed, M.S., Abdalla, M.I.: Kin-based adaptive background subtraction for hand gesture segmentation. In: 2015 IEEE International Conference on Electronics, Circuits, and Systems (ICECS), pp. 33–36. IEEE (2015)
Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J.: Fall detection from human shape and motion history using video surveillance. In: 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW’07), vol. 2, pp. 875–880. IEEE (2007)
Reinders, F., Post, F.H., Spoelder, H.J.W.: Visualization of time-dependent data with feature tracking and event detection. Vis. Comput. 17(1), 55–71 (2001)
Shen, J., Peng, J., Dong, X., Shao, L., Porikli, F.: Higher order energies for image segmentation. IEEE Trans. Image Process. 26(10), 4911–4922 (2017)
Goyette, N., Jodoin, P.-M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection. net: a new change detection benchmark dataset. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8. IEEE (2012)
Wang, Y., Jodoin, P.-M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 387–394 (2014)
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), vol. 2, pp. 246–252. IEEE (1999)
KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Remagnino, P., Jones, G.A., Paragios, N., Regazzoni, C.S. (eds.) Video-Based Surveillance Systems, pp. 135–144. Springer, Berlin (2002)
Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol. 2, pp. 28–31. IEEE (2004)
Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)
Lee, D.-S.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 5, 827–832 (2005)
Oliver, N.M., Rosario, B., Pentland, A.P.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)
Zhou, X., Yang, C., Weichuan, Y.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 597–610 (2013)
Gao, Z., Cheong, L.-F., Wang, Y.-X.: Block-sparse RPCA for salient motion detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 1975–1987 (2014)
Kawabata, S., Hiura, S., Sato, K.: Real-time detection of anomalous objects in dynamic scene. In: 18th International Conference on Pattern Recognition, 2006. ICPR 2006, vol. 3, pp. 1171–1174. IEEE (2006)
He, J., Balzano, L., Szlam, A.: Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1568–1575. IEEE (2012)
Feng, J., Xu, H., Yan, S.: Online robust PCA via stochastic optimization. In: Advances in Neural Information Processing Systems, pp. 404–412 (2013)
Sun, Y., Tao, X., Li, Y., Jianhua, L.: Robust 2D principal component analysis: a structured sparsity regularized approach. IEEE Trans. Image Process. 24(8), 2515–2526 (2015)
Wang, J., Bebis, G., Nicolescu, M., Nicolescu, M., Miller, R.: Improving target detection by coupling it with tracking. Mach. Vis. Appl. 20(4), 205–223 (2009)
Wang, J., Bebis, G., Miller, R.: Robust video-based surveillance by integrating target detection with tracking. In: Computer Vision and Pattern Recognition Workshop, 2006. CVPRW’06. Conference on, pp. 137–137. IEEE (2006)
Lin, H.-H., Liu, T.-L., Chuang, J.-H.: A probabilistic SVM approach for background scene initialization. In: Proceedings. 2002 International Conference on Image Processing. 2002, vol. 3, pp. 893–896. IEEE (2002)
Lin, H.-H., Liu, T.-L., Chuang, J.-H.: Learning a scene background model via classification. IEEE Trans. Signal Process. 57(5), 1641–1654 (2009)
Tavakkoli, A., Nicolescu, M., Nicolescu, M., Bebis, G.: Incremental SVDD training: improving efficiency of background modeling in videos. In: International Conference on Signal and Image Processing, Kailua-Kona, HI, pp. 1–6 (2008)
Kim, K., Chalidabhongse, T.H., David, H., Larry, D.: Real-time foreground–background segmentation using codebook model. Real Time Imaging 11(3), 172–185 (2005)
Ilyas, A., Scuturici, M., Miguet, S.: Real time foreground-background segmentation using a modified codebook model. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009. AVSS’09, pp. 454–459. IEEE (2009)
Chacon-Murguia, M.I., Gonzalez-Duarte, S.: An adaptive neural-fuzzy approach for object detection in dynamic backgrounds for surveillance systems. IEEE Trans. Ind. Electron. 59(8), 3286–3298 (2011)
Zeng, J., Xie, L., Liu, Z.-Q.: Type-2 fuzzy Gaussian mixture models. Pattern Recognit. 41(12), 3636–3643 (2008)
El Baf, F., Bouwmans, T., Vachon, B.: Type-2 fuzzy mixture of Gaussians model: application to background modeling. In: International Symposium on Visual Computing, pp. 772–781. Springer (2008)
Zhao, Z., Bouwmans, T., Zhang, X., Fang, Y.: A fuzzy background modeling approach for motion detection in dynamic backgrounds. In: International Conference on Multimedia and Signal Processing, pp. 177–185. Springer (2012)
Luo, L., Zhao, Z.-Q., Li, X.-P.: A novel surveillance video processing using stochastic low-rank and generalized low-rank approximation techniques. In: 2018 International Conference on Machine Learning and Cybernetics (ICMLC), vol. 1, pp. 91–98. IEEE (2018)
Elgammal, A., Harwood, D., Davis, L.: Non-parametric model for background subtraction. In: European Conference on Computer Vision, pp. 751–767. Springer (2000)
Tavakkoli, A., Nicolescu, M., Bebis, G.: An adaptive recursive learning technique for robust foreground object detection. In: Proceedings of the International Workshop on Statistical Methods in Multi-image and Video Processing (in Conjunction with ECCV06) (2006)
Tavakkoli, A., Nicolescu, M., Bebis, G.: Robust recursive learning for foreground region detection in videos with quasi-stationary backgrounds. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 315–318. IEEE (2006)
Tanaka, T., Shimada, A., Arita, D., Taniguchi, R.-I.: Non-parametric background and shadow modeling for object detection. In: Asian Conference on Computer Vision, pp. 159–168. Springer (2007)
Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis. 39(1), 41–56 (2000)
Chen, M., Wei, X., Yang, Q., Li, Q., Wang, G., Yang, M.-H.: Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1518–1525 (2018)
Shen, J., Peng, J., Shao, L.: Submodular trajectories for better motion segmentation in videos. IEEE Trans. Image Process. 27(6), 2688–2700 (2018)
Shen, J., Dong, X., Peng, J., Jin, X., Shao, L., Porikli, F.: Submodular function optimization for motion clustering and image segmentation. IEEE Trans. Neural Netw. Learn. Syst. 30(9), 2637–2649 (2019)
Liu, Y., Shen, J., Wang, W., Sun, H., Shao, L.: Better dense trajectories by motion in videos. IEEE Trans. Cybern. 49(1), 159–170 (2017)
Wang, W., Shen, J., Porikli, F., Yang, R.: Semi-supervised video object segmentation with super-trajectories. IEEE Trans Pattern Anal Mach Intell 41(4), 985–998 (2018)
Liu, C., Wang, W., Shen, J., Shao, L.: Stereo video object segmentation using stereoscopic foreground trajectories. IEEE Trans. Cybern. 49(10), 3665–3676 (2018)
Wang, H., Suter, D.: A consensus-based method for tracking: modelling background scenario and foreground appearance. Pattern Recognit. 40(3), 1091–1105 (2007)
Wang, H., Suter, D.: Background subtraction based on a robust consensus method. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 223–226. IEEE (2006)
Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)
Van Droogenbroeck, M., Barnich, O.: ViBe: a disruptive method for background subtraction. In: Background Modeling and Foreground Detection for Video Surveillance, pp. 7–1 (2014)
Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–43. IEEE (2012)
Yang, S., Hao, K., Ding, Y., Liu, J.: Improved visual background extractor with adaptive range change. Memet. Comput. 10(1), 53–61 (2018)
Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)
Heikkilä, M., Pietikäinen, M., Schmid, C.: Description of interest regions with local binary patterns. Pattern Recognit. 42(3), 425–436 (2009)
Xue, G., Sun, J., Song, L.: Dynamic background subtraction based on spatial extended center-symmetric local binary pattern. In: 2010 IEEE International Conference on Multimedia and Expo, pp. 1050–1054. IEEE (2010)
Wang, L., Pan, C.: Fast and effective background subtraction based on \(\varepsilon \)lbp. In: International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2010 (2010)
Wang, L., Wu, H., Pan, C.: Adaptive \(\varepsilon \) LBP for background subtraction. In: Asian Conference on Computer Vision, ACCV 2010, pp. 560–571. Springer (2010)
Liao, S., Zhao, G., Kellokumpu, V., Pietikäinen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1301–1306. IEEE (2010)
Noh, S., Jeon, M.: A new framework for background subtraction using multiple cues. In: Asian Conference on Computer Vision, pp. 493–506. Springer (2012)
Hefeng, W., Liu, N., Luo, X., Jiawei, S., Chen, L.: Real-time background subtraction-based video surveillance of people by integrating local texture patterns. Signal Image Video Process. 8(4), 665–676 (2014)
Xue, G., Song, L., Sun, J., Wu, M.: Hybrid center-symmetric local pattern for dynamic background subtraction. In: 2011 IEEE International Conference on Multimedia and Expo, pp. 1–6. IEEE (2011)
Lee, Y., Jung, J., Kweon, I.-S.: Hierarchical on-line boosting based background subtraction. In: 2011 17th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), pp. 1–5. IEEE (2011)
Bilodeau, G.-A., Jodoin, J.-P., Saunier, N.: Change detection in feature space using local binary similarity patterns. In: International Conference on Computer and Robot Vision, CRV 2013, pp. 106–112. IEEE (2013)
St-Charles, P.-L., Bilodeau, G.-A.: Improving background subtraction using local binary similarity patterns. In: 2014 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 509–515. IEEE (2014)
Silva, C., Bouwmans, T., Frélicot, C.: An extended center-symmetric local binary pattern for background modeling and subtraction in videos. In: International Joint Conference on Computer Vision. VISAPP, Imaging and Computer Graphics Theory and Applications, p. 2015 (2015)
Silva, C., Bouwmans, T., Frélicot, C.: Online weighted one-class ensemble for feature selection in background/foreground separation. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2216–2221. IEEE (2016)
Bouwmans, T., Silva, C., Marghes, C., Zitouni, M.S., Bhaskar, H., Frelicot, C.: On the role and the importance of features for background modeling and foreground detection. Comput. Sci. Rev. 28, 26–91 (2018)
St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)
St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: Universal background subtraction using word consensus models. IEEE Trans. Image Process. 25(10), 4768–4781 (2016)
Jiang, S., Xiaobo, L.: Wesambe: a weight-sample-based method for background subtraction. IEEE Trans. Circuits Syst. Video Technol. 28(9), 2105–2115 (2018)
Braham, M., Van Droogenbroeck, M.: Deep background subtraction with scene-specific convolutional neural networks. In: 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–4. IEEE (2016)
Liao, J., Guo, G., Yan, Y., Wang, H.: Multiscale cascaded scene-specific convolutional neural networks for background subtraction. In: Pacific Rim Conference on Multimedia, pp. 524–533. Springer (2018)
Liang, X., Liao, S., Wang, X., Liu, W., Chen, Y., Li, S.Z.: Deep background subtraction with guided learning. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2018)
Dong, X., Shen, J., Dongming, W., Guo, K., Jin, X., Porikli, F.: Quadruplet network with one-shot learning for fast visual object tracking. IEEE Trans. Image Process. 28(7), 3516–3527 (2019)
Liang, Z., Shen, J.: Local semantic siamese networks for fast tracking. IEEE Trans. Image Process. 29, 3351–3364 (2019)
Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 459–474 (2018)
Lai, Q., Wang, W., Sun, H., Shen, J.: Video saliency prediction using spatiotemporal residual attentive networks. IEEE Trans. Image Process. 29, 1113–1126 (2019)
Bouwmans, T., Javed, S., Sultana, M., Soon, K.J.: A systematic review and comparative evaluation. Neural Netw. Deep Neural Netw. Concepts Backgr. Subtraction 117, 8–66 (2019)
Singh, R.P., Sharma, P., Madarkar, J.: Compute-extensive background subtraction for efficient ghost suppression. IEEE Access 7, 130180–130196 (2019)
Singh RP, Sharma P: A light-weight change detection method using ycbcr-based texture consensus model. Int. J. Pattern Recognit. Artif. Intell (2019). https://doi.org/10.1142/S0218001420500238
Kalsotra, R., Arora, S.: A comprehensive survey of video datasets for background subtraction. IEEE Access 7, 59143–59171 (2019)
Chan, Y.-T., Wang, S.-J., Tsai, C.-H.: Real-time foreground detection approach based on adaptive ensemble learning with arbitrary algorithms for changing environments. Inf. Fusion 39, 154–167 (2018)
Shi, G., Huang, T., Dong, W., Jinjian, W., Xie, X.: Robust foreground estimation via structured Gaussian scale mixture modeling. IEEE Trans. Image Process. 27(10), 4810–4824 (2018)
Lee, S., Lee, G., Yoo, J., Kwon, S.: Wisenetmd: motion detection using dynamic background region analysis. Symmetry 11(5), 621 (2019)
Zeng, D., Zhu, M., Fang, X., Zhou, T.: Extended scale invariant local binary pattern for background subtraction. IET Image Process. 12(8), 1292–1302 (2018)
De Gregorio, M., Giordano, M.: Wisardrp for change detection in video sequences. In: ESANN (2017)
Wang, K., Gou, C., Wang, F.-Y.: \( M^{4} CD \): a robust change detection method for intelligent visual surveillance. IEEE Access 6, 15505–15520 (2018)
Isik, S., Özkan, K., Günal, S., Gerek, Ö.N.: SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos. J. Electron. Imaging 27(2), 023002 (2018)
Ramírez-Alonso, G., Chacón-Murguía, M.I.: Auto-adaptive parallel SOM architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175, 990–1000 (2016)
Allebosch, G., Van Hamme, D., Deboeverie, F., Veelaert, P., Philips, W.: C-EFIC: color and edge based foreground background segmentation with interior classification. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics, p. 433–454. Springer (2015)
Sedky, M., Moniri, M., Chibelushi, C.C.: Spectral-360: a physics-based technique for change detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 399–402 (2014)
Maddalena, L., Petrosino, A.: The sobs algorithm: what are the limits? In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 21–26. IEEE (2012)
Wang, B., Dudek, P.: A fast self-tuning background subtraction algorithm. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 395–398 (2014)
Bianco, S., Ciocca, G., Schettini, R.: Combination of video change detection algorithms by genetic programming. IEEE Trans. Evol. Comput. 21(6), 914–928 (2017)
Sajid, H.: Cheung S-CS: Universal multimode background subtraction. IEEE Trans. Image Process. 26(7), 3249–3260 (2017)
Lu, X.: A multiscale spatio-temporal background model for motion detection. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3268–3271. IEEE (2014)
Varadarajan, S., Miller, P., Zhou, H.: Spatial mixture of Gaussians for dynamic background modelling. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 63–68. IEEE (2013)
Funding
This work has been conducted with the research grants received from Ministry of Human Resource Development, India (MHRD) and Ministry of Electronics and Information Technology, India (MEITY).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, R.P., Sharma, P. Instance-vote-based motion detection using spatially extended hybrid feature space. Vis Comput 37, 1527–1543 (2021). https://doi.org/10.1007/s00371-020-01890-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-020-01890-w