Skip to main content
SpringerLink
Log in

Adjacent LBP and LTP based background modeling with mixed-mode learning for foreground detection

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Detection of video objects under bad weather and poor illumination condition is a challenging task. We address this issue using the notion of background modeling. LBP-based background modeling and model learning has been used to detect the video objects but performance degrades in the above complex background. We propose the adjacency- and reinforced adjacency-based variants of LBP for complex real-world background modeling and model learning for object detection. In this regard, we have proposed the four variants; (1) enhanced adjacent local binary pattern, (2) enhanced reinforced adjacent local binary pattern, (3) enhanced adjacent local ternary pattern, and (4) enhanced reinforced adjacent local ternary pattern. Besides, we have embedded the Gabor and LBP features to obtain an embedded feature, which is subsequently used with the notions of adjacency. Unlike the background learning approach where the model learns only the background, our model learning algorithm learns the background together with the foreground objects and hence named as mixed-mode learning strategy. These models together with the new learning strategy are tested with CD2014 (snowfall and blizzard) and PETS (2014 and 2016) data sets, and the performance of the proposed models has been compared with LBP-based methods and other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Wren CR, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19:780–785

    Article  Google Scholar 

  2. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real time tracking. In: IEEE conference on computer vision and pattern recognition, pp 246–252

  3. Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163

    Article  Google Scholar 

  4. Zivkovic Z, Heijden F (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit Lett 27:773–780

    Article  Google Scholar 

  5. Parag T, Elgammal A, Mittal A (2006) A framework for feature selection for background subtraction. In: IEEE conference on computer vision and pattern recognition, pp 1916–1923

  6. Bouwmans T, Silva C, Marghes C, Zitouni MS, Bhaskar H, Frelicot C (2018) On the role and the importance of features for background modeling and foreground detection. Comput Sci Rev 28:26–91

    Article  MathSciNet  Google Scholar 

  7. Ojala T, Pietikinen M, Harwood D (1996) A comparative study of texture measure with classifier based on feature distributions. Pattern Recognit 29:51–59

    Article  Google Scholar 

  8. Heikkil M, Pietiinen M (2006) A texture based method for modeling the background and detecting moving objects. IEEE Trans Pattern Anal Mach Intell 28:657–662

    Article  Google Scholar 

  9. Heikkil M, Pietikinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recognit 42:425–436

    Article  Google Scholar 

  10. Xue G, Song L, Sun J, Wu M (2011) Hybrid center-symmetric local pattern for dynamic background subtraction. In: IEEE international conference on multimedia and expo, pp 1–6

  11. Silva C, Bouwmans T, Frelicot C (2015) An eXtended center-symmetric local binary pattern for background modeling and subtraction in videos. In: International joint conference on computer vision, imaging and computer graphics theory and applications

  12. Nguyen D T, Zong Z, Ogunbona P, Li W (2010) Object detection using non-redundant local binary patterns. In: IEEE international conference on image processing, pp 4609–4612

  13. Wang L, Pan C (2010) Fast and effective background subtraction based on \(\epsilon \)LBP. In: IEEE international conference on acoustics, speech, and signal processing, pp 1394–1397

  14. Acharya S, Priyadarsini PS, Nanda PK (2016) Adjacent and functional LBP based background model learning for video object detection. In: IEEE region 10 conference (TENCON), pp 781–784

  15. Guo L, Xu D, Qiang Z (2016) Background subtraction using local SVD binary pattern. In: IEEE conference on computer vision and pattern recognition workshops, pp 1159–1167

  16. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19:1635–1650

    Article  MathSciNet  Google Scholar 

  17. Davarpanah SH, Khalid F, Abdullah LN, Golchin M (2016) A texture descriptor: background local binary pattern (BGLBP). Multimed Tools Appl 15:6549–6568

    Article  Google Scholar 

  18. Yeh C, Lin C, Muchtar K, Kang L (2014) Real-time background modeling based on a multi-level texture description. Inf Sci 269:106–127

    Article  MathSciNet  Google Scholar 

  19. Liu WC, Lin SZ, Yang MH, Huang CR (2013) Real-time binary descriptor based background modeling. In: IEEE Asian conference on pattern recognition, pp 722–726

  20. Zhao G, Ahonen T, Matas J, Pietikainen M (2012) Rotation-invariant image and video description with local binary pattern features. IEEE Trans Image Proces 21:1465–1477

    Article  MathSciNet  Google Scholar 

  21. Liao S, Zhao G, Kellokumpu V, Pietiknen M, Li SZ (2010) Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: IEEE conference on computer vision and pattern recognition, pp 1301–1306

  22. Zhang S, Yao H, Liu S (2008) Dynamic background modeling and subtraction using spatio-temporal local binary patterns. In: IEEE international conference on image processing, pp 1556–1559

  23. Mandal M, Nanda PK (2015) Embedded local feature based background modeling for video object detection. In: IEEE conference on power, communication and information technology, pp 691–696

  24. Chen Y, Wang J, Lu H (2015) Learning sharable models for robust background subtraction. In: IEEE international conference on multimedia and expo, pp 1–6

  25. Sajid H, Cheung SCS (2017) Universal multimode background subtraction. IEEE Trans Image Process 26:3249–3260

    Article  MathSciNet  Google Scholar 

  26. Jiang S, Lu X (2018) WeSamBE: a weight-sample-based method for background subtraction. IEEE Trans Circuits Syst Video Technol 28:2105–2115

    Article  Google Scholar 

  27. Wang Y, Jodoin PM, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) CDnet 2014: an expanded change detection benchmark dataset. In: IEEE conference on computer vision and pattern recognition workshops, pp 393–400

  28. Bouwmans T (2014) Traditional and recent approaches in background modeling for foreground detection: an overview. Comput Sci Rev 11–12:31–66

    Article  Google Scholar 

  29. Xu Y, Dong J, Zhang B, Xu D (2016) Background modeling methods in video analysis: a review and comparative evaluation. CAAI Trans Intell Technol 1(1):43–60

    Article  Google Scholar 

  30. Sheikh Y, Shah M (2005) Bayesian modeling of dynamic scenes for object detection. IEEE Trans Pattern Anal Mach Intell 27:1778–1792

    Article  Google Scholar 

  31. Subudhi BN, Nanda PK, Ghosh A (2011) A change information based fast algorithm for video object detection and tracking. IEEE Trans Circuits Syst Video Technol 21:993–1004

    Article  Google Scholar 

  32. Zhong Z, Xu Y, Li Z, Zhao Y (2017) Background modelling using discriminative motion representation. IET Comput Vis 11:463–470

    Article  Google Scholar 

  33. Barnich O, Van Droogenbroeck M (2011) ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20:1709–1724

    Article  MathSciNet  Google Scholar 

  34. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: IEEE conference on computer vision and pattern recognition workshops, pp 38–43

  35. Ojala T, Pietiinen M, Ma T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987

    Article  Google Scholar 

  36. Brahnam S, Jain LC, Nanni L, Lumini A (2014) Local binary patterns: new variants and applications. Springer, London

    Book  Google Scholar 

  37. Pietiinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns. Springer, London

    Book  Google Scholar 

  38. Liu L, Fieguth P, Guo Y, Wang X, Pietiinen M (2017) Local binary features for texture classification: taxonomy and experimental study. Pattern Recognit 62:135–160

    Article  Google Scholar 

  39. Heikki M, Pietiinen M, Heikki J (2004) A texture based method for detecting moving objects. In: British machine vision conference, pp 187–196

  40. Hassaballah M, Kenk MA, El-Henawy IM (2020) Local binary pattern-based on-road vehicle detection in urban traffic scene. Pattern Anal Appl. https://doi.org/10.1007/s10044-020-00874-9

    Article  Google Scholar 

  41. Yao J, Odobez JM (2007) Multi-layer background subtraction based on color and texture. In: IEEE conference on computer vision and pattern recognition, pp 1–8

  42. He W, Kim YK, Ko H, Wu J, Li W, Tu B (2019) Local compact binary count based nonparametric background modeling for foreground detection in dynamic scenes. IEEE Access 7:92329–92340

    Article  Google Scholar 

  43. Deboeverie F, Allebosch G, Van Haerenborgh D, Veelaert P, Philips W (2014) Edge-based foreground detection with higher order derivative local binary patterns for low-resolution video processing. In: IEEE international conference on computer vision theory and applications, pp 339–346

  44. Satpathy A, Jiang X, Eng HL (2014) LBP-based edge-texture features for object recognition. IEEE Trans Image Process 23:1953–1964

    Article  MathSciNet  Google Scholar 

  45. Yin H, Yang H, Su H, Zhang C (2013) Dynamic background subtraction based on appearance and motion pattern. In: IEEE international conference on multimedia and expo workshops, pp 1–6

  46. Takala V, Pietikainen M (2007) Multi-object tracking using color, texture and motion. In: IEEE conference on computer vision and pattern recognition, pp 1–7

  47. Ji Z, Wang W (2014) Detect foreground objects via adaptive fusing model in a hybrid feature space. Pattern Recognit 47:2952–2961

    Article  Google Scholar 

  48. Wu H, Liu N, Luo X, Su J, Chen L (2014) Real-time background subtraction-based video surveillance of people by integrating local texture patterns. Signal Image Video Process 8:665–676

    Article  Google Scholar 

  49. Chen Y, Wang J, Li J, Lu H (2015) Multiple features based shared models for background subtraction. In: IEEE international conference on image processing, pp 3946–3950

  50. Zhang B, Gao Y, Zhao S, Zhong B (2011) Kernel similarity modeling of texture pattern flow for motion detection in complex background. IEEE Trans Circuits Syst Video Technol 21:29–38

    Article  Google Scholar 

  51. Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23:3191–3202

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Image and Video Analysis Lab, Department of Electronics and Communication Engineering, Siksha O Anusandhan (Deemed to be University), Bhubaneswar, Odisha, India

    Subhabrata Acharya & Pradipta Kumar Nanda

Authors
  1. Subhabrata Acharya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradipta Kumar Nanda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Subhabrata Acharya.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Acharya, S., Nanda, P.K. Adjacent LBP and LTP based background modeling with mixed-mode learning for foreground detection. Pattern Anal Applic 24, 1047–1074 (2021). https://doi.org/10.1007/s10044-021-00967-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-021-00967-z

Keywords

Search

Navigation