Skip to main content
SpringerLink
Log in

Encoding features from multi-layer Gabor filtering for visual smoke recognition

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

It is a challenging task to accurately recognize smoke from visual scenes due to large variations in smoke shape, color and texture. To improve recognition accuracy, we propose a framework mainly with a robust local feature extraction module based on Gabor convolutional networks. We first propose a Gabor convolutional network, each layer of which mainly consists of Gabor convolution and feature fusion. To fuse feature maps generated by Gabor convolution, we present three Basic Grouping Methods, which divide the feature maps into several groups along orientation axis, scale axis and both of them. To avoid exponential growth of feature maps and preserve discriminative information simultaneously, we propose three element-wise aggregation functions, which are mean, min and max, to combine feature maps in each group. To further improve efficiency, we use local binary pattern to encode hidden and output maps of Gabor convolutional layers. In addition, we use a weight vector to optimize concatenation of histograms for further improvement. Experiments show that our method achieves very outstanding results on smoke, texture and material image datasets. Although the feature extraction step of our method is training-free, our framework consistently outperforms state-of-the-art methods on small smoke datasets, even including deep learning-based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Hwang U, Jeong J (2016) Computer vision based smoke detection method by using colour and object tracking. In: Proceedings of 5th international conference on information technology convergence and services (ITCS), Sydney, pp 53–59

  2. Jia Y, Yuan J, Wang J, Fang J, Zhang Q, Zhang Y (2016) A saliency-based method for early smoke detection in video sequences. Fire Technol 52(5):1–22

    Article  Google Scholar 

  3. Yu C, Fang J, Wang J, Zhang Y (2012) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663

    Google Scholar 

  4. Hu Y, Lu X (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimed Tools Appl 77(8):1–19

    Google Scholar 

  5. Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett 29(7):925–932

    Article  Google Scholar 

  6. Yuan F (2012) A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recognit 45(12):4326–4336

    Article  Google Scholar 

  7. Maruta H, Nakamura A, Kurokawa F (2010) A new approach for smoke detection with texture analysis and support vector machine. In: Proceedings of IEEE international symposium industrial electronics, Bari, Italy, pp 1550–1555

  8. Nguyen T, Kim J (2013) Multistage optical smoke detection approach for smoke alarm systems. Opt Eng 52(5):1–12

    Article  Google Scholar 

  9. Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139

    Article  Google Scholar 

  10. Yuan F, Shi J, Xia X, Fang Y, Fang Z, Mei T (2016) High-order local ternary patterns with locality preserving projection for smoke detection and image classification. Inf Sci 372:225–240

    Article  Google Scholar 

  11. Yuan F, Shi J, Xia X, Yang Y, Fang Y, Wang R (2016) Sub oriented histograms of local binary patterns for smoke detection and texture classification. KSII Trans Internet Inf Syst 10(4):1807–1823

    Google Scholar 

  12. Yuan F, Xia X, Shi J, Li H, Li G (2017) Non-linear dimensionality reduction and Gaussian process based classification method for smoke detection. IEEE Access 5:6833–6841

    Article  Google Scholar 

  13. Zhao Y, Zhou Z, Xu M (2015) Forest fire smoke video detection using spatiotemporal and dynamic texture features. J Electr Comput Eng 2015:1–7

    Google Scholar 

  14. Calderara S, Piccinini P, Cucchiara R (2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22:705–719

    Article  Google Scholar 

  15. Gonzalez-Gonzalez R, Alarcon-Aquino V, Rosas-Romero R, Starostenko O, Rodriguez-Asomoza J, Ramirez-Cortes J (2010) Wavelet-based smoke detection in outdoor video sequences. In: Proceedings of 53rd international midwest symposium on circuits and systems, midwest symposium on circuits and systems conference proceedings, IEEE, pp 383–387

  16. Gubbi J, Marusica S, Palaniswamia M (2009) Smoke detection in video using wavelets and support vector machines. Fire Saf J 44(8):1110–1115

    Article  Google Scholar 

  17. Maruta H, Kato Y, Nakamura A, Kurokawa F (2009) Smoke detection in open areas using its texture features and time series properties. In: Proceedings of IEEE international symposium on industrial electronics (ISIE), IEEE, pp 1904–1908

  18. Toreyin BU, Dedeoglu Y, Cetin A (2005) Wavelet based real-time smoke detection in video. In: Proceedings of European signal processing conference, Antalya, pp 4–8

  19. Ye S, Bai Z, Chen H, Bohush R, Ablameyko S (2017) An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognit Image Anal 27(1):131–138

    Article  Google Scholar 

  20. Tung T, Kim J (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf J 46(5):276–282

    Article  Google Scholar 

  21. Yin Z, Wan B, Yuan F, Xia X, Shi J (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438

    Article  Google Scholar 

  22. Yuan F, Fang Z, Wu S, Yang Z, Fang Y (2015) A real-time video smoke detection using staircase searching based dual threshold AdaBoost and dynamic analysis. IET Image Proc 9(10):849–856

    Article  Google Scholar 

  23. Dora L, Agrawal S, Panda R, Abraham A (2017) An evolutionary single Gabor kernel based filter approach to face recognition. Eng Appl Artif Intell 62:286–301

    Article  Google Scholar 

  24. Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 11:467–476

    Article  Google Scholar 

  25. Su Y, Shan S, Chen X, Gao W (2009) Hierarchical ensemble of global and local classifiers for face recognition. IEEE Trans Image Process 18:1885–1896

    Article  MathSciNet  MATH  Google Scholar 

  26. Hwang W, Huang X, Li S, Kim J (2015) Face recognition using extended curvature Gabor classifier bunch. Pattern Recognit 48(4):1247–1260

    Article  Google Scholar 

  27. Li C, Duan G, Zhong F (2015) Rotation invariant texture retrieval considering the scale dependence of Gabor wavelet. IEEE Trans Image Process 24(8):2344–2354

    Article  MathSciNet  MATH  Google Scholar 

  28. Jie Z, Qiang J, Nagy G (2007) A comparative study of local matching approach for face recognition. IEEE Trans Image Process 16:2617–2628

    Article  MathSciNet  Google Scholar 

  29. Cament L, Castillo L, Perez J, Galdames F, Perez C (2014) Fusion of local normalization and Gabor entropy weighted features for face identification. Pattern Recognit 47(2):568–577

    Article  Google Scholar 

  30. Shen L, Bai L, Auer D (2008) 3D Gabor wavelets for evaluating SPM normalization algorithm. Med Image Anal 12(3):375–383

    Article  Google Scholar 

  31. Zhan W, Shan S, Gao W, Chen X, Zhang H (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of 10th IEEE international conference on computer vision (ICCV), pp 786–791

  32. Elbakary M, Sundareshan M (2005) Accurate representation of local frequency using a computationally efficient Gabor filter fusion approach with application to image registration. Pattern Recognit Lett 26(14):2164–2173

    Article  Google Scholar 

  33. Wang H, Qi H, Wang X (2013) A new Gabor based approach for wood recognition. Neurocomputing 116:192–200

    Article  Google Scholar 

  34. Zhang B, Shan S, Chen X, Gao W (2007) Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Trans Image Process 16:57–68

    Article  MathSciNet  Google Scholar 

  35. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of advances in neural information processing systems, vol 25, pp 1097–1105

  36. Zeiler M, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of European conference on computer vision (ECCV), pp 818–833

  37. Balntas V, Riba E, Ponsa D, Mikolajczyk K (2016) Learning local feature descriptors with triplets and shallow convolutional neural networks. In: Proceedings of British machine vision conference (BMVC), vol 1, No. 2. pp 119.1–119.11

  38. Hu Y, Li C, Hu D, Yu W (2016) Gabor feature based convolutional neural network for object recognition in natural scene. In: Proceedings of international conference on information science and control engineering, IEEE, pp 386–390

  39. Wiskott L, Fellous JM, Kuiger N, Malsburg CVD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779

    Article  Google Scholar 

  40. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. Eur Conf Comput Vis 24(7):404–420

    MATH  Google Scholar 

  41. Caputo B, Hayman E, Mallikarjuna P (2005) Class-specific material categorisation. In: Proceedings of 10th IEEE international conference on computer vision, IEEE, vol 2, pp 1597–1604

  42. Guo Z, Zhang L, Zhang D (2010) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663

    Article  MathSciNet  MATH  Google Scholar 

  43. Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: A new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

    Article  MathSciNet  MATH  Google Scholar 

  44. Qian X, Hua X, Chen P, Ke L (2011) PLBP: An effective local binary patterns texture descriptor with pyramid representation. Pattern Recognit 44(10–11):2502–2515

    Article  Google Scholar 

  45. Zhang B, Gao Y, Zhao S, Liu J (2013) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544

    Article  MathSciNet  MATH  Google Scholar 

  46. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556

  47. Qi X, Xiao R, Li C, Qiao Y, Guo J, Tang X (2014) Pairwise rotation invariant co-occurrence local binary pattern. IEEE Trans Pattern Anal Mach Intell 36(11):2199–2211

    Article  Google Scholar 

  48. Ahonen T, Matas J, He C, Pietikäinen M (2009) Rotation invariant image description with local binary pattern histogram Fourier features. In: Proceedings of 16th image analysis, Scandinavian conference, Oslo, Norway, pp 61–70

  49. Zhang J, Liang J, Zhao H (2013) Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans Image Process 22(1):31–42

    Article  MathSciNet  MATH  Google Scholar 

  50. Crosier M, Griffin LD (2010) Using basic image features for texture classification. Int J Comput Vis 88(3):447–460

    Article  MathSciNet  Google Scholar 

  51. Sharma G, Hussain SU, Frédéric J (2012) Local higher-order statistics (LHS) for texture categorization and facial analysis. In: Proceedings of computer vision—European conference on computer vision. Springer, Berlin, pp 1–12

  52. Hong X, Zhao G, Pietikäinen M, Chen X (2014) Combining LBP difference and feature correlation for texture description. IEEE Trans Image Process 23(6):2557–2568

    Article  MathSciNet  MATH  Google Scholar 

  53. Ryu J, Hong S, Yang H (2015) Sorted consecutive local binary pattern for texture classification. IEEE Trans Image Process 24(7):2254–2265

    Article  MathSciNet  MATH  Google Scholar 

  54. Mehta R, Egiazarian K (2016) Texture classification using dense micro-block difference. IEEE Trans Image Process 25(4):1604–1616

    Article  MathSciNet  MATH  Google Scholar 

  55. Chen J, Shan S, He C, Zhao G, Pietikainen M, Chen X (2010) WLD: a robust local image descriptor. IEEE Trans Pattern Anal Mach Intell 32(9):1705–1720

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (61862029), the Open Project of Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (2018SDSJ01), the Natural Science Foundation of Jiangxi Province (20192BAB207011) and the Science Technology Projects of Jiangxi Education Department (GJJ170317, GJJ170892, GJJ180206).

Author information

Authors and Affiliations

  1. School of Information Technology, Jiangxi University of Finance and Economics, Nanchang, 330032, Jiangxi, China

    Feiniu Yuan, Gang Li, Xue Xia & Lin Zhang

  2. College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai, China

    Feiniu Yuan

  3. College of Mathematics and Computational Science, Yichun University, Yichun, 336000, Jiangxi, China

    Gang Li

  4. Vocational School of Teachers and Technology, Jiangxi Agricultural University, Nanchang, 330045, Jiangxi, China

    Jinting Shi

  5. School of Mathematics and Computer Science, Jiangxi Science and Technology Normal University, Nanchang, 330038, Jiangxi, China

    Lin Zhang

  6. Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, China Three Gorges University, Yichang, 443002, China

    Feiniu Yuan

Authors
  1. Feiniu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xue Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinting Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Feiniu Yuan or Gang Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, F., Li, G., Xia, X. et al. Encoding features from multi-layer Gabor filtering for visual smoke recognition. Pattern Anal Applic 23, 1117–1131 (2020). https://doi.org/10.1007/s10044-020-00864-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-020-00864-x

Keywords

Search

Navigation