Skip to main content

Advertisement

Log in

Video smoke detection base on dense optical flow and convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fire is one of the disasters with the highest probability among natural disasters and social disasters. It poses a serious threat to human life and life safety. In order to reduce fire losses, a reliable fire warning method is particularly important. But due to huge variations of smoke in color, shapes, and texture and complex application environments, the existing methods still do not meet the application requirements well. To solve these problems, in this paper, we propose a two-stage real-time video smoke detection method base on dense optical flow and convolutional neural network. In the first stage, we propose a fast pre-positioning module to obtain suspicious smoke areas through the dynamic characteristics of smoke which can greatly reduce the subsequent computational complexity, and only extract the moving optical flow of suspicious smoke areas as the dynamic features of the smoke which reduce the subsequent processing time cost. Instead of simply using moving optical flow as the dynamic characteristics of smoke, we found that the optical flow of the blue channel (OFBC) can effectively reflect the motion characteristics of smoke, so we combine the OFBC of suspicious smoke areas with its three RGB color channels to form a quaternion matrix for subsequent classification. In the second stage, we choose ResNet as our pre-classifier, and a temporal enhanced adjustment algorithm was proposed as the pre-classified follow-up fine optimization module, which can fully utilize the characteristics of the smoke movement in the video to improve detection rate. The experimental results show that compared with the existing smoke detection methods, our proposed method achieves high detection rate and low false alarm rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen JZ et al (2016) Dynamic smoke detection using cascaded convolutional neural network for surveillance videos. J Univ Electron Sci Technol China 46(6):992–996

    Google Scholar 

  2. Deng J, Dong W, Socher R, et al. (2009) ImageNet: a Large-Scale Hierarchical Image Database. 2009 IEEE computer society conference on computer vision and pattern recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA IEEE, 2009.

  3. Farneback G (2003) Two-frame motion estimation based on polynomial expansion. Scandinavian Conference on Image Analysis 2749:363–370

    Article  Google Scholar 

  4. Filonenko A, Kurnianggoro L, Jo K (2017) Comparative study of modern convolutional neural networks for smoke detection on image data. Proc. International Conference on Human System Interactions (HSI). pp. 64–68

  5. Garg S, Kaur K, Kumar N, Rodrigues JJPC (2019) Hybrid deep-learning-based anomaly detection scheme for suspicious flow detection in SDN: a social multimedia perspective. IEEE Transactions on Multimedia 21(3):566–578

    Article  Google Scholar 

  6. Garg S, Kaur K, Kumar N, Kaddoum G, Zomaya AY, Ranjan R (2019) A hybrid deep learning-based model for anomaly detection in cloud datacenter networks. IEEE Trans Netw Serv Manag 16(3):924–935

    Article  Google Scholar 

  7. He KM, Zhang XY, Ren SQ, et al. (2016) Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IEEE Computer Society, 2016.

  8. Hu YC, Lu XB (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimedia Tools and Applications 77:29283–29301

    Article  Google Scholar 

  9. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37

  10. Kaiser T (2000) Fire detection with temperature sensor arrays. Security Technology, 2000. Proceedings. IEEE 34th Annual 2000 International Carnahan Conference on IEEE

  11. Khan S, Muhammad K, Mumtaz S, Baik SW, de Albuquerque VHC (2019) Energy-efficient deep CNN for smoke detection in foggy IoT environment. IEEE Internet Things J 6(6):9237–9245

    Article  Google Scholar 

  12. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. International Conference on Learning Representations

  13. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Conference on Neural Information Processing Systems

  14. Lin G, Zhang Y, Zhang Q, Jia Y, Xu G, Wang J (2017) Smoke detection in video sequences based on dynamic texture using volume local binary patterns. KSII Trans Internet Inf Syst 11(11):5522–5536

    Google Scholar 

  15. Lu X, et al. (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  16. Munsell Color System (2009) In: Manutchehr-Danai M. Dictionary of Gems and Gemology

  17. Ren SQ, He KM, Ross G et al (2015) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence 39(6):1137–1149

    Article  Google Scholar 

  18. Ryser P, Pfister G (1991) Optical fire and security technology: sensor principles and detection intelligence. International Conference on Solid-state Sensors & Actuators IEEE

  19. Tao CY, Zhang J, Wang P (2016) Smoke detection based on deep convolutional neural networks. International Conference on Industrial Informatics-computing Technology IEEE

  20. Tung TX, Kim JM (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf J 46(5):276–282

    Article  Google Scholar 

  21. Verstockt S, Van Hoecke S, Beji T et al (2013) A multi-modal video analysis approach for car park fire detection. Fire Saf J 57(3):44–57

    Article  Google Scholar 

  22. Wang W, Lu X, Shen J, et al. (2019) Zero-shot video object segmentation via attentive graph neural networks. IEEE International Conference on Computer Vision (ICCV)

  23. Yin ZJ, Wan BY, Yuan FN et al (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438

    Article  Google Scholar 

  24. Yuan FN (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139

    Article  Google Scholar 

  25. Zhang F, Qin W, Liu Y, et al. (2020) dual-channel convolution neural network for image smoke detection. Multimedia Tools and Applications

  26. Zivkovic Z, Heijden FVD (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn Lett 27(7):773–780

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of Guangdong [Grant No.2018A030313994], and Guangzhou science and technology plan project [Grant No.202002030298].

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yan Wo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Chen, M., Wo, Y. et al. Video smoke detection base on dense optical flow and convolutional neural network. Multimed Tools Appl 80, 35887–35901 (2021). https://doi.org/10.1007/s11042-020-09870-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09870-x

Keywords

Navigation