Abstract
Fire is one of the disasters with the highest probability among natural disasters and social disasters. It poses a serious threat to human life and life safety. In order to reduce fire losses, a reliable fire warning method is particularly important. But due to huge variations of smoke in color, shapes, and texture and complex application environments, the existing methods still do not meet the application requirements well. To solve these problems, in this paper, we propose a two-stage real-time video smoke detection method base on dense optical flow and convolutional neural network. In the first stage, we propose a fast pre-positioning module to obtain suspicious smoke areas through the dynamic characteristics of smoke which can greatly reduce the subsequent computational complexity, and only extract the moving optical flow of suspicious smoke areas as the dynamic features of the smoke which reduce the subsequent processing time cost. Instead of simply using moving optical flow as the dynamic characteristics of smoke, we found that the optical flow of the blue channel (OFBC) can effectively reflect the motion characteristics of smoke, so we combine the OFBC of suspicious smoke areas with its three RGB color channels to form a quaternion matrix for subsequent classification. In the second stage, we choose ResNet as our pre-classifier, and a temporal enhanced adjustment algorithm was proposed as the pre-classified follow-up fine optimization module, which can fully utilize the characteristics of the smoke movement in the video to improve detection rate. The experimental results show that compared with the existing smoke detection methods, our proposed method achieves high detection rate and low false alarm rate.
Similar content being viewed by others
References
Chen JZ et al (2016) Dynamic smoke detection using cascaded convolutional neural network for surveillance videos. J Univ Electron Sci Technol China 46(6):992–996
Deng J, Dong W, Socher R, et al. (2009) ImageNet: a Large-Scale Hierarchical Image Database. 2009 IEEE computer society conference on computer vision and pattern recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA IEEE, 2009.
Farneback G (2003) Two-frame motion estimation based on polynomial expansion. Scandinavian Conference on Image Analysis 2749:363–370
Filonenko A, Kurnianggoro L, Jo K (2017) Comparative study of modern convolutional neural networks for smoke detection on image data. Proc. International Conference on Human System Interactions (HSI). pp. 64–68
Garg S, Kaur K, Kumar N, Rodrigues JJPC (2019) Hybrid deep-learning-based anomaly detection scheme for suspicious flow detection in SDN: a social multimedia perspective. IEEE Transactions on Multimedia 21(3):566–578
Garg S, Kaur K, Kumar N, Kaddoum G, Zomaya AY, Ranjan R (2019) A hybrid deep learning-based model for anomaly detection in cloud datacenter networks. IEEE Trans Netw Serv Manag 16(3):924–935
He KM, Zhang XY, Ren SQ, et al. (2016) Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IEEE Computer Society, 2016.
Hu YC, Lu XB (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimedia Tools and Applications 77:29283–29301
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37
Kaiser T (2000) Fire detection with temperature sensor arrays. Security Technology, 2000. Proceedings. IEEE 34th Annual 2000 International Carnahan Conference on IEEE
Khan S, Muhammad K, Mumtaz S, Baik SW, de Albuquerque VHC (2019) Energy-efficient deep CNN for smoke detection in foggy IoT environment. IEEE Internet Things J 6(6):9237–9245
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. International Conference on Learning Representations
Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Conference on Neural Information Processing Systems
Lin G, Zhang Y, Zhang Q, Jia Y, Xu G, Wang J (2017) Smoke detection in video sequences based on dynamic texture using volume local binary patterns. KSII Trans Internet Inf Syst 11(11):5522–5536
Lu X, et al. (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Munsell Color System (2009) In: Manutchehr-Danai M. Dictionary of Gems and Gemology
Ren SQ, He KM, Ross G et al (2015) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence 39(6):1137–1149
Ryser P, Pfister G (1991) Optical fire and security technology: sensor principles and detection intelligence. International Conference on Solid-state Sensors & Actuators IEEE
Tao CY, Zhang J, Wang P (2016) Smoke detection based on deep convolutional neural networks. International Conference on Industrial Informatics-computing Technology IEEE
Tung TX, Kim JM (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf J 46(5):276–282
Verstockt S, Van Hoecke S, Beji T et al (2013) A multi-modal video analysis approach for car park fire detection. Fire Saf J 57(3):44–57
Wang W, Lu X, Shen J, et al. (2019) Zero-shot video object segmentation via attentive graph neural networks. IEEE International Conference on Computer Vision (ICCV)
Yin ZJ, Wan BY, Yuan FN et al (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438
Yuan FN (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139
Zhang F, Qin W, Liu Y, et al. (2020) dual-channel convolution neural network for image smoke detection. Multimedia Tools and Applications
Zivkovic Z, Heijden FVD (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn Lett 27(7):773–780
Acknowledgments
This work was supported by the National Natural Science Foundation of Guangdong [Grant No.2018A030313994], and Guangzhou science and technology plan project [Grant No.202002030298].
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wu, Y., Chen, M., Wo, Y. et al. Video smoke detection base on dense optical flow and convolutional neural network. Multimed Tools Appl 80, 35887–35901 (2021). https://doi.org/10.1007/s11042-020-09870-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09870-x