Skip to main content
SpringerLink
Log in

Single Image Desmoking via Attentive Generative Adversarial Network for Smoke Detection Process

  • Published:
Fire Technology Aims and scope Submit manuscript

Abstract

Smoke removal has been a challenging problem which refers to the process of retrieving clear images from the smoky ones. It has an extensive demand while the performances of existing image desmoking methods have limitations owing to the homogeneous medium based model, hand-crafted features, and insufficient datasets. Unlike the existing desmoking methods using an atmospheric scattering model, the proposed method utilizes a general deblurring model. Inspired by Generative Adversarial Network (GAN), we propose an end-to-end attentive DesmokeGAN which implements the visual attention into the generative network to effectively learn the smoke features and their surroundings. The architecture of critic network is identical to PatchGAN by adding multi-component loss function to the image patch. Due to the scarce of various and quality smoke datasets, a graphics rendering engine is used to synthesize the smoky images. The quantitative and qualitative results show that the designed framework performs better than the recent state-of-the-art desmoking approaches on both synthetic and real images in indoor and outdoor scenes, especially for the thick smoky images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hasikin K, Mat Isa NA (2015) Adaptive fuzzy intensity measure enhancement technique for non-uniform illumination and low-contrast images. SIViP 9(6):1419–1442

    Article  Google Scholar 

  2. Rahman S et al (2016) (2016) An adaptive gamma correction for image enhancement. Eurasip J Image Video Process 1:35

    Article  Google Scholar 

  3. Seow M-J, Asari VK (2006) Ratio rule and homomorphic filter for enhancement of digital colour image. Neurocomputing 69(7):954–958

    Article  Google Scholar 

  4. Li S et al (2019) Real-time smoke removal for the surveillance images under fire scenario. SIViP 13(5):1037–1043

    Article  Google Scholar 

  5. Fattal R, (2008) Single image dehazing. In international conference on computer graphics and interactive techniques

  6. Baid A, et al., (2017) Joint desmoking, specularity removal, and denoising of laparoscopy images via graphical models and Bayesian inference. In international symposium on biomedical imaging. Melbourne, VIC

  7. Kotwal A, Bhalodia R and Awate SP (2016) Joint desmoking and denoising of laparoscopy images. In international symposium on biomedical imaging. Melbourne, VIC

  8. He K, Sun J, and Tang X (2009) Single image haze removal using dark channel prior. In: Computer vision and pattern recognition, pp 1956–1963

  9. Pei S, Chen Y, Chiu Y (2019) Single image desmoking using haze image model and human visual system. J Electron Imaging 28(04):043007

    Article  Google Scholar 

  10. Meng G et al. (2013) Efficient image dehazing with boundary constraint and contextual regularization. In international conference on computer vision

  11. Sulami M et al. (2014) Automatic recovery of the atmospheric light in hazy images. In IEEE international conference on computational photography Santa Clara, CA

  12. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533

    Article  MathSciNet  Google Scholar 

  13. Li B et al. (2017) AOD-Net: all-in-one dehazing network. In international conference on computer vision

  14. Chen D, et al. (2019) Gated context aggregation network for image dehazing and deraining. In IEEE winter conference on applications of computer vision

  15. He L, Bai J, Ru L (2019) Haze removal using aggregated resolution convolution network. IEEE Access 7:123698–123709

    Article  Google Scholar 

  16. Raj NB and Venkateswaran N (2018) Single image haze removal using a generative adversarial network arXiv: computer vision and pattern recognition

  17. Li R et al. (2018) Single image dehazing via conditional generative adversarial network. In computer vision and pattern recognition

  18. Zhang H, Sindagi VA and Patel VM (2017) Joint transmission map estimation and dehazing using deep networks. arXiv: computer vision and pattern recognition

  19. Engin D, Genc A, and Ekenel HK (2018) Cycle-dehaze: enhanced cycleGAN for single image dehazing. In computer vision and pattern recognition

  20. Yang X, Xu Z, and Luo J (2018) Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In national conference on artificial intelligence

  21. Tan M et al (2019) Image-dehazing method based on the fusion coding of contours and colors. IEEE Access 7:147857–147871

    Article  Google Scholar 

  22. Tchaka K, Pawar V and Stoyanov D (2017) Chromaticity based smoke removal in endoscopic images. In Proceedings of SPIE. Orlando, Florida, US

  23. Nishino K, Kratz L, Lombardi S (2012) Bayesian Defogging. Int J Comput Vision 98(3):263–278

    Article  MathSciNet  Google Scholar 

  24. Larochelle, H. and Hinton GE (2010) Learning to combine foveal glimpses with a third-order Boltzmann machine. In neural information processing systems

  25. Wang F et al. (2017) Residual attention network for image classification. In computer vision and pattern recognition

  26. Hu J et al. (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023

    Article  Google Scholar 

  27. Vaswani A et al. (2017) Attention is all you need. In neural information processing systems

  28. Parmar N et al. (2018) Image transformer. arXiv: computer vision and pattern recognition

  29. Wang X et al. (2018) Non-local neural networks. In computer vision and pattern recognition

  30. Zhang Y et al. (2018) Image super-resolution using very deep residual channel attention networks. In European conference on computer vision

  31. Qin X et al. (2019) FFA-Net: feature fusion attention network for single image dehazing. arXiv: computer vision and pattern recognition

  32. Goodfellow I et al. (2014) Generative adversarial nets. In advances in neural information processing systems

  33. Gulrajani I, et al. (2017) Improved training of wasserstein GANs. arXiv: Learning

  34. Isola, P., et al., (2017) Image-to-Image translation with conditional adversarial networks. In computer vision and pattern recognition. Honolulu, HI

  35. Ronneberger O, Fischer P, and Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In medical image computing and computer assisted intervention

  36. Mirza M and Osindero S, (2014) Conditional generative adversarial nets. arXiv: learning

  37. Bolkar S et al. (2018) Deep smoke removal from minimally invasive surgery videos. In proceedings of IEEE international conference on image processing

  38. Malav R et al. (2018) DHSGAN: An end to end dehazing network for fog and smoke. In: Jawahar C, Li H, Mori G, Schindler K (Eds) Asian conference on computer vision

  39. Sidorov O, Wang C, and Cheikh FA (2019) Generative smoke removal. arXiv: computer vision and pattern recognition

  40. Chen L et al (2019) De-smokeGCN: generative cooperative networks for joint surgical smoke detection and removal. IEEE Trans Med Imaging. 39(5):1–1

    Google Scholar 

  41. Cai B et al (2016) DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198

    Article  MathSciNet  Google Scholar 

  42. Tang K, Yang J and Wang J, (2014) Investigating haze-relevant features in a learning framework for image dehazing. In computer vision and pattern recognition

  43. He K et al. (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition

  44. Xu, B., et al., (2015) Empirical evaluation of rectified activations in convolutional network. arXiv: learning

  45. Ioffe S and Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv: learning

  46. Arjovsky M, Chintala S and Bottou L (2017) Wasserstein gan. ArXiv e-prints

  47. Mahapatra D et al. (2017) Image super resolution using generative adversarial networks and local saliency maps for retinal image analysis. In international conference on medical image computing and computer-assisted intervention. Springer

  48. Zeiler MD and Fergus R (2014) Visualizing and understanding convolutional networks. in European conference on computer vision. Springer

  49. Nah S, Hyun Kim T and Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition

  50. Ledig C et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition

  51. Zhang H, Sindagi VA, and Patel VM (2017) Image de-raining using a conditional generative adversarial network. arXiv: computer vision and pattern recognition

  52. Simonyan K and Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv: computer vision and pattern recognition

  53. Silberman N et al. (2012) Indoor segmentation and support inference from RGBD images. In european conference on computer vision

  54. Li B et al (2019) Benchmarking Single-Image Dehazing and Beyond. IEEE Trans Image Process 28(1):492–505

    Article  MathSciNet  Google Scholar 

  55. Abadi M et al. (2015) Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv: distributed, parallel, and cluster computing

  56. Kingma DP and Ba J (2014) Adam: a method for stochastic optimization. arXiv: learning

  57. Wang Z et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Shenyang Aerospace University, Daoyi South Avenue 37, Shenyang, 110136, Liaoning, China

    Yufeng Huang & Xiang Chen

  2. Shenyang Fire Research Institute of MEM, Wenda Road 218-20, Shenyang, 110136, Liaoning, China

    Lei Xu

  3. School of Safety Science and Emergency Management, Wuhan University of Technology, Luoshi Road 122, Wuhan, 430070, Hubei, China

    Kaiyuan Li

Authors
  1. Yufeng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kaiyuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yufeng Huang or Xiang Chen.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Chen, X., Xu, L. et al. Single Image Desmoking via Attentive Generative Adversarial Network for Smoke Detection Process. Fire Technol 57, 3021–3040 (2021). https://doi.org/10.1007/s10694-021-01096-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10694-021-01096-z

Keywords

Search

Navigation