Skip to main content
SpringerLink
Log in

Single image deraining via nonlocal squeeze-and-excitation enhancing network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Raindrop blur or rain streaks can severely degrade the visual quality of the images, which causes many practical vision systems to fail to work, such as autonomous driving and video surveillance. Hence, it is important to address the problem of single image de-raining. In this paper, we propose a novel deep network for single image de-raining. The proposed network consists of three stages, including encoder stage, Dense Non-Local Residual Block (DNLRB) stage, and decoder stage. As spatial contextual information has been analyzed to be meaningful for image de-raining (Huang et al. ??), we adopt squeeze-and-excitation enhancing on feature maps in each convolution layer for capturing spatial contextual information. In addition, to better leverage spatial contextual information for extracting rain components, the non-local mean operation has been embed in DNLRB. Both quantitative and qualitative experimental results demonstrate the proposed method performs favorably against the state-of-the-art de-raining methods. The source codes will be available at https://supercong94.wixsite.com/supercong94.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Bossu J, Hautière N, Tarel J (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93(3):348–367. https://doi.org/10.1007/s11263-011-0421-7

    Article  Google Scholar 

  2. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising with multi-layer perceptrons, part 1: comparison with existing algorithms and with bounds. arXiv:1211.1544

  3. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising with multi-layer perceptrons, part 2: training trade-offs and analysis of their mechanisms. arXiv:1211.1552

  4. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198. https://doi.org/10.1109/TIP.2016.2598681

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen C, Chen Q, Xu J, Koltun V (2018) Learning to see in the dark. In: CVPR., pp 3291–3300 https://doi.org/10.1109/CVPR.2018.00347

  6. Chen J, Tan C, Hou J, Chau L, Li H (2018) Robust video content alignment and compensation for rain removal in a CNN framework. In: CVPR, pp 6286–6295. https://doi.org/10.1109/CVPR.2018.00658

  7. Chen Y, Hsu C (2013) A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: ICCV, pp 1968–1975. https://doi.org/10.1109/ICCV.2013.247

  8. Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. In: ECCV, pp 49–64. https://doi.org/10.1007/978-3-319-10602-1_4

  9. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: ECCV, pp 184–199. https://doi.org/10.1007/978-3-319-10593-2_13

  10. Fu X, Huang J, Ding X, Liao Y, Paisley J (2017) Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956. https://doi.org/10.1109/TIP.2017.2691802

    Article  MathSciNet  MATH  Google Scholar 

  11. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: CVPR, pp 1715–1723. https://doi.org/10.1109/CVPR.2017.186

  12. Garg K, Nayar SK (2004) Detection and removal of rain from videos. In: CVPR, pp 528–535. https://doi.org/10.1109/CVPR.2004.79

  13. Garg K, Nayar SK (2007) Vision and rain. IJCV 75(1):3–27. https://doi.org/10.1007/s11263-006-0028-6

    Article  Google Scholar 

  14. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: CVPR, pp 7132–7141. https://doi.org/10.1109/CVPR.2018.00745

  15. Huang D, Kang L, Yang M, Lin C, Wang YF (2012) Context-aware single image rain removal. In: ICME, pp 164–169. https://doi.org/10.1109/ICME.2012.92

  16. Jain V, Seung HS (2008) Natural image denoising with convolutional networks. In: NIPS, pp 769–776

  17. Johnson J, Alahi A, Fei-Fei L. (2016) Perceptual losses for real-time style transfer and super-resolution. In: ECCV, pp 694–711. https://doi.org/10.1007/978-3-319-46475-6_43

  18. Kang L, Lin C, Fu Y (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755. https://doi.org/10.1109/TIP.2011.2179057

    Article  MathSciNet  MATH  Google Scholar 

  19. Kim J, Lee C, Sim J, Kim C (2013) Single-image deraining using an adaptive nonlocal means filter. In: ICIP, pp 914–917. https://doi.org/10.1109/ICIP.2013.6738189

  20. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  21. Li G., He X., Zhang W., Chang H., Dong L., Lin L. (2018) Non-locally enhanced encoder-decoder network for single image de-raining. In: ACM MM, pp 1056–1064. https://doi.org/10.1145/3240508.3240636. http://doi.acm.org/10.1145/3240508.3240636

  22. Li S, Araujo IB, Ren W, Wang Z, Tokuda EK, Junior RH, Cesar-Junior R, Zhang J, Guo X, Cao X (2019) Single image deraining: a comprehensive benchmark analysis CVPR

  23. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV, pp 262–277. https://doi.org/10.1007/978-3-030-01234-2_16

  24. Li Y, Tan RT, Guo X, Lu J, Brown MS (2016) Rain streak removal using layer priors. In: CVPR, pp 2736–2744. https://doi.org/10.1109/CVPR.2016.299

  25. Liu Z, Yeh RA, Tang X, Liu Y, Agarwala A (2017) Video frame synthesis using deep voxel flow. In: ICCV, pp 4473–4481. https://doi.org/10.1109/ICCV.2017.478

  26. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: ICCV, pp 3397–3405. https://doi.org/10.1109/ICCV.2015.388

  27. Mustaniemi J, Kannala J, Särkkä S, Matas J, Heikkilä J (2018) Inertial-aided motion deblurring with deep networks. CoRR arXiv:1810.00986

  28. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. In: CVPR

  29. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M (2016) Single image dehazing via multi-scale convolutional neural networks. In: ECCV, pp 154–169. https://doi.org/10.1007/978-3-319-46475-6_10

  30. Sharma V, Diba A, Neven D, Brown MS, Gool LV, Stiefelhagen R (2018) Classification-driven dynamic image enhancement. In: CVPR, pp 4033–4041. https://doi.org/10.1109/CVPR.2018.00424

  31. Su S, Delbracio M, Wang J, Sapiro G, Heidrich W, Wang O (2017) Deep video deblurring for hand-held cameras. In: CVPR, pp 237–246. https://doi.org/10.1109/CVPR.2017.33

  32. Tao X, Gao H, Liao R, Wang J, Jia J (2017) Detail-revealing deep video super-resolution. In: ICCV, pp 4482–4490. https://doi.org/10.1109/ICCV.2017.479

  33. Tripathi AK, Mukhopadhyay S (2014) Removal of rain from videos: a review. SIViP 8(8):1421–1430. https://doi.org/10.1007/s11760-012-0373-6

    Article  Google Scholar 

  34. Wang X, Girshick RB, Gupta A, He K (2018) Non-local neural networks. In: CVPR, pp 7794–7803. https://doi.org/10.1109/CVPR.2018.00813

  35. Xu N, Price BL, Cohen S, Huang TS (2017) Deep image matting. In: CVPR, pp 311–320. https://doi.org/10.1109/CVPR.2017.41

  36. Yang W, Tan RT, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: CVPR, pp 1685–1694. https://doi.org/10.1109/CVPR.2017.183

  37. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: CVPR. https://doi.org/10.1109/CVPR.2018.00079om/content_cvpr_2018/html/Zhang_Density-Aware_Single_Image_, pp 695–704

  38. Zhang H, Sindagi V, Patel VM (2017) Image de-raining using a conditional generative adversarial network. arXiv:1701.05957

  39. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: ECCV, pp 294–310. https://doi.org/10.1007/978-3-030-01234-2_18

  40. Zhu L, Fu C, Lischinski D, Heng P (2017) Joint bi-layer optimization for single-image rain streak removal. In: ICCV, pp 2545–2553. https://doi.org/10.1109/ICCV.2017.276

Download references

Author information

Authors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Cong Wang, Wanshu Fan, Honghe Zhu & Zhixun Su

  2. Guilin University of Electronic Technology, Guilin, China

    Zhixun Su

Authors
  1. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanshu Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Honghe Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhixun Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhixun Su.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Natural Science Foundation of China [grant numbers 61572099]; Major National Science and Technology Project of China [grant number 2018ZX04016001-011].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Fan, W., Zhu, H. et al. Single image deraining via nonlocal squeeze-and-excitation enhancing network. Appl Intell 50, 2932–2944 (2020). https://doi.org/10.1007/s10489-020-01693-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01693-5

Keywords

Search

Navigation