Skip to main content
SpringerLink
Log in

Single image deraining via deep shared pyramid network

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Single image deraining is a highly ill-posed problem. Existing deep neural network-based algorithms usually use larger deep models to solve this problem, which is less effective and efficient. In this paper, we propose a deep neural network based on feature pyramid to solve image deraining. Our algorithm is motivated that the features at different pyramid levels share similar structures. Based on this property, we develop an effective deep neural network, where the deep models at different feature pyramid levels share the same weight parameters. In addition, we further develop a multi-stream dilation convolution to deal with complex rainy streaks. To preserve the image detail, we develop dense connections that can maintain important features from different levels. Our algorithm is trained in an end-to-end manner. Quantitative and qualitative experimental results demonstrate that the proposed method performs favorably against state-of-the-art deraining methods in terms of accuracy as well as model sizes. The source code and dataset will be available at https://supercong94.wixsite.com/supercong94.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Althoff, M., Stursberg, O., Buss, M.: Model-based probabilistic collision detection in autonomous driving. IEEE Trans. Intell. Transp. Syst. 10(2), 299–310 (2009). https://doi.org/10.1109/TITS.2009.2018966

    Article  Google Scholar 

  2. Chen, Y., Hsu, C.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: ICCV, pp. 1968–1975 (2013). https://doi.org/10.1109/ICCV.2013.247

  3. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: CVPR, pp. 7103–7112 (2018). https://doi.org/10.1109/CVPR.2018.00742. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Chen_Cascaded_Pyramid_Network_CVPR_2018_paper.html

  4. Cong, W., Wu, Y., Cai, Y., Yao, G., Su, Z., Wang, H.: Single image deraining via deep pyramid network with spatial contextual information aggregation. Appl. Intell. 50, 1437–1447 (2020)

    Article  Google Scholar 

  5. Cui, Z., Chang, H., Shan, S., Zhong, B., Chen, X.: Deep network cascade for image super-resolution. In: ECCV, pp. 49–64 (2014). https://doi.org/10.1007/978-3-319-10602-1_4

  6. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  7. Fan, X., Tang, X., Hou, M., Luo, Z.: Fast example searching for input-adaptive data-driven dehazing with gaussian process regression. Vis. Comput. 35(4), 565–577 (2019). https://doi.org/10.1007/s00371-018-1485-y

    Article  Google Scholar 

  8. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017). https://doi.org/10.1109/TIP.2017.2691802

    Article  MathSciNet  MATH  Google Scholar 

  9. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: CVPR, pp. 1715–1723 (2017). https://doi.org/10.1109/CVPR.2017.186

  10. Gonzalez-Garcia, A., van de Weijer, J., Bengio, Y.: Image-to-image translation for cross-domain disentanglement. In: NeurIPS, pp. 1294–1305 (2018). URL http://papers.nips.cc/paper/7404-image-to-image-translation-for-cross-domain-disentanglement

  11. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141 (2018). https://doi.org/10.1109/CVPR.2018.00745. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.html

  12. Huang, D., Kang, L., Yang, M., Lin, C., Wang, Y.F.: Context-aware single image rain removal. In: ICME, pp. 164–169 (2012). https://doi.org/10.1109/ICME.2012.92

  13. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of psnr in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  14. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: ECCV, pp. 694–711 (2016). https://doi.org/10.1007/978-3-319-46475-6_43

  15. Kang, L., Lin, C., Fu, Y.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2012). https://doi.org/10.1109/TIP.2011.2179057

    Article  MathSciNet  MATH  Google Scholar 

  16. Kim, J., Lee, C., Sim, J., Kim, C.: Single-image deraining using an adaptive nonlocal means filter. In: ICIP, pp. 914–917 (2013). https://doi.org/10.1109/ICIP.2013.6738189

  17. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2015). URL http://arxiv.org/abs/1412.6980

  18. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: All-in-one dehazing network. In: ICCV, pp. 4780–4788 (2017). https://doi.org/10.1109/ICCV.2017.511

  19. Li, G., He, X., Zhang, W., Chang, H., Dong, L., Lin, L.: Non-locally enhanced encoder-decoder network for single image de-raining. In: ACM MM, pp. 1056–1064 (2018). https://doi.org/10.1145/3240508.3240636

  20. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV, pp. 262–277 (2018). https://doi.org/10.1007/978-3-030-01234-2_16

  21. Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: CVPR, pp. 2736–2744 (2016). https://doi.org/10.1109/CVPR.2016.299

  22. Lin, T., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: CVPR, pp. 936–944 (2017). https://doi.org/10.1109/CVPR.2017.106

  23. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., Berg, A.C.: SSD: single shot multibox detector. In: ECCV, pp. 21–37 (2016). https://doi.org/10.1007/978-3-319-46448-0_2

  24. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965

  25. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: ICCV, pp. 3397–3405 (2015). https://doi.org/10.1109/ICCV.2015.388

  26. Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network. In: CVPR, pp. 2720–2729 (2017). https://doi.org/10.1109/CVPR.2017.291

  27. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.: Single image dehazing via multi-scale convolutional neural networks. In: ECCV, pp. 154–169 (2016). https://doi.org/10.1007/978-3-319-46475-6_10

  28. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017). https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  29. Wang, B., Chen, S., Wang, J., Hu, X.: Residual feature pyramid networks for salient object detection. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01779-3

    Article  Google Scholar 

  30. Wang, X., Shrivastava, A., Gupta, A.: A-fast-rcnn: Hard positive generation via adversary for object detection. In: CVPR, pp. 3039–3048 (2017). https://doi.org/10.1109/CVPR.2017.324

  31. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  32. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: CVPR, pp. 1685–1694 (2017). https://doi.org/10.1109/CVPR.2017.183

  33. Yuan, Q., Li, J., Zhang, L., Wu, Z., Liu, G.: Blind motion deblurring with cycle generative adversarial networks. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01762-y

    Article  Google Scholar 

  34. Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network. In: CVPR, pp. 3194–3203 (2018). https://doi.org/10.1109/CVPR.2018.00337. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_Densely_Connected_Pyramid_CVPR_2018_paper.html

  35. Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: CVPR, pp. 695–704 (2018). https://doi.org/10.1109/CVPR.2018.00079. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_Density-Aware_Single_Image_CVPR_2018_paper.html

  36. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE TCSVT (2019)

  37. Zhang, S., He, F.: Drcdn: learning deep residual convolutional dehazing networks. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01774-8

    Article  Google Scholar 

  38. Zhang, S., He, F., Ren, W., Yao, J.: Joint learning of image detail and transmission map for single image dehazing. Vis. Comput. 36(2), 305–316 (2020). https://doi.org/10.1007/s00371-018-1612-9

    Article  Google Scholar 

  39. Zhang, S., Ren, W., Yao, J.: Feed-net: Fully end-to-end dehazing. In: IEEE ICME, pp. 1–6 (2018). https://doi.org/10.1109/ICME.2018.8486435

  40. Zhang, Y., Wang, L., Qi, J., Wang, D., Feng, M., Lu, H.: Structured siamese network for real-time visual tracking. In: ECCV, pp. 355–370 (2018). https://doi.org/10.1007/978-3-030-01240-3_22

Download references

Funding

This work was supported by the National Science and Technology Major Project [Grant Nos. 2018ZX04041001-007].

Author information

Authors and Affiliations

  1. Dalian University of Technology, Dalian, People’s Republic of China

    Cong Wang & Zhixun Su

  2. Tsinghua University, Beijing, People’s Republic of China

    Xiaoying Xing

  3. Chengdu University of Technology, Chengdu, People’s Republic of China

    Guangle Yao

  4. Guilin University of Electronic Technology, Guilin, People’s Republic of China

    Zhixun Su

Authors
  1. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoying Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangle Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhixun Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Wang.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Xing, X., Yao, G. et al. Single image deraining via deep shared pyramid network. Vis Comput 37, 1851–1865 (2021). https://doi.org/10.1007/s00371-020-01944-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01944-z

Keywords

Search

Navigation