Skip to main content
SpringerLink
Log in

Shading-aware shadow detection and removal from a single image

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Shadow removal is a challenging problem due to its sensitivity to lighting and material conditions. In this paper, we propose a shading-aware shadow processing algorithm, which can automatically detect and remove complex shadows from a single color image. Our framework consists of two key steps. We firstly conduct a shadow-preserving filter upon the image which will effectively remove the image texture while preserving the shadow and shading information. Shadow regions are estimated by establishing a confidence map from the filtered image incorporating depth cue. We then develop a shading-aware optimization framework to remove shadows and recover shading in these regions. The extensive experimental results show that the proposed algorithm produces visually compelling results in a series of challenging images and it can handle complex shadows in both indoor and outdoor scenes. Quantitative and qualitative comparisons with current state-of-the-art methods strongly demonstrate the efficacy of our proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Anat, L., Dani, L., Yair, W.: A closed-form solution to natural image matting. IEEE TPAMI 30(2), 228–242 (2007)

    Google Scholar 

  2. Arbel, E., Helor, H.: Shadow removal using intensity surfaces and texture anchor points. IEEE TPAMI 33(6), 1202–1216 (2011)

    Article  Google Scholar 

  3. Barron, J.T., Malik, J.: Intrinsic scene properties from a single rgb-d image. In: CVPR, pp. 17–24 (2013)

  4. Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: ICCV, pp. 241–248 (2013)

  5. Cho, H., Lee, H., Kang, H., Lee, S.: Bilateral texture filtering. ACM TOG 33(4), 1–8 (2014)

    Article  Google Scholar 

  6. Chuang, Y.Y., Goldman, D.B., Curless, B., Salesin, D.H., Szeliski, R.: Shadow matting and compositing. In: ACM SIGGRAPH, pp. 494–500 (2003)

  7. Cucchiara, R., Grana, C., Piccardi, M., Prati, A., Sirotti, S.: Improving shadow suppression in moving object detection with HSV color information. In: Intelligent Transportation Systems, pp. 334–339 (2002)

  8. Cun, X., Pun, C.M., Shi, C.: Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN. AAAI (2020)

  9. Ding, B., Long, C., Zhang, L., Xiao, C.: Argan: attentive recurrent generative adversarial network for shadow detection and removal. In: ICCV, pp. 10213–10222 (2019)

  10. Eh, L., Jj, M.: Lightness and retinex theory. J. Opt. Soc. Am. 61(1), 1–11 (1971)

    Article  Google Scholar 

  11. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE TPAMI 28(1), 59–68 (2006)

    Article  Google Scholar 

  12. Godard, C., Aodha, O.M., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: CVPR, pp. 6602–6611 (2017)

  13. Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth estimation. In: ICCV, pp. 3828–3838 (2019)

  14. Gryka, M., Terry, M., Brostow, G.J.: Learning to remove soft shadows. ACM TOG 34(5), 1–15 (2015)

    Article  Google Scholar 

  15. Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. In: CVPR, pp. 2033–2040 (2011)

  16. Hachama, M., Ghanem, B., Wonka, P.: Intrinsic scene decomposition from rgb-d images. In: ICCV, pp. 810–818 (2015)

  17. Hu, X., Fu, C.W., Zhu, L., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection and removal. In: IEEE TPAMI, pp. 7454–7462 (2019)

  18. Hu, X., Jiang, Y., Fu, C.W., Heng, P.A.: Mask-shadowgan: learning to remove shadows from unpaired data. In: ICCV, pp. 2472–2481 (2019)

  19. Jeon, J., Cho, S., Tong, X., Lee, S.: Intrinsic image decomposition using structure-texture separation and surface normals. In: ECCV, pp. 218–233 (2014)

  20. Karsch, K., Sunkavalli, K., Hadap, S., Carr, N., Jin, H., Fonte, R., Sittig, M., Forsyth, D.: Automatic scene inference for 3d object compositing. ACM TOG 33(3), 1–15 (2014)

    Article  Google Scholar 

  21. Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. In: Fourth International Conference on 3D Vision, pp. 239–248 (2016)

  22. Lalonde, J.F., Efros, A.A., Narasimhan, S.G.: Detecting ground shadows in outdoor consumer photographs. Lect. Notes Comput. Sci. 6312, 322–335 (2010)

    Article  Google Scholar 

  23. Le, H., Samaras, D.: Shadow removal via shadow image decomposition. In: ICCV, pp. 8578–8587 (2019)

  24. Le, H., Vicente, Y., Tomas, F., Nguyen, V., Hoai, M., Samaras, D.: A+ D net: training a shadow detector with adversarial shadow attenuation. In: ECCV, pp. 662–678 (2018)

  25. Li, Z., Snavely, N.: Learning intrinsic image decomposition from watching the world. In: CVPR, pp. 9039–9048 (2018)

  26. Liu, F., Gleicher, M.: Texture-consistent shadow removal. In: ECCV, pp. 437–450 (2008)

  27. Liu, F., Shen, C., Lin, G.: Deep convolutional neural fields for depth estimation from a single image. In: CVPR, pp. 5162–5170 (2015)

  28. Meka, A., Richardt, C., Theobalt, C.: Live intrinsic video. ACM TOG 35(4), 109 (2016)

    Article  Google Scholar 

  29. Mikic, I., Cosman, P.C., Kogut, G.T., Trivedi, M.M.: Moving shadow and object detection in traffic scenes. In: IEEE TPAMI, vol. 1, pp. 321–324 (2000)

  30. Mohan, A., Tumblin, J., Choudhury, P.: Editing soft shadows in a digital photograph. IEEE Comput. Graph. Appl. 27(2), 23–31 (2007)

    Article  Google Scholar 

  31. Nguyen, V., Vicente, T.F.Y., Zhao, M., Hoai, M., Samaras, D.: Shadow detection with conditional generative adversarial networks. In: ICCV, pp. 4520–4528 (2017)

  32. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: ICCV, pp. 1693–1700 (2009)

  33. Qu, L., Tian, J., He, S., Tang, Y., Lau, R.W.H.: Deshadownet: a multi-context embedding deep network for shadow removal. In: CVPR, pp. 2308–2316 (2017)

  34. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2002)

    Google Scholar 

  35. Sidorov, O.: Conditional GANS for multi-illuminant color constancy: revolution or yet another approach? In: CVPR (2019)

  36. Vicente, T.F.Y., Hou, L., Yu, C.P., Hoai, M., Samaras, D.: Large-scale training of shadow detectors with noisily-annotated shadow examples. In: ECCV, pp. 816–832 (2016)

  37. Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: CVPR, pp. 1788–1797 (2018)

  38. Wei, J., Long, C., Zou, H., Xiao, C.: Shadow inpainting and removal using generative adversarial networks with slice convolutions. Comput. Graph. Forum 38, 381–392 (2019)

    Article  Google Scholar 

  39. Wu, T.P., Tang, C.K., Brown, M.S., Shum, H.Y.: Natural shadow matting. ACM TOG 26(2), 8 (2007)

    Article  Google Scholar 

  40. Xiao, C., Gan, J.: Fast image dehazing using guided joint bilateral filter. Vis. Comput. 28(6–8), 713–721 (2012)

    Article  Google Scholar 

  41. Xiao, C., She, R., Xiao, D., Ma, K.L.: Fast shadow removal using adaptive multi-scale illumination transfer. Comput. Graph. Forum 32(8), 207–218 (2013)

    Article  Google Scholar 

  42. Xiao, Y., Tsougenis, E., Tang, C.: Shadow removal from single rgb-d images. In: CVPR, pp. 3011–3018 (2014)

  43. Xu, D., Ricci, E., Ouyang, W., Wang, X., Sebe, N.: Multi-scale continuous crfs as sequential deep networks for monocular depth estimation, pp. 161–169 (2017)

  44. Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via relative total variation. ACM TOG 31(6), 1–10 (2012)

    Google Scholar 

  45. Zhang, L., Long, C., Zhang, X., Xiao, C.: Ris-gan: Explore residual and illumination with generative adversarial networks for shadow removal. In: AAAI (2020)

  46. Zhang, L., Yan, Q., Liu, Z., Zou, H., Xiao, C.: Illumination decomposition for photograph with multiple light sources. IEEE Trans. Image Process. 26(9), 4114–4127 (2017)

    Article  MathSciNet  Google Scholar 

  47. Zhang, L., Yan, Q., Zhu, Y., Zhang, X., Xiao, C.: Effective shadow removal via multi-scale image decomposition. Vis. Comput. 35(6–8), 1091–1104 (2019)

    Article  Google Scholar 

  48. Zhang, L., Zhang, Q., Xiao, C.: Shadow remover: image shadow removal based on illumination recovering optimization. IEEE Trans. Image Process. 24(11), 4623–4636 (2015)

    Article  MathSciNet  Google Scholar 

  49. Zheng, Q., Qiao, X., Cao, Y., Lau, R.W.: Distraction-aware shadow detection. In: CVPR, pp. 5167–5176 (2019)

Download references

Acknowledgements

Funding was provided by the Key Technological Innovation Projects of Hubei Province (Grant No. 2018AAA062), NSFC (Grant Nos. 61972298, 61672390, 61902286), National Key Research and Development Program of China (Grant No. 2017YFB1002600), China Postdoctoral Science Found (No. 2018M642933), and Wuhan University - Huawei GeoInformatics Innovation Lab.

Author information

Authors and Affiliations

  1. School of Computer, Wuhan University, Wuhan, 430072, China

    Xinyun Fan, Wenjun Wu, Gang Fu, Zipei Chen & Chunxia Xiao

  2. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ling Zhang

  3. JD.com American Technologies Corporation, Mountain View, CA, 94043, USA

    Qingan Yan

  4. JD Digits, Mountain View, CA, 94043, USA

    Chengjiang Long

Authors
  1. Xinyun Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gang Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zipei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chengjiang Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chunxia Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunxia Xiao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, X., Wu, W., Zhang, L. et al. Shading-aware shadow detection and removal from a single image. Vis Comput 36, 2175–2188 (2020). https://doi.org/10.1007/s00371-020-01916-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01916-3

Keywords

Search

Navigation