Skip to main content
SpringerLink
Log in

Multi-scale joint network based on Retinex theory for low-light enhancement

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Due to the limitations of devices, images taken in low-light environments are of low contrast and high noise without any manual intervention. Such images will affect the visual experience and hinder further visual processing tasks, such as target detection and target tracking. To alleviate this issue, we propose a multi-scale joint low-light enhancement network based on the Retinex theory. The network consists of a decomposition part and an enhancement part. As a joint network, the decomposition and enhancement parts are mutually constrained, and the parameters are updated at the same time so that the image processing results are more excellent in detail. Our algorithm avoids the separation and recombination of decomposition and enhancement. Therefore, less information is lost in the processing of low-light images, and the enhancement result of the proposed algorithm is very close to the ground truth. In addition, in the enhancement part, we adopt a multi-scale network to fully extract image features. The multi-scale network maintains a balance between the global and local luminance of the illumination image. Retinex theory can effectively solve the problem of noise amplification and color distortion. At the same time, we have added color loss to solve the problem of color distortion, so that the enhancement result is closer to the normal-light image in color. The enhancement results are intuitively excellent, and the peak signal-to-noise ratio and structural similarity index results also reflect the reliability of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Maini, R., Aggarwal, H.: A comprehensive review of image enhancement techniques (2010). arXiv preprint arXiv:1003.4053

  2. Pisano, E.D., Zong, S., Hemminger, B.M., DeLuca, M., Johnston, R.E., Muller, K., Braeuning, M.P., Pizer, S.M.: Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. J. Digit. Imaging 11(4), 193 (1998)

    Article  Google Scholar 

  3. Dong, X., Wang, G., Pang, Y., Li, W., Wen, J., Meng, W., Lu, Y.: Fast efficient algorithm for enhancement of low lighting video. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 1–6 (2011)

  4. Land, E.H., McCann, J.J.: Lightness and Retinex theory. Josa 61(1), 1–11 (1971)

    Article  Google Scholar 

  5. Jobson, D.J., Rahman, Z.U., Woodell, G.A.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)

    Article  Google Scholar 

  6. Jobson, D.J., Rahman, Z.U., Woodell, G.A.: A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 6(7), 965–976 (1997)

    Article  Google Scholar 

  7. Fu, X., Zeng, D., Huang, Y., Zhang, X.P., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2782–2790 (2016)

  8. Guo, X., Li, Y., Ling, H.: Lime: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)

    Article  MathSciNet  Google Scholar 

  9. Li, X., Song, D., Dong, Y.: Hierarchical feature fusion network for salient object detection. IEEE Trans. Image Process. 29, 9165–9175 (2020)

    Article  Google Scholar 

  10. Liu, L., Cao, J.: End-to-end learning interpolation for object tracking in low frame-rate video. IET Image Process. 14(6), 1066–1072 (2020)

    Article  Google Scholar 

  11. Lore, K.G., Akintayo, A., Sarkar, S.: Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 61, 650–662 (2017)

    Article  Google Scholar 

  12. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: Wespe: weakly supervised photo enhancer for digital cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 691–700 (2018)

  13. Ren, W., Liu, S., Ma, L., Xu, Q., Xu, X., Cao, X., Du, J., Yang, M.H.: Low-light image enhancement via a deep hybrid network. IEEE Trans. Image Process. 28(9), 4364–4375 (2019)

    Article  MathSciNet  Google Scholar 

  14. Ibrahim, H., Kong, N.S.P.: Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 53(4), 1752–1758 (2007)

    Article  Google Scholar 

  15. Wang, C., Ye, Z.: Brightness preserving histogram equalization with maximum entropy: a variational perspective. IEEE Trans. Consum. Electron. 51(4), 1326–1334 (2005)

    Article  Google Scholar 

  16. Chen, S.D., Ramli, A.R.: Minimum mean brightness error bi-histogram equalization in contrast enhancement. IEEE Trans. Consum. Electron. 49(4), 1310–1319 (2003)

    Article  Google Scholar 

  17. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. J. VLSI Signal Process. Syst. Signal Image Video Technol. 38(1), 35–44 (2004)

    Article  Google Scholar 

  18. Stark, J.A.: Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Trans. Image Process. 9(5), 889–896 (2000)

    Article  Google Scholar 

  19. Li, L., Wang, R., Wang, W., Gao, W.: A low-light image enhancement method for both denoising and contrast enlarging. In: Proceedings of International Conference on Image Processing, pp. 3730–3734 (2015)

  20. Shen, L., Yue, Z., Feng, F., Chen, Q., Liu, S., Ma, J.: Msr-net: low-light image enhancement using deep convolutional network (2017). arXiv preprint arXiv:1711.02488

  21. Lee, C.H., Shih, J.L., Lien, C.C., Han, C.C.: Adaptive multiscale retinex for image contrast enhancement. In: Proceedings of International Conference on Signal-Image Technology and Internet-Based Systems, pp. 43–50 (2013)

  22. Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6849–6857 (2019)

  23. Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300 (2018)

  24. Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.H., Liao, Q.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimed. 21(12), 3106–3121 (2019)

    Article  Google Scholar 

  25. Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. Image Process. 27(4), 2049–2062 (2018)

    Article  MathSciNet  Google Scholar 

  26. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the ACM International Conference on Multimedia, pp. 1632–1640 (2019)

  27. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement (2018). arXiv preprint arXiv:1808.04560

  28. Gharbi, M., Chen, J., Barron, J.T., Hasinoff, S.W., Durand, F.: Deep bilateral learning for real-time image enhancement. ACM Trans. Graph. 36(4), 1–12 (2017)

    Article  Google Scholar 

  29. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: Dslr-quality photos on mobile devices with deep convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3277–3285 (2017)

  30. Li, M., Liu, J., Yang, W., Sun, X., Guo, Z.: Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828–2841 (2018)

    Article  MathSciNet  Google Scholar 

  31. Xiaochu, W., Guijin, T., Xiaohua, L., Ziguan, C., Suhuai, L.: Low-light color image enhancement based on nsst. J. China Univ. Posts Telecommun. 5, 6 (2019)

    Google Scholar 

  32. Ying, Z., Li, G., Ren, Y., Wang, R., Wang, W.: A new low-light image enhancement algorithm using camera response model. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 3015–3022 (2017)

  33. Ying, Z., Li, G., Ren, Y., Wang, R., Wang, W.: A new image contrast enhancement algorithm using exposure fusion framework. In: Proceedings of International Conference on Computer Analysis of Images and Patterns, pp. 36–46 (2017)

Download references

Author information

Authors and Affiliations

  1. Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, 710119, China

    Xijuan Song, Jijiang Huang, Jianzhong Cao & Dawei Song

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Xijuan Song & Dawei Song

Authors
  1. Xijuan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jijiang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianzhong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dawei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jijiang Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, X., Huang, J., Cao, J. et al. Multi-scale joint network based on Retinex theory for low-light enhancement. SIViP 15, 1257–1264 (2021). https://doi.org/10.1007/s11760-021-01856-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01856-y

Keywords

Search

Navigation