Skip to main content
SpringerLink
Log in

Grey is the new RGB: How good is GAN-based image colorization for image compression?

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

GAN-based image colorization techniques are capable of producing highly realistic color in real-time. Subjective assessment of these approaches has demonstrated that humans are unable to differentiate between a true RGB image and a colorized image. In this work, we evaluate the fidelity of such colorization and for the first time analyze the GAN-based image colorization scheme in the context of image compression. Our analysis shows that the palette (set of colors) recommended by the GAN-based framework is very limited even for highly realistic interactive colorization. We propose two novel methods of automatic palette generation that allows for the GAN-based framework to be useful for image compression. We demonstrate that provided true colors at a few pixel locations, GAN-based approach results in good spread of color to other image regions. Subjective analysis on a number of public datasets shows that the current system has low fidelity but performs better than JPEG at low data rate regimes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. BT RECOMMENDATION ITU-R (2002) Methodology for the subjective assessment of the quality of television pictures

  2. Cheng Z, Yang Q, Sheng B (2015) Deep colorization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 415–423

  3. Christopoulos C, Skodras A, Ebrahimi T (2000) The JPEG2000 still image coding system: an overview. IEEE Trans Consum Electron 46(4):1103–1127. https://doi.org/10.1109/30.920468

    Article  Google Scholar 

  4. Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. International journal of computer vision 59(2):167–181

    Article  Google Scholar 

  5. Girod B (1993) What’s wrong with mean-squared error?, Digital images and human vision, 207–220

  6. Gygli M, Grabner H, Riemenschneider H, Nater F, Van Gool L (2013) The interestingness of images. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1633–1640

  7. He Y, Xuan Y, Chen W, Fu X (2009) Subjective image quality assessment: A method based on signal detection theory. In: 2009 IEEE International Conference on Systems, Man and Cybernetics, pp 4915–4919

  8. Khosla A, Das Sarma A, Hamid R (2014) What makes an image popular?. In: Proceedings of the 23rd international conference on World wide web, ACM, pp 867–876

  9. Kikuchi H, Huttunen H, Hwang J, Yukawa M, Muramatsu S, Shin J (2012) Color-tone similarity on digital images. In: Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, IEEE, pp 1–4

  10. Leger A M, Omachi T, Wallace G K (1991) JPEG still picture compression algorithm. Opt Eng 30(7):947–955

    Article  Google Scholar 

  11. Mantiuk R K, Tomaszewska A, Mantiuk R (2012) Comparison of four subjective methods for image quality assessment. In: Computer graphics forum, vol 31, Wiley Online Library, pp 2478–2491

  12. Mohammadi P, Ebrahimi-Moghadam A, Shirani S (2014) Subjective and objective quality assessment of image: A survey. arXiv:1406.7799

  13. Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Image Process 21(8):3378–3389

    Article  MathSciNet  Google Scholar 

  14. Rippel O, Bourdev L (2017) Real-time adaptive image compression. arXiv:1705.05823

  15. Thakur N, Devi S (2011) A new method for color image quality assessment. International Journal of Computer Applications 15(2):10–17

    Article  Google Scholar 

  16. Wang Z, Bovik A C (2009) Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE signal processing magazine 26(1):98–117

    Article  Google Scholar 

  17. Wang Z, Bovik A C, Lu L (2002) Why is image quality assessment so difficult?. In: ICASSP, vol 4, pp 3313–3316

  18. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4):600–612

    Article  Google Scholar 

  19. Zhang R, Isola P, Efros AA (2016) Colorful image colorization. In: European Conference on Computer Vision, Springer, pp 649–666

  20. Zhang R, Zhu J-Y, Isola P, Geng X, Lin AS, Yu T, Efros AA (2017) Real-time user-guided image colorization with learned deep priors. arXiv:1705.02999

Download references

Acknowledgements

The authors thank the participants of the experiment for subjective image quality assessment and acknowledge the computational support provided by the Supercomputing Research and Education Centre (ScREC) at National University of Sciences and Technology (NUST), Islamabad, Pakistan.

Author information

Authors and Affiliations

  1. Research Centre for Modelling and Simulation (RCMS), Islamabad, Pakistan

    Aroosh Fatima & Shahzad Rasool

  2. National University of Sciences and Technology (NUST), Islamabad, Pakistan

    Aroosh Fatima, Wajahat Hussain & Shahzad Rasool

  3. School of Electrical Engineering and Computer Science (SEECS), Islamabad, Pakistan

    Wajahat Hussain

Authors
  1. Aroosh Fatima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wajahat Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahzad Rasool
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahzad Rasool.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fatima, A., Hussain, W. & Rasool, S. Grey is the new RGB: How good is GAN-based image colorization for image compression?. Multimed Tools Appl 80, 3775–3791 (2021). https://doi.org/10.1007/s11042-020-09861-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09861-y

Keywords

Search

Navigation