Skip to main content
SpringerLink
Log in

Robust HDR image quality assessment using combination of quality metrics

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We propose a framework to combine various quality metrics using a full reference approach for High Dynamic Range (HDR) Image quality assessment (IQA). We combine scores from metrics exclusively designed for different applications such as HDR, Standard Dynamic Range (SDR) and color difference measures, in a non-linear manner using machine learning (ML) approaches with weights determined during an offline training process. We explore various ML techniques and find that support vector machine regression and gradient boosting regression trees are effective. To improve performance and reduce complexity, we use the back-tracking based Sequential Floating Forward Selection technique during training to include a subset of metrics from a list of quality metrics in our model. We evaluate the performance on five publicly available calibrated HDR databases with different types of distortion (including different types of compression, Gaussian noise, gamut mismatch, chromatic distortions and so on) and demonstrate improved performance using our method as compared to several existing IQA metrics. We perform extensive statistical analysis to demonstrate significant improvement over existing approaches and show the generality and robustness of our approach using cross-database validation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We use the terms LDR and SDR interchangeably in this paper

References

  1. BT (2016) 2100: Image parameter values for high dynamic range television for use in production and international programme exchange. International Telecommunication Union

  2. Aydin T, Mantiuk R, Myszkowski K, Seidel HP (2008) Dynamic range independent image quality assessment. ACM Trans Graph 27(3):69:1–69:10

    Google Scholar 

  3. Azimi M, Banitalebi-Dehkordi A, Dong Y, Pourazad M, Nasiopoulos P (2014) Evaluating the performance of existing full-reference quality metrics on high dynamic range HDR video content. In: International conference on multimedia signal processing

  4. Basak D, Pal S, Patranabis D (2007) Support vector regression. In: Neural information processing letters and reviews, pp. 203–224

  5. Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  6. Breiman L (2001) Random forests. Machine Learning 45(1):5–32

    MATH  Google Scholar 

  7. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth and brooks, monterey CA

  8. Choudhury A, Daly S (2018) HDR image quality assessment using machine-learning based combination of quality metrics. In: 2018 IEEE global conference on signal and information processing (GlobalSIP), Anaheim, CA, USA, pp 91–95

  9. Choudhury A, Daly S (2019) Combining quality metrics for improved HDR image quality assessment. In: 2Nd IEEE conference on multimedia information processing and retrieval, MIPR 2019, san jose, CA, USA, March 28-30, 2019, pp. 179–184

  10. Choudhury A, Daly S (2019) Combining quality metrics using machine learning for improved and robust HDR image quality assessment. Electronic Imaging 2019 (10):307–1–307-7. https://doi.org/10.2352/ISSN.2470-1173.2019.10.IQSP-307

    Article  Google Scholar 

  11. Choudhury A, Pytlarz J, Daly S (2019) HDR And WCG image quality assessment using color difference metrics. In: SMPTE 2019 Annual technical conference and exhibition

  12. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  13. Duda RO, Hart PE, Stork DG (2000) Pattern classification 2Nd edition. Wiley-interscience, new york, NY USA

  14. Freitas PG, Akamine WYL, Farias MCQ (2018) No-reference image quality assessment using orthogonal color planes patterns. IEEE Transactions on Multimedia 20(12):3353–3360

    Google Scholar 

  15. Friedman JH (1997) On bias, variance, 0/1—loss, and the curse-of-dimensionality. Data Min Knowl Disc 1(1):55–77

    MathSciNet  Google Scholar 

  16. Friedman JH (2000) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232

    MathSciNet  MATH  Google Scholar 

  17. Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38 (4):367–378

    MathSciNet  MATH  Google Scholar 

  18. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning MIT Press . http://www.deeplearningbook.org

  19. Gu K, Zhai G, Yang X, Zhang W (2013) A new psychovisual paradigm for image quality assessment: from differentiating distortion types to discriminating quality conditions. SIViP 7(3):423–436

    Google Scholar 

  20. Gu K, Zhai G, Yang X, Zhang W (2015) Using free energy principle for blind image quality assessment. IEEE Transactions on Multimedia 17(1):50–63

    Google Scholar 

  21. Guan F, Jiang G, Song Y, Yu M, Peng Z, Chen F (2018) No-reference high-dynamic-range image quality assessment based on tensor decomposition and manifold learning. Applied Optics 57:839

    Google Scholar 

  22. Hanhart P, Bernardo M, Pereira M, Pinheiro AMG, Ebrahimi T (2015) Benchmarking of objective quality metrics for HDR image quality assessment. EURASIP Journal on Image and Video Processing 2015(1):39

    Google Scholar 

  23. Hanhart P, Rerabek M, Ebrahimi T (2016) Subjective and objective evaluation of hdr video coding technologies. In: Qo MEX, pp. 1–6

  24. ITU-R BT (2019) 2124-0: Objective metric for the assessment of the potential visibility of colour differences in television

  25. ITU-T (2012) Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models

  26. Jia S, Zhang Y, Agrafiotis D, Bull D (2017) Blind high dynamic range image quality assessment using deep learning. In: IEEE ICIP, pp. 765–769

  27. Korshunov P, Hanhart P, Richter T, Artusi A, Mantiuk R, Ebrahimi T (2015) Subjective quality assessment database of HDR images compressed with jpeg xt. In: QoMEX, pp 1–6

  28. Kuang J, Johnson GM, Fairchild MD (2007) icam06: A refined image appearance model for hdr image rendering. Journal of Visual Communication and Image Representation 18(5), 406–414. Special issue on High Dynamic Range Imaging

  29. Li Z, Aaron A, Katsavounidis I, Moorthy A, Manohara M. (2016) Toward a practical perceptual video quality metric

  30. Li Z, Norkin A, Aaron A (2016) VMAF - Video quality metric alternative to PSNR Joint Video Exploration Team JVET of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG, 11

  31. Lin CY, Jheng KR, Shih T (2018) Objective hdr image quality assessment. Multimedia Tools and Applications 78:1–21

    Google Scholar 

  32. Lin JY, Liu TJ, Wu ECH, Kuo CCJ (2014) A fusion-based video quality assessment fvqa index. In: Signal and information processing association annual summit and conference (APSIPA), pp 1–5

  33. Liu TJ, Lin W, Kuo CCJ (2013) Image quality assessment using multi-method fusion. IEEE Trans Image Process 22(5):1793–1807

    MathSciNet  MATH  Google Scholar 

  34. Luo MR, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula: CIEDE 2000. Color Research & Application 26(5):340–350

    Google Scholar 

  35. Ma L, Li S, Zhang F, Ngan KN (2011) Reduced-reference image quality assessment using reorganized dct-based image representation. IEEE Transactions on Multimedia 13(4):824–829

    Google Scholar 

  36. Mai Z, Mansour H, Mantiuk R, Nasiopoulos P, Ward R, Heidrich W (2011) Optimizing a tone curve for backward-compatible high dynamic range image and video compression. IEEE Trans Image Process 20(6):1558–1571

    MathSciNet  MATH  Google Scholar 

  37. Mantiuk R, Kim KJ, Rempel AG, Heidrich W (2011) HDR-VDP,-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans Graph 30(4):40:1–40:14

    Google Scholar 

  38. Mantiuk R, Myszkowski K, Seidel HP (2006) A perceptual framework for contrast processing of high dynamic range images. ACM Trans. Appl Percept 3 (3):286–308

    Google Scholar 

  39. Miller S, Nezamabadi M, Daly S (2012) Perceptual signal coding for more efficient usage of bit codes The 2012 annual technical conference exhibition, pp 1–9

  40. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    MathSciNet  MATH  Google Scholar 

  41. Moorthy A, Bovik A (2011) Visual quality assessment algorithms: What does the future hold Multimedia Tools and Applications 51:675–696

    Google Scholar 

  42. Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364

    MathSciNet  MATH  Google Scholar 

  43. Nafchi HZ, Shahkolaei A, Hedjam R, Cheriet M (2016) Mean deviation similarity index: Efficient and reliable full-reference image quality evaluator. IEEE Access 4:5579–5590

    Google Scholar 

  44. Nafchi HZ, Shahkolaei A, Moghaddam RF, Cheriet M (2015) FSITM: A feature similarity index for tone-mapped images. IEEE Signal Processing Letters 22 (8):1026–1029

    Google Scholar 

  45. Narwaria M, Mantiuk R, Silva MPD, Callet PL (2015) HDR-VDP-2.2:A calibrated method for objective quality prediction of high-dynamic range and standard images. J Electron Imaging 24(24–24):3

    Google Scholar 

  46. Narwaria M, Perreira Da Silva M, Le Callet P, Pepion R (2013) Tone mapping-based high-dynamic-range image compression: Study of optimization criterion and perceptual quality. Opt Eng 102008:52

    Google Scholar 

  47. Narwaria M, Silva MPD, Callet PL (2015) HDR - VQM: An objective quality measure for high dynamic range video. Signal Processing:, Image Communication 35:46–60

    Google Scholar 

  48. Pieri E, Pytlarz J (2017) Hitting the mark - a new color difference metric for hdr and wcg imagery. In: SMPTE 2017 Annual technical conference and exhibition, pp. 1–13

  49. Pudil P, Novovičová J., Kittler J (1994) Floating search methods in feature selection. Pattern Recogn Lett 15(11):1119–1125

    Google Scholar 

  50. Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Image Process 21(8):3378–3389

    MathSciNet  MATH  Google Scholar 

  51. Reinhard E, Stark M, Shirley P, Ferwerda J (2002) Photographic tone reproduction for digital images. ACM Trans Graph 21(3):267–276

    Google Scholar 

  52. Rousselot M, Auffret E, Ducloux X, Le Meur O, Cozot R (2018) Impacts of viewing conditions on hdr-vdp2. In: EUSIPCO, pp. 1442–1446

  53. Rousselot M, Le Meur O, Cozot R, Ducloux X (2019) Quality assessment of hdr/wcg images using hdr uniform color spaces Journal of Imaging 5(1)

  54. Rumelhart DE, Hinton GE, Williams RJ (1986) In: Parallel Distributed processing: Explorations in the Microstructure of Cognition, Vol. 1, chap Learning Internal Representations by Error Propagation, pp. 318–362. MIT Press, Cambridge, MA USA

  55. Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of jpeg compressed images. In: International conference on image processing, pp. 477–480

  56. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE TIP 15(2):430–444

    Google Scholar 

  57. Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing 14(12):2117–2128

    Google Scholar 

  58. Aydin TR, Mantiuk HS (2008) Extending quality metrics to full luminance range images. pp. 6806–6806 – 10

  59. Valenzise G, Simone FD, Lauga P, Dufaux F (2014) Performance evaluation of objective quality metrics for hdr image compression. In: SPIE Optical engineering + applications, international society for optics and photonics

  60. Ververidis D, Kotropoulos C (2005) Sequential forward feature selection with low computational cost. In: 13Th european signal processing conference, pp. 1–4

  61. VQEG (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment

  62. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Processing Letters 9(3):81–84

    Google Scholar 

  63. Wang Z, Simoncelli EP (2005) Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. In: SPIE Human vision and electronic imaging, vol. 5666, pp. 149–159

  64. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: 37Th asilomar conference on signals, systems and computers, vol. 2, pp. 1398–1402. IEEE

  65. Whitney AW (1971) A direct method of nonparametric measurement selection. IEEE Transactions on Computers C-20(9):1100–1103

  66. Wu Q, Li H, Wang Z, Meng F, Luo B, Li W, Ngan KN (2017) Blind image quality assessment based on rank-order regularized regression. IEEE Transactions on Multimedia 19(11):2490–2504

    Google Scholar 

  67. Xue W, Zhang L, Mou X, Bovik AC (2014) Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684–695

    MathSciNet  MATH  Google Scholar 

  68. Yu H, He F, Yiteng P (2019) A scalable region-based level set method using adaptive bilateral filter for noisy image segmentation. Multimedia Tools and Applications 79:5743–5765

    Google Scholar 

  69. Zerman E, Valenzise G, Dufaux F (2017) An extensive performance evaluation of full-reference HDR image quality metrics. Quality and User Experience 2(1):5

    Google Scholar 

  70. Zhang J, He F, Chen Y (2019) A new haze removal approach for sky/river alike scenes based on external and internal clues. Multimedia Tools and Applications 79:2085–2107

    Google Scholar 

  71. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: A feature similarity index for image quality assessment. IEEE TIP 20(8):2378–2386

    MathSciNet  MATH  Google Scholar 

  72. Ziaei Nafchi H, Cheriet M (2018) Efficient no-reference quality assessment and classification model for contrast distorted images. IEEE Trans Broadcast 64(2):518–523

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dolby Laboratories Inc., Sunnyvale, CA, 94085, USA

    Anustup Choudhury

Authors
  1. Anustup Choudhury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anustup Choudhury.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choudhury, A. Robust HDR image quality assessment using combination of quality metrics. Multimed Tools Appl 79, 22843–22867 (2020). https://doi.org/10.1007/s11042-020-08985-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08985-5

Keywords

Search

Navigation