Skip to main content
SpringerLink
Log in

Color and direction-invariant nonlocal self-similarity prior and its application to color image denoising

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Nonlocal self-similarity (NSS) is one of the most commonly used priors in computer vision and image processing. It aims to make use of the fact that a natural image often possesses many repetitive local patterns, and thus a local image patch always has many similar patches across the image. Through compensatively integrating these similar image patches, their insightful patterns hiding under corrupted noises can be intrinsically extracted. However, for using this prior knowledge, current methods search the similar patches by using simple block matching strategy with Euclidean distance, which largely ignores those patches containing similar local patterns but with different texture-directions and colors. To more sufficiently explore similar patches over an image, in this paper, we propose two new representations for image patches, which facilitate an easy NSS prior for measuring direction-invariant and color-invariant nonlocal self-similarity possessed by image patches. Specifically, based on this prior term, we formulate the color image denoising problem as a concise Bayesian posterior estimation framework, and design an efficient expectation-maximization (EM) algorithm to solve it. A series of experiments implemented on simulated and real noisy color images demonstrate the superiority of the proposed method as compared with the state-of-the-arts both visually and quantitatively, verifying the potential usefulness of this new NSS prior.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Buades A, Coll B, Morel J M. A non-local algorithm for image denoising. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005

  2. Dabov K, Foi A, Katkovnik V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process, 2007, 16: 2080–2095

    Article  MathSciNet  Google Scholar 

  3. Gu S H, Xie Q, Meng D Y, et al. Weighted nuclear norm minimization and its applications to low level vision. Int J Comput Vis, 2017, 121: 183–208

    Article  Google Scholar 

  4. Ji H, Liu C C, Shen Z W, et al. Robust video denoising using low rank matrix completion. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010

  5. Yu G S, Sapiro G, Mallat S. Solving inverse problems with piecewise linear estimators: from Gaussian mixture models to structured sparsity. IEEE Trans Image Process, 2012, 21: 2481–2499

    Article  MathSciNet  Google Scholar 

  6. Mairal J, Bach F, Ponce J, et al. Non-local sparse models for image restoration. In: Proceedings of the 12th International Conference on Computer Vision, 2009. 2272–2279

  7. Dong W S, Zhang L, Shi G M, et al. Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process, 2013, 22: 1620–1630

    Article  MathSciNet  Google Scholar 

  8. Dabov K, Foi A, Katkovnik V, et al. Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space. In: Proceedings of IEEE International Conference on Image Processing, 2007

  9. Xu J, Zhang L, Zhang D, et al. Multi-channel weighted nuclear norm minimization for real color image denoising. In: Proceedings of IEEE International Conference on Computer Vision, 2017. 1096–1104

  10. McLachlan G J, Basford K E. Mixture Models: Inference and Applications to Clustering. New York: Marcel Dekker, 1988

    MATH  Google Scholar 

  11. Rudin L I, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Phys D-Nonlinear Phenom, 1992, 60: 259–268

    Article  MathSciNet  Google Scholar 

  12. Zuo W M, Zhang L, Song C W, et al. Texture enhanced image denoising via gradient histogram preservation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2013. 1203–1210

  13. Do M N, Vetterli M. Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance. IEEE Trans Image Process, 2002, 11: 146–158

    Article  MathSciNet  Google Scholar 

  14. Portilla J, Strela V, Wainwright M J, et al. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Process, 2003, 12: 1338–1351

    Article  MathSciNet  Google Scholar 

  15. Peyre G, Bougleux S, Cohen L. Non-local regularization of inverse problems. In: Proceedings of European Conference on Computer Vision, 2008. 57–68

  16. Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In: Proceedings of International Conference on Computer Vision, 2011. 479–486

  17. Dong W S, Shi G M, Li X. Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process, 2013, 22: 700–711

    Article  MathSciNet  Google Scholar 

  18. Rajwade A, Rangarajan A, Banerjee A. Image denoising using the higher order singular value decomposition. IEEE Trans Pattern Anal Mach Intell, 2013, 35: 849–862

    Article  Google Scholar 

  19. Zhang K, Zuo W M, Chen Y J, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process, 2017, 26: 3142–3155

    Article  MathSciNet  Google Scholar 

  20. Chen Y J, Yu W, Pock T. On learning optimized reaction diffusion processes for effective image restoration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015. 5261–5269

  21. Chen F, Zhang L, Yu H M. External patch prior guided internal clustering for image denoising. In: Proceedings of IEEE International Conference on Computer Vision, 2015. 603–611

  22. Elad M, Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process, 2006, 15: 3736–3745

    Article  MathSciNet  Google Scholar 

  23. Pang Y W, Xie J, Li X L. Visual haze removal by a unified generative adversarial network. IEEE Trans Circ Syst Video Technol, 2019, 29: 3211–3221

    Article  Google Scholar 

  24. Pang Y W, Li Y Z, Shen J B, et al. Towards bridging semantic gap to improve semantic segmentation. In: Proceedings of IEEE International Conference on Computer Vision, 2019. 4230–4239

  25. Zhou Z H. Abductive learning: towards bridging machine learning and logical reasoning. Sci China Inf Sci, 2019, 62: 076101

    Article  MathSciNet  Google Scholar 

  26. Kervrann C, Boulanger J, Coupe P. Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal. In: Proceedings of International Conference on Scale Space and Variational Methods in Computer Vision, 2007. 520–532

  27. Zhu F Y, Chen G Y, Heng P A. From noise modeling to blind image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016. 420–429

  28. Lebrun M, Colom M, Morel J M. Multiscale image blind denoising. IEEE Trans Image Process, 2015, 24: 3149–3161

    Article  MathSciNet  Google Scholar 

  29. Lebrun M, Colom M, Morel J M. The noise clinic: a blind image denoising algorithm. Image Process Line, 2015, 5: 1–54

    Article  Google Scholar 

  30. Nam S, Hwang Y, Matsushita Y, et al. A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016. 1683–1691

  31. Elmoataz A, Lezoray O, Bougleux S. Nonlocal discrete regularization on weighted graphs: a framework for image and manifold processing. IEEE Trans Image Process, 2008, 17: 1047–1060

    Article  MathSciNet  Google Scholar 

  32. Gilboa G, Osher S. Nonlocal operators with applications to image processing. Multiscale Model Simul, 2009, 7: 1005–1028

    Article  MathSciNet  Google Scholar 

  33. International Telecommunication Union. Parameter values for the HDTV standards for production and international programme exchange. BT.709-5. https://www.itu.int/rec/R-REC-BT.709/en

  34. Roth S, Black M J. Fields of experts. Int J Comput Vis, 2009, 82: 205–229

    Article  Google Scholar 

  35. Cao X Y, Chen Y, Zhao Q, et al. Low-rank matrix factorization under general mixture noise distributions. In: Proceedings of IEEE International Conference on Computer Vision, 2015. 1493–1501

  36. Yong H W, Meng D Y, Zuo W M, et al. Robust online matrix factorization for dynamic background subtraction. In: Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

  37. Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc-Ser B, 1977, 39: 1–22

    MathSciNet  MATH  Google Scholar 

  38. Boyd S, Parikh N, Chu E, et al. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Boston: Now Foundations and Trends, 2011

    MATH  Google Scholar 

  39. Krishnan D, Fergus R. Fast image deconvolution using hyper-Laplacian priors. In: Proceedings of International Conference on Neural Information Processing Systems, 2009. 1033–1041

  40. Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13: 600–612

    Article  Google Scholar 

  41. Zhang L, Zhang L, Mou X Q, et al. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process, 2011, 20: 2378–2386

    Article  MathSciNet  Google Scholar 

  42. Wang Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment. In: Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers, 2003. 1398–1402

  43. Chen G Y, Zhu F Y, Heng P A. An efficient statistical method for image noise level estimation. In: Proceedings of IEEE International Conference on Computer Vision, 2015. 477–485

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xi’an Jiaotong University, Shaanxi, 710049, China

    Qi Xie, Qian Zhao, Zongben Xu & Deyu Meng

  2. Macau Institute of Systems Engineering, Macau University of Science and Technology, Macau, 999078, China

    Deyu Meng

Authors
  1. Qi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zongben Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Deyu Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deyu Meng.

Supplementary File

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Q., Zhao, Q., Xu, Z. et al. Color and direction-invariant nonlocal self-similarity prior and its application to color image denoising. Sci. China Inf. Sci. 63, 222101 (2020). https://doi.org/10.1007/s11432-020-2880-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-2880-3

Keywords

Search

Navigation