Abstract
A structure-adaptive vector filter for removal of impulse noise from color images is presented. The proposed method is based on local orientation estimation. A color image is represented in quaternion form, and then, quaternion Fourier transform is used to compute the orientation of the pattern in a local neighborhood. Since the computation in quaternion frequency domain is extremely time-consuming, we prove a theorem that the integral of the product of frequency variables and the magnitude of quaternion frequency signals can be computed directly in spatial domain, which results that the color orientation detection problem can be solved in spatial domain. Based on the local orientation and orientation strength, the size, shape, and orientation of the support window of vector median filter (VMF) are adaptively determined, leading to an effective structure-adaptive VMF. Unlike the classical VMF restricting the output to the existing color samples, this paper computes the output of VMF over the entire 3D data space, which boosts the filtering performance effectively. To further improve denoising effect, a deep convolutional neural network is employed to detect impulse noise in color images and integrated into the proposed denoising framework. The experimental results exhibit the effectiveness of the proposed denoiser by showing significant performance improvements both in noise suppression and in detail preservation, compared to other color image denoising methods.
Similar content being viewed by others
References
Shen, Y., Han, B., Braverman, E.: Removal of mixed gaussian and impulse noise using directional tensor product complex tight framelets. J. Math. Imaging Vis. 54(1), 64–77 (2016)
Plataniotis, K.N., Venetsanopoulos, A.N.: Color Image Processing and Applications. Springer, Berlin (2000)
Villar, S.A., Torcida, S., Acosta, G.G.: Median filtering: a new insight. J. Math. Imaging Vis. 58(1), 130–146 (2017)
Astola, J., Haavisto, P., Neuvo, Y.: Vector median filters. Proc. IEEE 78(4), 678–689 (1990)
Trahanias, P.E., Venetsanopoulos, A.N.: Vector directional filters: a new class of multichannel image processing filters. IEEE Trans. Image Process. 2(4), 528–534 (1993)
Karakos, D.G., Trahanias, P.E.: Generalized multichannel image-filtering structures. IEEE Trans. Image Process. 6(7), 1038–1045 (1997)
Lukac, R.: Adaptive vector median filtering. Pattern Recogn. Lett. 24(12), 1889–1899 (2003)
Smolka, B., Chydzinski, A.: Fast detection and impulsive noise removal in color images. Real Time Images 11(5/6), 389–402 (2005)
Malinski, L., Smolka, B.: Fast averaging peer group filter for the impulsive noise removal in color images. J. Real Time Image Process. 11(3), 427–444 (2016)
Celebi, M.E., Aslandogan, Y.A.: Robust switching vector median filter for impulsive noise removal. J. Electron. Imaging 17(4), 043006 (2008)
Shen, Y., Barner, K.E.: Fast adaptive optimization of weighted vector median filters. IEEE Trans. Signal Process. 54(7), 2497–2510 (2006)
Lukac, R., Smolka, B., Plataniotis, K.N., Venetsanopoulos, A.N.: Selection weighted vector directional filters. Comput. Vis. Image Underst. 94(1/3), 140–167 (2004)
Plataniotis, K.N., Androutsos, D., Venetsanopoulos, A.N.: Color image processing using adaptive vector directional filters. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process 45(10), 1414–1419 (1998)
Camarena, J.-G., Gregori, V., Morillas, S., Sapena, A.: A simple fuzzy method to remove mixed gaussian-impulsive noise from color images. IEEE Trans. Fuzzy Syst. 21(5), 971–978 (2013)
Ananthi, V.P., Balasubramaniam, P.: A new image denoising method using interval-valued intuitionistic fuzzy sets for the removal of impulse noise. Signal Process. 121, 81–93 (2016)
Portilla, J., Strela, V., Wainwright, M.J., Simoncelli, E.P.: Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans. Image Process. 12(11), 1338–1351 (2003)
Fathi, A., Naghsh, A.R.: Efficient image denoising method based on a new adaptive wavelet packet thresholding function. IEEE Trans. Image Process. 21(9), 3981–3990 (2012)
Kim, S.: PDE-based image restoration: a hybrid model and color image denoising. IEEE Trans. Image Process. 15(5), 1163–1170 (2006)
Tiirola, J.: Image denoising using directional adaptive variable exponents model. J. Math. Imaging Vis. 57(1), 56–74 (2017)
Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)
Hajiaboli, M.R.: An anisotropic fourth-order diffusion filter for image noise removal. Int. J. Comput. Vis. 92(2), 177–191 (2011)
Weickert, J.: Coherence-enhancing diffusion of colour images. Image Vis. Comput. 17(3–4), 201–212 (1999)
Estellers, V., Soatto, S., Bresson, X.: Adaptive regularization with the structure tensor. IEEE Trans. Image Process. 24(6), 1777–1790 (2015)
Felsberg, M.: Autocorrelation-driven diffusion filtering. IEEE Trans. Image Process. 20(7), 1797–1806 (2011)
Liu, F., Liu, J.: Anisotropic diffusion for image denoising based on diffusion tensors. J. Vis. Commun. Image Represent. 23(3), 516–521 (2012)
Jin, L., Zhu, Z., Xu, X., Li, X.: Two-stage quaternion switching vector filter for color impulse noise removal. Signal Process. 128, 171–185 (2016)
Gai, S., Yang, G., Wan, M., Wang, L.: Denoising color images by reduced quaternion matrix singular value decomposition. Multidimens. Syst. Signal Process. 26(1), 307–320 (2015)
Jin, L., Liu, H., Xu, X., Song, E.: Quaternion-based impulse noise removal from color video sequences. IEEE Trans. Circuits Syst. Video Technol. 23(5), 741–755 (2013)
Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)
Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)
Chen, C.L.P., Liu, L., Chen, L., Tang, Y.Y., Zhou, Y.: Weighted couple sparse representation with classified regularization for impulse noise removal. IEEE Trans. Image Process. 24(11), 4014–4026 (2015)
Jiang, J., Zhang, L., Yang, J.: Mixed noise removal by weighted encoding with sparse nonlocal regularization. IEEE Trans. Image Process. 23(6), 2651–2662 (2014)
Ji, H., Huang, S., Shen, Z., Xu, Y.: Robust video restoration by joint sparse and low rank matrix approximation. SIAM J. Imaging Sci. 4(4), 1122–1142 (2011)
Wang, R., Pakleppa, M., Trucco, E.: Low-rank prior in single patches for nonpointwise impulse noise removal. IEEE Trans. Image Process. 24(5), 1485–1496 (2015)
Jin, K.H., Ye, J.C.: Sparse and low-rank decomposition of a Hankel structured matrix for impulse noise removal. IEEE Trans. Image Process. 27(3), 1448–1461 (2018)
Roy, A., Laskar, R.H.: Multiclass SVM based adaptive filter for removal of high density impulse noise from color images. Appl. Soft Comput. 46, 816–826 (2016)
Lin, T.-C.: Decision-based filter based on SVM and evidence theory for image noise removal. Neural Comput. Appl. 21(4), 695–703 (2012)
Turkmen, I.: The ANN based detector to remove random-valued impulse noise in images. J. Vis. Commun. Image Represent. 34, 28–36 (2016)
Nair, M.S., Shankar, V.: Predictive-based adaptive switching median filter for impulse noise removal using neural network-based noise detector. Signal Image Video Process. 7(6), 1041–1070 (2013)
Kaliraj, G., Baskar, S.: An efficient approach for the removal of impulse noise from the corrupted image using neural network based impulse detector. Image Vis. Comput. 28, 458–466 (2010)
Liang, S.F., Lu, S.M., Chang, J.Y., Lin, C.T.: A novel two-stage impulse noise removal technique based on neural networks and fuzzy decision. IEEE Trans. Fuzzy Syst. 16(4), 863–873 (2008)
Forstner, M.A., Gulch, E.: A fast operator for detection and precise location of distinct points, corners and centers of circular features. In: Proceedings of the ISPRS Intercommission Conference on Fast Processing of Phonogrammic Data, pp. 281–305 (1987)
Hamilton, W.R.: Elements of Quaternions. Longmans Green, London (1866)
Xu, D., Mandic, D.P.: The theory of quaternion matrix derivatives. IEEE Trans. Signal Process. 63(6), 1543–1556 (2015)
Subakan, O.N., Vemuri, B.C.: A quaternion framework for color image smoothing and segmentation. Int. J. Comput. Vis. 91(3), 233–250 (2011)
Chen, B., Shu, H., Coatrieux, G., Chen, G., Sun, X., Coatrieux, J.L.: Color image analysis by quaternion-type moments. J. Math. Imaging. Vis. 51(1), 124–144 (2014)
Yang, G.Z., Burger, P., Firmin, D.N., Underwood, S.R.: Structure adaptive anisotropic image filtering. Image Vis. Comput. 14(2), 135–145 (1996)
Greenberg, S., Kogan, D.: Improved structure-adaptive anisotropic filter. Pattern Recognit. Lett. 27(1), 59–65 (2006)
Jin, L., Jin, M., Xu, X., Song, E.: Structure-adaptive vector median filter for impulse noise removal in color images. In: IEEE International Conference on Image Processing (ICIP), pp. 690–694 (2017)
Ell, T.A., Sangwine, S.J.: Hypercomplex Fourier transforms of color images. IEEE Trans. Image Process. 16(1), 22–35 (2007)
Said, S., Le Bihan, N., Sangwine, S.J.: Fast complexified quaternion Fourier transform. IEEE Trans. Signal Process. 56(4), 1522–1531 (2008)
Pei, S.-C., Ding, J.-J., Chang, J.-H.: Efficient implementation of quaternion Fourier transform, convolution, and correlation by 2-D complex FFT. IEEE Trans. Signal Process. 49(11), 2783–2797 (2001)
Jin, L., Liu, H., Xu, X., Song, E.: Improved direction estimation for Di Zenzo’s multichannel image gradient operator. Pattern Recognit. 45(12), 4300–4311 (2012)
Di Zenzo, S.: A note on the gradient of a multi-image. Comput. Vis. Graph. Image Process. 33(1), 116–125 (1986)
Shui, P.-L.: Image denoising algorithm via doubly local Wiener filtering with directional windows in wavelet domain. IEEE Signal Process. Lett. 12(10), 681–684 (2005)
Xiao, S., Hu, J., Wang, Y.: Non-local means with steer search window and adaptive parameter. In: IEEE International Conference on Image Processing, pp. 76–80 (2018)
Vardi, Y., Zhang, C.-H.: A modified Weiszfeld algorithm for the Fermat-Weber location problem. Mathematical Programming A 90, 559–566 (2001)
Spence, C., Fancourt, C.: An iterative method for vector median filtering. In: IEEE IEEE International Conference on Image Processing (ICIP), pp. 265–268 (2007)
Zhang, W., Jin, L., Song, E., Xu, X.: Removal of impulse noise in color images based on convolution neural network. Appl. Soft Comput. J. 82, 10558 (2019)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)
Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)
Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of International Conference on Learning Representations, (2015)
Acknowledgements
The authors would like to thank Dr. Kyong Hwan Jin for providing the program for the algorithm ALOHA [35]. This work was supported by the National Natural Science Foundation of China under Grant No. 61370181.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Wenhua Zhang—deceased.
Appendices
Appendix 1: Proofs of Lemmas 2 and 3
Note that in the following proofs, j denotes the imaginary unit of a complex number.
Proof of Lemma 2
To prove Lemma 3, the following conclusion is needed:
Lemma 4
Let\( f(x,y) \)denote a complex function defined in\( [ - \infty ,\; + \infty ] \times [ - \infty ,\; + \infty ] \)and\( F_{f} (u,v) \)be its CFT. Then, the following conclusions hold:
- (1)
\( F_{{f_{x}^{{\prime }} }} (u,v) = ju \cdot F_{f} (u,v) \)when\( \mathop {\lim }\limits_{\left| x \right| \to \infty } \left| {f(x,y)} \right| = 0. \)
- (2)
\( F_{{f_{y}^{\prime } }} (u,v) = jv \cdot F_{f} (u,v) \)when\( \mathop {\lim }\limits_{\left| y \right| \to \infty } \left| {f(x,y)} \right| = 0. \)
The proof of Lemma 4 can be found in general textbooks and documents. We no longer prove it again and only give the proof of Lemma 3.
Proof of Lemma 3
Similarly, by letting \( \int {\int {uv \cdot F(u,v)G^{*} (u,v){\text{d}}u{\text{d}}v} } = \int {\int { \, \left( {jv \cdot F(u,v)} \right) \cdot \left( { - ju \cdot G^{*} (u,v)} \right)} } {\text{d}}u{\text{d}}v, \) we can obtain
In the same way, we have
Similarly, we have
This finishes the proof of Lemma 3.
Appendix 2: Performance Evaluation on Individual Images
This appendix gives the PSNR, MAE, NCD, FSIMc scores on each of the images shown in Fig. 4.
Rights and permissions
About this article
Cite this article
Jin, L., Song, E. & Zhang, W. Denoising Color Images Based on Local Orientation Estimation and CNN Classifier. J Math Imaging Vis 62, 505–531 (2020). https://doi.org/10.1007/s10851-019-00942-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-019-00942-8