Blind motion deconvolution for binary images
Introduction
An image is binary if it has only two different values for each pixel. Binary images are prevalent in digital systems and have a wide range of applications including texts, fingerprint recognition, handwritten signatures, stellar astronomy, barcodes, and vehicle license plates [1], [2], [3]. Due to environmental effects and imperfections in the imaging system, the recorded binary images are usually degraded by blur and additive noise. For example, blurring often arises from the optical aberrations, relative movement between scene and camera, or object motion during exposure time, and noise is caused by malfunctioning pixels in camera sensors, faulty memory locations in hardware, or transmission in a noisy channel [4]. In the digital image processing, the degradation of the original image can be modeled as where “” denotes the two-dimensional linear convolution operator, is the degraded image, is the original image, is the blur kernel and is the additive Gaussian noise.
Common nonblind image deconvolution aims to estimate the true image from the degraded observations assuming that the blur kernel is known. A proper incorporation of prior knowledge about the original image into the restoration process could be a good option to solve this ill-posed problem. In the past, the nonblind deconvolution approaches such as Tikhonov regularization, total variation (TV) regularization and wavelet frame regularization have been proposed [5], [6], [7], [8], [9]. Unfortunately, there are most real applications where the blur kernel is either unknown or partially known, and little information is available about the original image [10], [11], [12]. In this case, the original image estimation process is called blind deconvolution. We then need to simultaneously estimate the blur kernel and recover the original image directly from the degraded observation with only partial or no information about degradation and the imaging system. The major difficulty experienced in blind deconvolution lies in the insufficient information of two unknown variables and the existence of the additive noise. This poses difficult problems in the process of image restoration when there may be many or even possibly an infinite number of unmeaningful solutions.
Blind deconvolution is a very challenging bilinear ill-posed inverse problem with respect to the original image and the blur kernel. To compensate for the ill-posedness, some certain priors are required for both the sharp image and the blur kernel respectively to regularize the solution space. Usually, regularization methods formulate the image restoration problem as a minimization problem of the form where is the image to recover, is the blur kernel to estimate, is the observed image, and are positive regularization parameters, and are the regularization terms on the recovered image and the estimated blur kernel respectively. Early regularization methods assume the smooth priors on the estimated image and the blur kernel. In [13], You and Kaveh used the norm of image and kernel derivatives as the regularization terms for blind deconvolution. In [14], Krishnan et al. investigated the normalized sparsity measure method for blind deconvolution of images. In the kernel estimation process, the regularization function is the ratio of the norm to the norm on the estimated images. Once the kernel has been estimated, the nonblind deconvolution method based on the hyper-Laplacian prior is used to recover the original images. In recent years, the TV norms and its variations have been popular choices of the regularization term to solve various blind deconvolution problems. The main advantage of the TV regularization method is that it allows the discontinuities in both the original image and the blur kernel, thus making it superior to the norm regularization method in the cases where the original image and the blur kernel have discontinuities [15]. In [16], an effective regularization approach is proposed to remove motion blurring from the image by regularizing the sparsity of both the original image and the motion blur kernel under tight wavelet frame systems. An adapted version of the split Bregman method is proposed to efficiently solve the resulting minimization problem. Some other regularization methods have also been proposed for the blind image deconvolution problem. For example, in [17], an enhanced low-rank prior method is proposed for blind image deconvolution by using the low-rank prior of similar patches from both the original image and its gradient map as the regularization term on the original image and the norm of the blur kernel as the regularization term on the blur kernel. The iteration-wise -norm regularizers together with data-driven strategy were proposed in [18] for blind image deconvolution. In [19], Levin et al. analyzed and evaluated recent blind deconvolution algorithms both theoretically and experimentally. The conclusions from their analysis are useful for directing blind deconvolution research. In [20], Perrone et al. studied a Logarithmic image prior for blind deconvolution. To minimize the Maximum a Posteriori (MAP) formulation, both the primal–dual approach and the majorization–minimization algorithm were used to cope with the non-convexity of the logarithmic prior. In [21], Xu et al. considered a generalized and mathematically sound sparse expression for motion deblurring in the kernel estimation process. In the final step, they restored the natural image by nonblind deconvolution given the final kernel estimate. A hyper-Laplacian prior with the norm regularization is used. In [22], Wang et al. proposed an effective image decomposition model for the blind image deconvolution problem. Their idea is to make use of a cartoon-plus-texture image decomposition procedure into the deconvolution problem. A variational approach was developed in [23] for restoring images corrupted by noisy blur kernels and additive noise. The objective function is composed of three terms: the data-fitting term between the observed image and the product of the estimated blurring function and the estimated image, the squared difference between the estimated blurring function and its mean, and the total variation regularization term for the estimated image.
The blind deconvolution of binary images has attracted considerable attention in recent years due to its practical applications. There have many attempts to deal with the blind binary image deconvolution problem. In [24], an approach that guarantees to reach the global minimum of the Rudin–Osher–Fatemi (ROF) TV model restricted to the set of binary images was proposed. The approach is to consider the unconstrained minimization of a convex functional whose minimum is reached for the constrained global minimizer of the ROF model. In [25], an effective method that treats the blind binary image restoration problem as an iterated quadratic programming optimization problem was proposed. The method has the properties of fast convergence and good numerical stability, due to established schemes such as the interior-point algorithm. The output of the resulting algorithm is very nearly binary. In [26], a novel joint nonuniform illumination estimation and deblurring method for barcode signals based on a penalized nonlinear squares objective function was investigated. The objective function is based on the proper parameterization of a barcode signal and nonuniform illumination as well as a regularization on the illumination using a smoothness penalty. By the minimization of the objective function, the proposed method simultaneously estimates the barcode signal and illumination in the spatial domain. In [27], based on the bimodal characteristics of barcode image histograms, a simple target function that measures how similar a deconvoluted image is to a barcode was proposed. The most likely blur kernel can be found by minimizing the target function over the set of possible convolution kernels. In [28], a joint nonuniform illumination estimation and blind deconvolution was proposed for barcode signals by using evolutionary algorithms. A genetic algorithm combining discrete and continuous optimization which is successfully applied to decode real images with very strong noise and blur was constructed. In [29], a regularization approach was proposed for blind deblurring and denoising of Quick Response(QR) codes. The proposed method includes four steps: denoising the signal via a weighted TV flow, estimating the blur kernel by a higher-order smooth regularization method based upon comparison of the known finder pattern in the upper left corner with the denoised signal from the first step in the same corner, applying appropriately regularized deconvolution with the estimated kernel from the second step and thresholding the output of the third step. In [30], an increment constrained least squares filter was designed for estimating the out-of-focus blur kernel and recovering the barcode image. In [31], a partially blind deblurring method was proposed for barcode images when partial knowledge of the clean barcode is available. An image formation model based on geometrical optics, which involves the point spread function for the out-of-focus blur, was constructed for yielding good restorations. In [32], an effective regularizer based on intensity and gradient prior was proposed for text image deblurring. The proposed image prior is based on distinctive properties of text images, with which they developed an efficient optimization algorithm based on the half-quadratic splitting method to generate reliable intermediate results for kernel estimation. The splitting method guarantees that each sub-problem has a closed-form solution. For the final latent image restoration step, they presented an effective nonblind deconvolution method to remove artifacts for better deblurred results. Other blind deconvolution methods for binary images can be found in [33], [34], [35], [36] and references therein.
In this paper, we investigate the blind binary image deconvolution problem by fully utilizing the intrinsic characteristics of binary images. We propose a novel blind image deconvolution model using the sum of the rounding function of image pixel and the norm function of image gradient as the regularizer for the binary image, and the norm function as the regularizer for the motion blur kernel. The rounding function is used for binary constraint of the estimated image while the norm term is used to enforce the sparsity of image gradient. In the blind deconvolution process, the rounding term can force the pixel values of the estimated image to approach the binary pixel values. The norm regularizer is used to stabilize the solution of the blur kernel. We use a two-step iterative method based on decoupling of image restoration and blur kernel estimation to alternatively solve the proposed model. The half-quadratic splitting method is applied for the solution in the image restoration step, in which each subproblem has closed-form solution. In the blur kernel estimation step, the solution can be efficiently computed by the fast Fourier transforms (FFTs). By solving the proposed model iteratively, we get the estimated blur kernel and the intermediate restoration image. We compute the output approximate original image by the final deconvolution with the estimated blur kernel. Extensive experiments for the blind binary image deconvolution problem are given to demonstrate that the proposed method outperforms some existing state-of-the-art methods in terms of visual quality and peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).
Section snippets
Mathematical preliminaries
In this section, we are going to give the rounding operator and the half-quadratic splitting method.
Problem formulation
In [32], an effective regularizer based on intensity and gradient prior was proposed for the blind deconvolution of text images. However, the regularizer may not be the best choice, since the binary images have only two possible values for each pixel. Based on the specific property of the binary images, we apply the rounding function regularizer for the blind deconvolution of the binary images. The rounding function regularizer attains zero at the two pixel values and is a quadratic
Experimental results
In this section, we evaluate the performance of the proposed method for the blind deconvolution of binary images by comparing it with the following closely related methods:
Krishnan et al. [14]1: They investigated the normalized sparsity measure method for blind deconvolution of images. In the blur kernel estimation process, the regularization function of the estimated image is the ratio of the norm to the norm, and the regularization
Conclusions
In this paper, we investigate the problem of recovering the sharp binary image and the blur kernel from the motion degraded observation. In the proposed model, the binary prior of image pixel and the norm of image gradient are used to enforce the estimated image to be binary and the image gradient to be sparse respectively. We propose an effective numerical optimization algorithm for solving the proposed model based on the half-quadratic splitting scheme, which ensures that each sub-problem
Acknowledgments
This work was supported by NSF of Jiangsu Province, PR China (BK20181483), NSFC, PR China (11701079, 11671002, 61731009), STCSM, PR China project (13dz2260400), CPSF, PR China (2016M601360), Jiangsu Key Lab for NSLSCS, PR China (201806), Hai Yan project, PR China, Lianyungang 521 project, PR China and NSF of HHIT, PR China (Z2017004), Jilin Provincial Department of Education, PR China (JJKH20190293KJ).
References (43)
- et al.
Binary image deblurring with automatic binary value estimation
J. Electron. Imaging
(2018) - et al.
Nonlinear total variation based noise removal algorithms
Physica D
(1992) - et al.
Regularized iterative weiner filter method for blind image deconvolution
J. Comput. Appl. Math.
(2018) - et al.
Two-dimensional bar code out-of-focus deblurring via the increment constrained least squares filter
Pattern Recognit. Lett.
(2013) - et al.
Bar code recovery via the EM algorithm
IEEE Trans. Signal Process.
(1998) Blind deconvolution of bar code signals
Inverse Problems
(2003)An alternating minimization algorithm for binary image restoration
IEEE Trans. Image Process.
(2012)- et al.
Solution of Ill-Posed Problems
(1977) - et al.
Convex blind image deconvolution with inverse filtering
Inverse Problems
(2018) - et al.
Image restoration: total variation, wavelet frames, and beyond
J. Amer. Math. Soc.
(2012)