Blind motion deconvolution for binary images

https://doi.org/10.1016/j.cam.2021.113500Get rights and content

Abstract

Binary images are prevalent in digital systems and have a wide range of applications including texts, fingerprint recognition, handwritten signatures, stellar astronomy, barcodes, and vehicle license plates. The recorded binary images are often degraded by blur and additive noise due to environmental effects and imperfections in the imaging system. In this paper, we study the problem of recovering the sharp binary image and the blur kernel from the motion degraded observation. We propose a new minimization model by using the binary prior of image pixel and the l0 norm of image gradient to enforce the estimated image to be binary and the image gradient to be sparse respectively. An effective numerical optimization algorithm is applied for solving the proposed model. Extensive experiments for blind binary image deconvolution demonstrate that the proposed method outperforms some existing state-of-the-art methods in terms of visual quality and peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

Introduction

An image is binary if it has only two different values for each pixel. Binary images are prevalent in digital systems and have a wide range of applications including texts, fingerprint recognition, handwritten signatures, stellar astronomy, barcodes, and vehicle license plates [1], [2], [3]. Due to environmental effects and imperfections in the imaging system, the recorded binary images are usually degraded by blur and additive noise. For example, blurring often arises from the optical aberrations, relative movement between scene and camera, or object motion during exposure time, and noise is caused by malfunctioning pixels in camera sensors, faulty memory locations in hardware, or transmission in a noisy channel [4]. In the digital image processing, the degradation of the original image can be modeled as g=hf+η,where “” denotes the two-dimensional linear convolution operator, g is the degraded image, f is the original image, h is the blur kernel and η is the additive Gaussian noise.

Common nonblind image deconvolution aims to estimate the true image from the degraded observations assuming that the blur kernel is known. A proper incorporation of prior knowledge about the original image into the restoration process could be a good option to solve this ill-posed problem. In the past, the nonblind deconvolution approaches such as Tikhonov regularization, total variation (TV) regularization and wavelet frame regularization have been proposed [5], [6], [7], [8], [9]. Unfortunately, there are most real applications where the blur kernel is either unknown or partially known, and little information is available about the original image [10], [11], [12]. In this case, the original image estimation process is called blind deconvolution. We then need to simultaneously estimate the blur kernel and recover the original image directly from the degraded observation with only partial or no information about degradation and the imaging system. The major difficulty experienced in blind deconvolution lies in the insufficient information of two unknown variables and the existence of the additive noise. This poses difficult problems in the process of image restoration when there may be many or even possibly an infinite number of unmeaningful solutions.

Blind deconvolution is a very challenging bilinear ill-posed inverse problem with respect to the original image and the blur kernel. To compensate for the ill-posedness, some certain priors are required for both the sharp image and the blur kernel respectively to regularize the solution space. Usually, regularization methods formulate the image restoration problem as a minimization problem of the form minh,f12hfgF2+λ1J(f)+λ2J(h),where fRm×n is the image to recover, hRp×q is the blur kernel to estimate, gRm×n is the observed image, λ1 and λ2 are positive regularization parameters, J(f) and J(h) are the regularization terms on the recovered image f and the estimated blur kernel h respectively. Early regularization methods assume the smooth priors on the estimated image and the blur kernel. In [13], You and Kaveh used the l2 norm of image and kernel derivatives as the regularization terms for blind deconvolution. In [14], Krishnan et al. investigated the normalized sparsity measure method for blind deconvolution of images. In the kernel estimation process, the regularization function is the ratio of the l1 norm to the l2 norm on the estimated images. Once the kernel has been estimated, the nonblind deconvolution method based on the hyper-Laplacian prior is used to recover the original images. In recent years, the TV norms and its variations have been popular choices of the regularization term to solve various blind deconvolution problems. The main advantage of the TV regularization method is that it allows the discontinuities in both the original image and the blur kernel, thus making it superior to the l2 norm regularization method in the cases where the original image and the blur kernel have discontinuities [15]. In [16], an effective regularization approach is proposed to remove motion blurring from the image by regularizing the sparsity of both the original image and the motion blur kernel under tight wavelet frame systems. An adapted version of the split Bregman method is proposed to efficiently solve the resulting minimization problem. Some other regularization methods have also been proposed for the blind image deconvolution problem. For example, in [17], an enhanced low-rank prior method is proposed for blind image deconvolution by using the low-rank prior of similar patches from both the original image and its gradient map as the regularization term on the original image and the l2 norm of the blur kernel as the regularization term on the blur kernel. The iteration-wise lp-norm regularizers together with data-driven strategy were proposed in [18] for blind image deconvolution. In [19], Levin et al. analyzed and evaluated recent blind deconvolution algorithms both theoretically and experimentally. The conclusions from their analysis are useful for directing blind deconvolution research. In [20], Perrone et al. studied a Logarithmic image prior for blind deconvolution. To minimize the Maximum a Posteriori (MAP) formulation, both the primal–dual approach and the majorization–minimization algorithm were used to cope with the non-convexity of the logarithmic prior. In [21], Xu et al. considered a generalized and mathematically sound l0 sparse expression for motion deblurring in the kernel estimation process. In the final step, they restored the natural image by nonblind deconvolution given the final kernel estimate. A hyper-Laplacian prior with the l0.5 norm regularization is used. In [22], Wang et al. proposed an effective image decomposition model for the blind image deconvolution problem. Their idea is to make use of a cartoon-plus-texture image decomposition procedure into the deconvolution problem. A variational approach was developed in [23] for restoring images corrupted by noisy blur kernels and additive noise. The objective function is composed of three terms: the data-fitting term between the observed image and the product of the estimated blurring function and the estimated image, the squared difference between the estimated blurring function and its mean, and the total variation regularization term for the estimated image.

The blind deconvolution of binary images has attracted considerable attention in recent years due to its practical applications. There have many attempts to deal with the blind binary image deconvolution problem. In [24], an approach that guarantees to reach the global minimum of the Rudin–Osher–Fatemi (ROF) TV model restricted to the set of binary images was proposed. The approach is to consider the unconstrained minimization of a convex functional whose minimum is reached for the constrained global minimizer of the ROF model. In [25], an effective method that treats the blind binary image restoration problem as an iterated quadratic programming optimization problem was proposed. The method has the properties of fast convergence and good numerical stability, due to established schemes such as the interior-point algorithm. The output of the resulting algorithm is very nearly binary. In [26], a novel joint nonuniform illumination estimation and deblurring method for barcode signals based on a penalized nonlinear squares objective function was investigated. The objective function is based on the proper parameterization of a barcode signal and nonuniform illumination as well as a regularization on the illumination using a smoothness penalty. By the minimization of the objective function, the proposed method simultaneously estimates the barcode signal and illumination in the spatial domain. In [27], based on the bimodal characteristics of barcode image histograms, a simple target function that measures how similar a deconvoluted image is to a barcode was proposed. The most likely blur kernel can be found by minimizing the target function over the set of possible convolution kernels. In [28], a joint nonuniform illumination estimation and blind deconvolution was proposed for barcode signals by using evolutionary algorithms. A genetic algorithm combining discrete and continuous optimization which is successfully applied to decode real images with very strong noise and blur was constructed. In [29], a regularization approach was proposed for blind deblurring and denoising of Quick Response(QR) codes. The proposed method includes four steps: denoising the signal via a weighted TV flow, estimating the blur kernel by a higher-order smooth regularization method based upon comparison of the known finder pattern in the upper left corner with the denoised signal from the first step in the same corner, applying appropriately regularized deconvolution with the estimated kernel from the second step and thresholding the output of the third step. In [30], an increment constrained least squares filter was designed for estimating the out-of-focus blur kernel and recovering the barcode image. In [31], a partially blind deblurring method was proposed for barcode images when partial knowledge of the clean barcode is available. An image formation model based on geometrical optics, which involves the point spread function for the out-of-focus blur, was constructed for yielding good restorations. In [32], an effective l0 regularizer based on intensity and gradient prior was proposed for text image deblurring. The proposed image prior is based on distinctive properties of text images, with which they developed an efficient optimization algorithm based on the half-quadratic splitting method to generate reliable intermediate results for kernel estimation. The splitting method guarantees that each sub-problem has a closed-form solution. For the final latent image restoration step, they presented an effective nonblind deconvolution method to remove artifacts for better deblurred results. Other blind deconvolution methods for binary images can be found in [33], [34], [35], [36] and references therein.

In this paper, we investigate the blind binary image deconvolution problem by fully utilizing the intrinsic characteristics of binary images. We propose a novel blind image deconvolution model using the sum of the rounding function of image pixel and the l0 norm function of image gradient as the regularizer for the binary image, and the l2 norm function as the regularizer for the motion blur kernel. The rounding function is used for binary constraint of the estimated image while the l0 norm term is used to enforce the sparsity of image gradient. In the blind deconvolution process, the rounding term can force the pixel values of the estimated image to approach the binary pixel values. The l2 norm regularizer is used to stabilize the solution of the blur kernel. We use a two-step iterative method based on decoupling of image restoration and blur kernel estimation to alternatively solve the proposed model. The half-quadratic splitting method is applied for the solution in the image restoration step, in which each subproblem has closed-form solution. In the blur kernel estimation step, the solution can be efficiently computed by the fast Fourier transforms (FFTs). By solving the proposed model iteratively, we get the estimated blur kernel and the intermediate restoration image. We compute the output approximate original image by the final deconvolution with the estimated blur kernel. Extensive experiments for the blind binary image deconvolution problem are given to demonstrate that the proposed method outperforms some existing state-of-the-art methods in terms of visual quality and peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

Section snippets

Mathematical preliminaries

In this section, we are going to give the rounding operator and the half-quadratic splitting method.

Problem formulation

In [32], an effective l0 regularizer based on intensity and gradient prior was proposed for the blind deconvolution of text images. However, the l0 regularizer may not be the best choice, since the binary images have only two possible values for each pixel. Based on the specific property of the binary images, we apply the rounding function regularizer for the blind deconvolution of the binary images. The rounding function regularizer attains zero at the two pixel values and is a quadratic

Experimental results

In this section, we evaluate the performance of the proposed method for the blind deconvolution of binary images by comparing it with the following closely related methods:

Krishnan et al. [14]1: They investigated the normalized sparsity measure method for blind deconvolution of images. In the blur kernel estimation process, the regularization function of the estimated image is the ratio of the l1 norm to the l2 norm, and the regularization

Conclusions

In this paper, we investigate the problem of recovering the sharp binary image and the blur kernel from the motion degraded observation. In the proposed model, the binary prior of image pixel and the l0 norm of image gradient are used to enforce the estimated image to be binary and the image gradient to be sparse respectively. We propose an effective numerical optimization algorithm for solving the proposed model based on the half-quadratic splitting scheme, which ensures that each sub-problem

Acknowledgments

This work was supported by NSF of Jiangsu Province, PR China (BK20181483), NSFC, PR China (11701079, 11671002, 61731009), STCSM, PR China project (13dz2260400), CPSF, PR China (2016M601360), Jiangsu Key Lab for NSLSCS, PR China (201806), Hai Yan project, PR China, Lianyungang 521 project, PR China and NSF of HHIT, PR China (Z2017004), Jilin Provincial Department of Education, PR China (JJKH20190293KJ).

References (43)

  • WangWei et al.

    Convex regularized inverse filtering methods for blind image deconvolution

    Signal Image Video Process.

    (2016)
  • ChaudhuriSubhasis et al.

    Blind Image Deconvolution: Methods and Convergence

    (2014)
  • CampisiPatrizio et al.

    Blind Image Deconvolution: Theory and Applications

    (2007)
  • YouYuli et al.

    A regularization approach to joint blur identification and image restoration

    IEEE Trans. Image Process.

    (1996)
  • Dilip Krishnan, Terence Tay, Rob Fergus, Blind deconvolution using a normalized sparsity measure, in: IEEE Conference...
  • ChanTony F. et al.

    Total variation blind deconvolution

    IEEE Trans. Image Process.

    (1998)
  • CaiJianfeng et al.

    Framelet-based blind motion deblurring from a single image

    IEEE Trans. Image Process.

    (2012)
  • RenWenqi et al.

    Image deblurring via enhanced low-rank prior

    IEEE Trans. Image Process.

    (2016)
  • ZuoWangmeng et al.

    Learning iteration-wise generalized shrinkage-thresholding operators for blind deconvolution

    IEEE Trans. Image Process.

    (2016)
  • LevinAnat et al.

    Understanding blind deconvolution algorithms

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • PerroneDaniele et al.

    A logarithmic image prior for blind deconvolution

    Int. J. Comput. Vis.

    (2016)
  • Cited by (0)

    View full text