Salient edges combined with image structures for image deblurring

Highlights

• We develop a novel structure-based deblurring term to provide guidance information for image structures.

• We build a more expressive model with image structure and salient edges.

• We incorporate the ${\mathit{L}}_0$-norm of the gradient for edge-preserving and detail removal.

• Quantitative and qualitative evaluations show that our method is comparable to state-of-the-art methods.

Abstract

This work investigates a simple yet effective image deblurring method that combines salient image edges with structures. Typically, the vast majority of edge-based work focuses on advancing the salient edges information for blind image deblurring while ignoring the refined scale of the image structures. With this in mind, we show how to effectively exploit the image structure to blind image deblurring in this paper, i.e., we adopt a mutually guided image filter to guide the restoration of the image structure while salient edges provide edge information for blur kernel estimation. Thus, a more expressive deblurring model that employs ${\mathit{L}}_2$-regularization for salient edges and image structure, together with the ${\mathit{L}}_0$-regularization for image gradients is proposed. We find that the quality of the recovered kernel is thereby improved, and the deblurring results are more satisfactory. Experimental results show that our method outperforms state-of-the-art methods in terms of benchmark-based datasets and real scenarios, as well as computational efficiency.

Introduction

Blind deconvolution is a common issue in digital image processing. The purpose of blind deconvolution is to recover a blur kernel K and a desirable clear image I from the input blurry image B. If the blur is uniform and spatially invariant, a standard deblurring model is usually formulated as

$$B=I\otimes K+N$$

where the $\otimes$ stands for the convolution operator, N represents the addition Gaussian noise. Model (1) is named blind deconvolution when the blur kernel is unknown; Otherwise, it is non-blind deblurring [1], [2]. More recently, Dong et al. [3], [4] creatively proposed two simple and effective deep non-blind methods, one of which is learning a spatially variant MAP model for non-blind image deblurring [3], and the other is to perform an explicit deconvolution process in feature space by combining a classical wiener deconvolution framework with learned deep features [4]. Both methods can progressively restore fine-scale structures and details, along with robustness to noise and saturation. As it is hard to find reasonably pairs (I, K) from the blurred image B, several methods utilize extra knowledge for this problem.

In recent years, deblurring methods using image prior information have been leveraged in the maximum a posterior (MAP) framework [5]. For instances, sparse priors of the blur kernels or latent images [6], channel-based priors [7], [8], gradient-based priors [9], [10], patch-based priors [11], [12], [13], low-rank priors [14], [15]. Such statistical priors focus on finding certain features of the image that are sufficient to distinguish between blurred and clear images and prefer clear ones in the MAP framework. But the prior-based approach is time-consuming due to the extensive computation involved.

Contrary to investigated blind image deblurring works such as those mentioned above, the methods that focus on the edge-based are widely used to explicitly or implicitly achieve salient edges to facilitate kernel estimation [16], [17], [18], [19], [20], [21], [22], [23]. Commonly, salient edge methods mainly use filtering techniques [24], [25], [26], [27], [28] for edge extraction to provide salient edge information for kernel estimation, and this simple operation leads to a time-saving implementation. However, most of the previously published literature is very limited because they do not consider the use of image structure while studying the salient edge selection. As a result, the recovery result was not ideal.

We develop a novel structure-based deconvolution term that enforces the estimated latent image to contain finer scales and builds a more expressive model with image structure and salient edges through MAP. In addition, an ${\mathit{L}}_0$ constraint on the gradient is also applied for edge-preserving and detail removal. Fig. 1 is a visual example showing the effectiveness and robustness of the proposed image deblurring model. The contributions are highlighted as follows.

(1) We develop a novel structure-based deconvolution term that uses mutually guided image filtering (muGIF) to provide guidance information for image structures.

(2) We enhance the robustness of the salient edge selection by building a more expressive model with image structure and salient edges through MAP framework.

(3) A new salient edge selection model is proposed and further the ${\mathit{L}}_0$-norm of the gradient is adopted for edge-preserving and detail removal.

(4) Extensive experiments have shown that our approach achieves better results on both benchmark datasets and real scenarios. Quantitative and qualitative evaluations show that our method is comparable to state-of-the-art methods.