Depth-aware total variation regularization for underwater image dehazing

https://doi.org/10.1016/j.image.2021.116408Get rights and content

Abstract

Underwater images often show severe quality degradation due to the light absorption and scattering effects in water medium. This paper introduces a scene depth regularized underwater image dehazing method to obtain high-quality underwater images. Unlike previous underwater image dehazing methods that usually calculate a transmission map or a scene depth map using priors, we construct an exponential relationship between transmission map and normalized scene depth map. An initial scene depth is first estimated by the difference between color channels. Then it is refined by total variation regularization to keep structures while smoothing excessive details. An alternating direction algorithm is given to solve the optimization problem. Extensive experiments demonstrate that the proposed method can effectively improve the visual quality of degraded underwater images, and yields high-quality results comparative to the state-of-the-art underwater image enhancement methods quantitatively and qualitatively.

Introduction

Nowadays, various marine applications and services such as scene understanding, image/video compression and transmission, and underwater surveillance depend to a large extent on the availability of high-quality input images to meet expectations for quality of experience. However, captured underwater images often exhibit degradation effects, such as inaccurate colors and low contrast, which poses challenges for exploration of complex marine environments. As light travels in water, it is exponentially attenuated by the absorption and scattering effects. These effects are due not only to the water itself, but also to small observable floating particles or the dissolved organic matter [1], [2]. The absorption effect reduces the amount of light based on the wavelength, rendering underwater images to be dominated by inaccurate colors such as green or blue. Moreover, the reduction of light limits the distance the sensor perceives the radiation of an object. The scattering effect introduces a distance-dependent additive component to the image, reducing the contrast of the image. The further the object, there is more intervening medium and thus the scattering increases.

In physics-based underwater image recovery, researchers usually use a common image formation model [3] to improve the visibility of degraded underwater images. Image recovery based on the image formation model is an ill-posed problem that is typically separated into two parts: estimating the medium transmission and the global veiling-light. Here we still estimate the two parameters but estimate the transmission through a normalized scene depth instead of direct estimate the transmission using the priors. Additionally, the image formation model describes light propagation between the scene and the sensor on a horizontal LOS, but does not include the extinction when light propagates vertically. In haze, the color cast caused by this extinction is very small and can be ignored. Many underwater image recovery methods originate from dehazing methods and thus often continue with this assumption. However, both the scene radiance and the veiling light are heavily affected by the vertical propagation in underwater environments. Here, we treat the effect of the vertical propagation as a global color cast, and use a local white balance method to remove it.

Contributions: Previous underwater image dehazing methods usually calculate a transmission map or a scene depth map based on dark channel prior [4] or the maximum intensity prior [5]. In this paper, we construct an exponential relationship between the transmission map and the normalized scene depth map. An initial normalized scene depth map is first estimated by the difference between color channels. Since the amount of haze on an object is related to its depth, not its texture or color, the desired normalized scene depth map is expected to be smooth except at depth discontinuities. To realize this purpose, we refine the initial normalized scene depth map by imposing a structure prior on it, as the final normalized scene depth map. An augmented Lagrange multiplier based alternating direction minimization algorithm is given to exactly solve the optimization problem. Having the well-constructed normalized scene depth map, the enhancement can be achieved accordingly. Additionally, we adopt a local white balance prior to the image dehazing algorithm to reduce the effect of the vertical propagation not included in the image dehazing algorithm. Experiments on a number of challenging images are conducted to reveal the advantages of our method in comparison with other state-of-the-art methods.

Section snippets

Related work

Underwater image restoration and enhancement has drawn considerable attention during the past few years to help with underwater environmental exploration. Generally, the relevant work can be divided into two categories according to using physical imaging models or not. Physics-based image restoration is ill-posed and need to calculate some model parameters. Image enhancement without using any physical models directly adjusts the pixel values to generate a more visually pleasing image.

Most of

Visibility restoration algorithm

As illustrated in Fig. 1, homogeneous skylight is attenuated while it propagates through water to the scene, and attenuated further between the scene and the sensor. The common underwater image formation model [3] describing light propagation between the scene and the sensor is given by: Ic(x)=Jc(x)tc(x)+Ac(1tc(x))where x is the position of the pixel in the image, Ic(x) denotes the total radiance reaching the sensor at each pixel x and color channel c{r,g,b}. Jc(x) is the scene radiance that

Experimental results

This section demonstrates the utility of the proposed method in the context of underwater image enhancement applications. First, in Fig. 4, Fig. 5 we prove the effectiveness of the proposed local white balance (LWB) and variation-based dehazing (VD) approaches, respectively. Then, in Fig. 6, Fig. 7, Fig. 8 and Table 1, Table 2, we compare the proposed method (LWB + VD) with several state-of-the-art underwater image enhancement methods qualitatively and quantitatively. The objective performance

Conclusion

In this paper, we propose an underwater image recovery method with scene depth regularization. Unlike existing single underwater image enhancement methods, our method construct an exponential relationship between the transmission map and the normalized scene depth map. We estimate an initial scene depth by the difference between color channels, and then refine it by total variation regularization to keep structures while smoothing excessive details. An alternating direction algorithm is used to

CRediT authorship contribution statement

Xueyan Ding: Writing – original draft, Writing – review & editing, Conceptualization, Methodology. Zheng Liang: Validation, Data curation, Methodology. Yafei Wang: Writing – review & editing, Validation, Data curation, Funding acquisition. Xianping Fu: Supervision, Project administration, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

The authors sincerely thank the editors and anonymous reviewers for the very helpful and kind comments to assist in improving the presentation of our paper. This work was supported in part by the National Natural Science Foundation of China Grant 61802043, Grant 62002043 and Grant 62176037, by the Liaoning Revitalization Talents Program, China Grant XLYC1908007, by the Foundation of Liaoning Key Research and Development Program, China Grant 201801728, by the Fundamental Research Funds for the

References (46)

  • ZhaoXinwei et al.

    Deriving inherent optical properties from background color and underwater image enhancement

    Ocean Eng.

    (2015)
  • SchettiniRaimondo et al.

    Underwater image processing: state of the art of restoration and image enhancement methods

    EURASIP J. Adv. Signal Process.

    (2010)
  • JianMuwei et al.

    Underwater image processing and analysis: A review

    Signal Process., Image Commun.

    (2020)
  • KoschmiederHarald

    Theorie der horizontalen sichtweite

    Beitrage Zur Phys. Freien Atmos.

    (1924)
  • HeKaiming et al.

    Single image haze removal using dark channel prior

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • Carlevaris-BiancoNicholas et al.

    Initial results in underwater single image dehazing

  • PengYan-Tsung et al.

    Underwater image restoration based on image blurriness and light absorption

    IEEE Trans. Image Process.

    (2017)
  • LiChong-Yi et al.

    Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior

    IEEE Trans. Image Process.

    (2016)
  • BermanDana et al.

    Underwater single image color restoration using haze-lines and a new quantitative dataset

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2020)
  • LiangZheng et al.

    Effective polarization-based image dehazing with regularization constraint

    IEEE Geosci. Remote Sens. Lett.

    (2020)
  • DrewsPaulo LJ et al.

    Underwater depth estimation and image restoration based on single images

    IEEE Comput. Graph. Appl.

    (2016)
  • PengYan-Tsung et al.

    Generalization of the dark channel prior for single image restoration

    IEEE Trans. Image Process.

    (2018)
  • GhaniAhmad Shahrizan Abdul et al.

    Underwater image quality enhancement through integrated color model with Rayleigh distribution

    Appl. Soft Comput.

    (2015)
  • Cited by (27)

    • Attention-guided dynamic multi-branch neural network for underwater image enhancement

      2022, Knowledge-Based Systems
      Citation Excerpt :

      Handcrafted prior-based methods. Based on the underwater physical imaging model [17], a large number of handcrafted methods [18–20] execute the enhancement procedure in an inverse manner. Without any extra information, specific priors [21] as constraints are used to estimate the derived parameters (background light and transmission map) of the physical model [22].

    • FW-GAN: Underwater image enhancement using generative adversarial network with multi-scale fusion

      2022, Signal Processing: Image Communication
      Citation Excerpt :

      Song et al. [20] proposed a method combining underwater image restoration and color correction, which used background light statistical model and transmission map optimization for calculation. Ding et al. [21] proposed an underwater image dehazing method based on scene depth regularization to obtain high-quality underwater images. The initial scene depth is estimated by the difference between color channels and then refined by the total variational regularization method.

    • Conditional generative adversarial network with dual-branch progressive generator for underwater image enhancement

      2022, Signal Processing: Image Communication
      Citation Excerpt :

      Nevertheless, the model-free methods usually over-expose or under-enhance underwater images, and even introduce artifact halos to the achieved results when no reference standard can be used. Physical model-based pipelines [15–27] need to build the degradation model of underwater images and then improve the quality of underwater images by assuming and calculating many complex parameters in the constructed model, which limits the generalization ability and implementation of these methods. By contrast, deep learning-based models have achieved great success and gratifying results on low-level vision tasks in recent years, e.g., image deraining [28], image dehazing [29], and low-light image enhancement [30,31].

    View all citing articles on Scopus
    View full text