Elsevier

Pattern Recognition Letters

Volume 155, March 2022, Pages 159-164
Pattern Recognition Letters

GEVE: A generative adversarial network for extremely dark image/video enhancement

https://doi.org/10.1016/j.patrec.2021.10.030Get rights and content

Abstract

A variety of information of the real-time scenes is carried by the images and videos. Processing these images and videos in an intelligent way helps in many domains such as computer vision, object detection, deep learning and 3D reconstruction leaving its large usage in applications such as auto pilots, augmented reality, smart vehicles, etc. The quality of image and videos plays a vital role in case of real-time systems. One such scenario is where the images are captured without sufficient illumination. Images captured in cameras where sufficient light is not present, leads to noisy and information-loss images. It is a fact that, dark images have mainly two aspects which make its study a difficult task. They are at its low dynamic range and high propensity for generating high noise levels. Hence, an approach based on deep learning based system is adopted. For this purpose, Generative Adversarial Network (GAN) based Extremely Dark Video Enhancement Network (GEVE) model is proposed. The main objective of GEVE is to team the model with low /normal- light image pairs. Thus, GAN network learns the translation from light feeble images and images captured under normal illumination and automatically translate original images taken under extremely low light conditions into images of quality. It is clearly observed that the proposed GEVE outperforms the known state-of-art techniques. We are the view that the proposed system is an ideal candidate to handle dark image/video frames.

Introduction

Definition of dark image/videos is not limited to images taken during night and encomposes all the images captured under different levels of light from various sources and luminance level less than the day time light outside the studio. We define low light videos as indoor photographs without adequate lighting or light which is not visible to eye at night. Images captured under these conditions have lower contrast besides induction of noise in the images due to inadequate and varying illumination. The causes for uneven illuminations may be low level of lighting or weather conditions [1] prevailing, or it can be due to long distance from light source or poor quality of sensors used in the image capturing system. Due to the above said factors the quality of images and the integrity of the images suffer resulting in unnatural look. Aim of dark image enhancement is to bring such images on par with the images taken under extremely dark condition is much difficult because the originally taken images through surveillance cameras are under deteriorated condition of illumination interspaced with uneven lighting. An alternative way to enhance images/videos taken under relative luminance of 0.1 lux or less is proposed. Proposed model for dark image is based on deep learning technology GAN [2].The major issue taken in this research is the Extreme Dark Video Enhancement Network. There are many techniques available in literature but the expected level of enhancement could not be achieved. In order to bring in more enhancement, this paper proposes GAN (“Generative Adversarial Network”) that are based on GEVE model for solving the issues with the image and video enhancements. The remaining section of the paper isorganized as follows. An overview of the existing methods such as contrast enhancements, intelligent enhancements of low lights videos that are available in literature are discussed in Section 2. Section 3 clearly describes the proposed methodology i.e. the GAN and GEVE. The results of the proposed approach and its comparison with existing works are discussed in the Section 4. Section 5 concludes the findings of the research work.

Section snippets

Traditional image contrast enhnacement methods

Traditional techniques used for image enhancement follows the histogram equalization. This method attempts to stretch the intensity evenly over the image to provided on enhanced contrast. Histogram enhancement [3] aims at an adaptive histographic enhancement. Contrast limited adaptive histogram equalization (CLAHE) [4] controls the level of brightness upgradation obtained from HE. HE has limitations as the color contrasts are highly compromised. Color in the darkened areas of the images are not

Methodology

In the Fig. 1, GAN is shown as consisting of two networks namely generating network and discriminating network. The generating continuously generates data which can deceive the discriminating network. At the same time discriminatory network drives continuously to distinguish between actual and fake image/video frame till a balanced state is arrived. In this extremely low light image /Video frame are provided as input to generator. This generator in turn produces an enhanced image. Training of

Limitation and future work

The present GEVE model works well for indoor data with no illumination. Instead of Google Colab, GPU enabled systems can be used to get more running time and batch size, image size can also be increased while training. Our future work concentrates on 3D convolution layer for video enhancement and the unsatisfied case of outdoor videos.

Conclusion

The GAN-based approach that is discussed here is for boosting extremely dim light images/videos. On the other hand GEVE approach is for adaptively enhance image contrast of extremely low light image/video frame. It works well for indoor data without illumination. It is our assessment that GCF and SSIM methods eminently provides a method for boosting dim light images. GLAD-Net, MSRCR and Retinex-Net algorithms provides lower accuarcy of 79%, 84% and 88% respectively whereas the proposed GEVE

Declaration of Competing Interest

None.

References (47)

  • X. Fu et al.

    A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation

    IEEE Trans. Image Process.

    (2015)
  • X. Guo et al.

    Lime: low-light image enhancement via illumination map estimation

    IEEE Trans. Image Process.

    (2017)
  • K. He et al.

    Single image haze removal using dark channel prior

    Pattern Anal. Mach. Intell. (PAMI), IEEE Trans.

    (2011)
  • L. Li

    A low-light image enhancement method for both denoising and contrast enlarging

  • L. Tao et al.

    LLCNN: a convolutional neural network for low-light image enhancement

  • L. Shen, Z. Yue, F. Feng, Q. Chen, S. Liu, and J. Ma. 2017. MSR-net:low-light image enhancement using deep...
  • C. Wei, W. Wang, W. Yang, and J. Liu. "Deep retinex decomposition for low-light enhancement." arXiv preprint...
  • C. Ledig

    Photo-realistic single image super-resolution using a generative adversarial network

  • X. Wang

    Esrgan: enhanced super-resolution generative adversarial networks

  • W. Wang

    Gladnet: low-light enhancement network with global awareness

  • J. Chen

    Image blind denoising with generative adversarial network based noise modeling

  • L. Wang, and L. Shao. 2011. "An algorithm for repairing low-quality video enhancement techniques based on trained...
  • X. Dong, W. Li, G. Wang, Y. Lu, and W. Meng. 2011. "An efficient and integrated algorithm for video enhancement in...
  • Cited by (0)

    View full text