1 February 2021 Robust deep learning-based multi-image super-resolution using inpainting
Henry Yau, Xian Du
Author Affiliations +
Abstract

Traditional super-resolution techniques are generally presented as optimization problems with variations in the choice of optimization methods and cost functions. Even for the overdetermined cases, the problem is ill-conditioned. The situation is worsened when considering underdetermined cases with unknown regions due to occlusions or lack of data. Deep learning-based methods have shown promise in solving a similar problem. One recent advancement has come in the form of partial convolutions, which were developed to perform infilling of holes in images. When used in an appropriate deep neural network, this particular variant of the convolutional filter has shown great promise in approximating missing spatial information. The method described is formulated as a two-stage process. Lower resolution images are first registered and placed on a high-resolution grid. The problem is then treated as an in-painting task where the missing regions are reconstructed using a deep neural network with partial convolutional filters. We compare our method against deep learning-based single image super-resolution methods and classical multi-image super-resolution techniques using two similarity metrics and show that our method is more robust to occlusions and errors in registration while also producing higher quality outputs.

© 2021 SPIE and IS&T 1017-9909/2021/$28.00 © 2021 SPIE and IS&T
Henry Yau and Xian Du "Robust deep learning-based multi-image super-resolution using inpainting," Journal of Electronic Imaging 30(1), 013005 (1 February 2021). https://doi.org/10.1117/1.JEI.30.1.013005
Received: 16 April 2020; Accepted: 9 December 2020; Published: 1 February 2021
Lens.org Logo
CITATIONS
Cited by 2 scholarly publications.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Lawrencium

Image registration

Super resolution

Fourier transforms

Image resolution

Image quality

Image processing

Back to Top