Traditional super-resolution techniques are generally presented as optimization problems with variations in the choice of optimization methods and cost functions. Even for the overdetermined cases, the problem is ill-conditioned. The situation is worsened when considering underdetermined cases with unknown regions due to occlusions or lack of data. Deep learning-based methods have shown promise in solving a similar problem. One recent advancement has come in the form of partial convolutions, which were developed to perform infilling of holes in images. When used in an appropriate deep neural network, this particular variant of the convolutional filter has shown great promise in approximating missing spatial information. The method described is formulated as a two-stage process. Lower resolution images are first registered and placed on a high-resolution grid. The problem is then treated as an in-painting task where the missing regions are reconstructed using a deep neural network with partial convolutional filters. We compare our method against deep learning-based single image super-resolution methods and classical multi-image super-resolution techniques using two similarity metrics and show that our method is more robust to occlusions and errors in registration while also producing higher quality outputs.

中文翻译：

---

强大的基于深度学习的多图像超分辨率修复

传统的超分辨率技术通常作为优化问题提出，其优化方法和成本函数的选择存在差异。即使对于超定情况，问题也是病态的。当考虑由于遮挡或缺乏数据而导致区域未知的不确定病例时，情况会更加恶化。基于深度学习的方法已显示出解决类似问题的希望。最近的一项进步是以部分卷积的形式出现的，它被开发用来填充图像中的孔。当在适当的深度神经网络中使用时，卷积滤波器的此特定变体在近似缺少的空间信息方面显示出了巨大的希望。所描述的方法被表述为两个阶段的过程。首先注册较低分辨率的图像，然后将其放置在高分辨率网格上。然后将问题视为绘画任务，其中使用带有部分卷积滤波器的深层神经网络重建缺失区域。我们将我们的方法与使用两个相似性指标的基于深度学习的单图像超分辨率方法和经典的多图像超分辨率技术进行了比较，结果表明我们的方法在遮挡和配准错误方面更强大，同时还可以产生更高质量的输出。