Image completion involves filling plausible contents to missing regions in images. Current image completion methods produce only one result for a given masked image, although there may be many reasonable possibilities. In this paper, we present an approach for pluralistic image completion—the task of generating multiple and diverse plausible solutions for free-form image completion. A major challenge faced by learning-based approaches is that usually only one ground truth training instance per label for this multi-output problem. To overcome this, we propose a novel and probabilistically principled framework with two parallel paths. One is a reconstructive path that utilizes the only one ground truth to get prior distribution of missing patches and rebuild the original image from this distribution. The other is a generative path for which the conditional prior is coupled to the distribution obtained in the reconstructive path. Both are supported by adversarial learning. We then introduce a new short+long term patch attention layer that exploits distant relations among decoder and encoder features, to improve appearance consistency between the original visible and the generated new regions. Experiments show that our method not only yields better results in various datasets than existing state-of-the-art methods, but also provides multiple and diverse outputs.

图像补全涉及将合理的内容填充到图像中的缺失区域。当前的图像补全方法对于给定的蒙版图像仅产生一个结果，尽管可能存在许多合理的可能性。在本文中，我们提出了一种多元图像补全的方法- 为自由形式的图像完成生成多种不同的合理解决方案的任务。基于学习的方法面临的一个主要挑战是，对于这个多输出问题，每个标签通常只有一个地面实况训练实例。为了克服这个问题，我们提出了一种具有两条平行路径的新颖的概率原则框架。一种是重建路径，它利用唯一的一个基本事实来获得缺失补丁的先验分布并从该分布重建原始图像。另一种是生成路径，其中条件先验与在重建路径中获得的分布耦合。两者都得到对抗性学习的支持。然后我们引入了一个新的短期+长期补丁注意层，它利用了解码器和编码器特征之间的远距离关系，提高原始可见区域和生成的新区域之间的外观一致性。实验表明，与现有的最先进方法相比，我们的方法不仅在各种数据集中产生了更好的结果，而且还提供了多种不同的输出。