Image inpainting attempts to fill the missing areas of an image with plausible content that is visually coherent with the image context. Semantic image inpainting has remained a challenging task even with the emergence of deep learning-based approaches. We propose a deep semantic inpainting model built upon a generative adversarial network and a dense U-Net network. Such a design helps achieve feature reuse while avoiding feature explosion along the upsampling path of the U-Net. The model also uses a composite loss function for the generator network to enforce a joint global and local content consistency constraint. More specifically, our new loss function combines the global reconstruction loss characterizing the semantic similarity between the missing and known image regions with the local total variation loss characterizing the natural transitions among adjacent regions. Experimental results on CelebA-HQ and Paris StreetView datasets have demonstrated encouraging performance when compared with other state-of-the-art methods in terms of both quantitative and qualitative metrics. For the CelebA-HQ dataset, the proposed method can more faithfully infer the semantics of human faces; for the StreetView dataset, our method achieves improved inpainting results in terms of more natural texture transitions, better structural consistency, and enriched textural details.

中文翻译：

---

走向语义图像修复：全局上下文与局部几何图形相遇的地方

图像修补尝试用与图像上下文在视觉上一致的合理内容填充图像的缺失区域。即使出现了基于深度学习的方法，语义图像修复仍然是一项艰巨的任务。我们提出了一个基于生成对抗网络和密集U-Net网络的深度语义修复模型。这样的设计有助于实现功能重用，同时避免沿U-Net的上采样路径发生功能爆炸。该模型还将生成器网络的复合损失函数用于实施联合的全局和局部内容一致性约束。进一步来说，我们的新损失函数将整体重建损失（表征缺失和已知图像区域之间的语义相似性）与局部总变化损失（表征相邻区域之间的自然过渡）相结合。在定量和定性指标方面，与其他最新方法相比，CelebA-HQ和Paris StreetView数据集上的实验结果证明了令人鼓舞的性能。对于CelebA-HQ数据集，该方法可以更真实地推断人脸的语义。对于StreetView数据集，我们的方法在更自然的纹理过渡，更好的结构一致性和丰富的纹理细节方面实现了改进的修复效果。在定量和定性指标方面，与其他最新方法相比，CelebA-HQ和Paris StreetView数据集上的实验结果证明了令人鼓舞的性能。对于CelebA-HQ数据集，该方法可以更真实地推断人脸的语义。对于StreetView数据集，我们的方法在更自然的纹理过渡，更好的结构一致性和丰富的纹理细节方面实现了改进的修复效果。在定量和定性指标方面，与其他最新方法相比，CelebA-HQ和Paris StreetView数据集上的实验结果证明了令人鼓舞的性能。对于CelebA-HQ数据集，该方法可以更真实地推断人脸的语义。对于StreetView数据集，我们的方法在更自然的纹理过渡，更好的结构一致性和丰富的纹理细节方面实现了改进的修复效果。