Completing a corrupted image by filling in correct structures and reasonable textures for a complex scene remains an elusive challenge. In case that a missing hole involves diverse semantic information, conventional two-stage approaches based on structural information often lead to unreliable structural prediction and ambiguous visual texture generation. To address the problem, we propose a SEmantic GUidance and Estimation Network (SeGuE-Net) that iteratively evaluates the uncertainty of inpainted visual contents based on pixel-wise semantic inference and optimize structural priors and inpainted contents alternatively. Specifically, SeGuE-Net utilizes semantic segmentation maps as guidance in each iteration of image inpainting, under which location-dependent inferences are re-estimated, and, accordingly, poorly-inferred regions are refined in subsequent iterations. Extensive experiments on real-world images demonstrate the superiority of our proposed method over state-of-the-art approaches in terms of clear boundaries and photo-realistic textures.

中文翻译：

---

不确定性的语义指导和图像修复估计

通过为复杂场景填充正确的结构和合理的纹理来完成损坏的图像仍然是一项艰巨的挑战。如果缺失的洞涉及不同的语义信息，则基于结构信息的常规两阶段方法通常会导致不可靠的结构预测和模糊的视觉纹理生成。为了解决这个问题，我们提出了一个语义指导和估计网络（SeGuE-Net），该网络基于像素语义推理来迭代评估已修复视觉内容的不确定性，并交替优化结构先验和已修复内容。具体而言，SeGuE-Net在图像修复的每次迭代中都使用语义分割图作为指导，在此基础上，重新估计与位置相关的推理，因此，较差的推断区域在后续迭代中进行细化。在真实世界的图像上进行的大量实验证明，在清晰的边界和逼真的纹理方面，我们提出的方法优于最新方法。