Ancient paintings are valuable culture legacy which can help archaeologists and culture researchers to study history and humanity. Most ancient artworks have damage problems, such as degradation, flaking and cracking. This work presents a novel controllable image inpainting framework with capability of incorporating suggestions from experts, which can help artists envisage how the ancient painting may have looked after a restoration. The framework leverages the content prediction power of deep convolutional neural network (CNN) and the nearest neighbor based pixel matching, where a deep CNN is designed to produce a coarse estimation of complete paintings by filling in missing regions and nearest neighbor based pixel matching is designed to map a mid-frequency estimation obtained from the deep CNN to high quality outputs in a controllable manner. In addition, we design a pixel descriptor using multi-scale neural features from different layers of a pre-trained deep network to capture different amounts of spatial context. Experimental results demonstrate that the proposed approach successfully predicts information in large missing regions and generates controllable high-frequency photo-realistic inpainting results.

古代绘画是宝贵的文化遗产，可以帮助考古学家和文化研究人员研究历史和人类。大多数古代艺术品都有损坏问题，例如降解，剥落和开裂。这项工作提出了一个新颖的可控图像修复框架，该框架具有吸收专家建议的能力，可以帮助艺术家设想古代绘画在修复后的样子。该框架利用了深度卷积神经网络（CNN）的内容预测能力和基于最近邻的像素匹配，其中深CNN被设计为通过填充缺失区域来生成完整绘画的粗略估计，并设计了基于最近邻的像素匹配以可控制的方式将从深CNN获得的中频估计映射到高质量输出。此外，我们使用来自预先训练的深度网络不同层的多尺度神经特征来设计像素描述符，以捕获不同数量的空间上下文。实验结果表明，该方法成功地预测了较大缺失区域中的信息，并生成了可控制的高频照片级逼真的修复结果。