This paper addresses the traditional issue of restoring a high-resolution (HR) facial image from a low-resolution (LR) counterpart. Current state-of-the-art super-resolution (SR) methods commonly adopt the convolutional neural networks to learn a non-linear complex mapping between paired LR and HR images. They discriminate local patterns expressed by the neighboring pixels along the planar directions but ignore the intrinsic 3D proximity including the depth map. As a special case of general images, the face has limited geometric variations, which we believe that the relevant depth map can be learned and used to guide the face SR task. Motivated by it, we design a network including two branches: one for auxiliary depth map estimation and the other for the main SR task. Adaptive geometric features are further learned from the depth map and used to modulate the mid-level features of the SR branch. The whole network is implemented in an end-to-end trainable manner under the extra supervision of depth map. The supervisory depth map is either a paired one from RGB-D scans or a reconstructed one by a 3D prior model of faces. The experiments demonstrate the effectiveness of the proposed method and achieve improved performance over the state of the arts.

本文解决了从低分辨率（LR）副本恢复高分辨率（HR）面部图像的传统问题。当前最新的超分辨率（SR）方法通常采用卷积神经网络来学习成对的LR和HR图像之间的非线性复杂映射。他们区分了相邻像素沿平面方向表示的局部图案，但忽略了包括深度图在内的固有3D邻近度。作为一般图像的一种特殊情况，人脸具有有限的几何变化，我们认为可以学习相关的深度图并将其用于指导人脸SR任务。因此，我们设计了一个包括两个分支的网络：一个分支用于辅助深度图估计，另一个分支用于主要SR任务。可以从深度图中进一步学习自适应几何特征，并将其用于调制SR分支的中层特征。整个网络在深度图的额外监督下以端到端的可培训方式实现。监控深度图可以是来自RGB-D扫描的配对图像，也可以是通过3D先验面孔模型重建的图像。实验证明了所提出方法的有效性，并在现有技术水平上实现了改进的性能。