In this article, we present an end-to-end learning framework for detailed 3D face reconstruction from a single image. Our approach uses a 3DMM-based coarse model and a displacement map in UV-space to represent a 3D face. Unlike previous work addressing the problem, our learning framework does not require supervision of surrogate ground-truth 3D models computed with traditional approaches. Instead, we utilize the input image itself as supervision during learning. In the first stage, we combine a photometric loss and a facial perceptual loss between the input face and the rendered face, to regress a 3DMM-based coarse model. In the second stage, both the input image and the regressed texture of the coarse model are unwrapped into UV-space, and then sent through an image-to-image translation network to predict a displacement map in UV-space. The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage. The advantage of learning displacement map in UV-space is that face alignment can be explicitly done during the unwrapping, thus facial details are easier to learn from large amount of data. Extensive experiments demonstrate the superiority of our method over previous work.

中文翻译：

---

详细 3D 人脸重建的自监督学习。


在本文中，我们提出了一个端到端学习框架，用于从单个图像进行详细的 3D 人脸重建。我们的方法使用基于 3DMM 的粗略模型和 UV 空间中的位移图来表示 3D 脸部。与之前解决该问题的工作不同，我们的学习框架不需要监督使用传统方法计算的替代地面实况 3D 模型。相反，我们在学习过程中利用输入图像本身作为监督。在第一阶段，我们将输入面部和渲染面部之间的光度损失和面部感知损失相结合，以回归基于 3DMM 的粗略模型。在第二阶段，输入图像和粗模型的回归纹理都被展开到 UV 空间中，然后通过图像到图像转换网络发送以预测 UV 空间中的位移图。位移图和粗糙模型用于渲染最终的详细面部，它可以再次与原始输入图像进行比较，作为第二阶段的光度损失。在UV空间中学习位移图的优点是可以在展开过程中显式地完成面部对齐，因此更容易从大量数据中学习面部细节。大量的实验证明了我们的方法相对于以前的工作的优越性。