Given a monocular face image as input, 3D face geometry reconstruction aims to recover a corresponding 3D face mesh. Recently, both optimization-based and learning-based face reconstruction methods have taken advantage of the emerging differentiable renderer and shown promising results. However, the differentiable renderer, mainly based on graphics rules, simplifies the realistic mechanism of the illumination, reflection, \etc, of the real world, thus cannot produce realistic images. This brings a lot of domain-shift noise to the optimization or training process. In this work, we introduce a novel Generative Adversarial Renderer (GAR) and propose to tailor its inverted version to the general fitting pipeline, to tackle the above problem. Specifically, the carefully designed neural renderer takes a face normal map and a latent code representing other factors as inputs and renders a realistic face image. Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization. Equipped with the elaborated GAR, we further proposed a novel approach to predict 3D face parameters, in which we first obtain fine initial parameters via Renderer Inverting and then refine it with gradient-based optimizers. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed generative adversarial renderer and the novel optimization-based face reconstruction framework. Our method achieves state-of-the-art performances on multiple face reconstruction datasets.

中文翻译：

---

逆向生成对抗性渲染器，用于人脸重建

给定单眼脸部图像作为输入，3D脸部几何形状重建旨在恢复相应的3D脸部网格。最近，基于优化和基于学习的面部重建方法都利用了新兴的可区分渲染器，并显示出令人鼓舞的结果。但是，主要基于图形规则的可区分渲染器简化了现实世界中照明，反射等的逼真的机制，因此无法生成逼真的图像。这给优化或训练过程带来了很多域偏移噪声。在这项工作中，我们介绍了一种新颖的Generative Adversarial Renderer（GAR），并提出将其反向版本调整为适合一般的装配线，以解决上述问题。具体来说，经过精心设计的神经渲染器将脸部法线贴图和代表其他因素的潜在代码作为输入，并渲染出逼真的脸部图像。由于GAR学会了对复杂的真实世界图像进行建模，而不是依赖于简化的图形规则，因此它能够生成逼真的图像，从而从根本上抑制了训练和优化过程中的域偏移噪声。配备了精心设计的GAR，我们进一步提出了一种预测3D人脸参数的新颖方法，该方法首先通过Renderer Inverting获得精细的初始参数，然后使用基于梯度的优化器对其进行优化。已经进行了广泛的实验，以证明所提出的对抗式渲染器和新颖的基于优化的人脸重建框架的有效性。