We present a fully automatic system that can produce high-fidelity, photo-realistic three-dimensional (3D) digital human heads with a consumer RGB-D selfie camera. The system only needs the user to take a short selfie RGB-D video while rotating his/her head and can produce a high-quality head reconstruction in less than 30 s. Our main contribution is a new facial geometry modeling and reflectance synthesis procedure that significantly improves the state of the art. Specifically, given the input video a two-stage frame selection procedure is first employed to select a few high-quality frames for reconstruction. Then a differentiable renderer-based 3D Morphable Model (3DMM) fitting algorithm is applied to recover facial geometries from multiview RGB-D data, which takes advantages of a powerful 3DMM basis constructed with extensive data generation and perturbation. Our 3DMM has much larger expressive capacities than conventional 3DMM, allowing us to recover more accurate facial geometry using merely linear basis. For reflectance synthesis, we present a hybrid approach that combines parametric fitting and Convolutional Neural Networks (CNNs) to synthesize high-resolution albedo/normal maps with realistic hair/pore/wrinkle details. Results show that our system can produce faithful 3D digital human faces with extremely realistic details. The main code and the newly constructed 3DMM basis is publicly available.

中文翻译：

---

从 RGB-D Selfies 创建高保真 3D 数字人头

我们展示了一个全自动系统，该系统可以使用消费级 RGB-D 自拍相机制作高保真、逼真的三维 (3D) 数字人头。该系统只需要用户在旋转头部的同时拍摄一小段自拍RGB-D视频，即可在不到30秒的时间内产生高质量的头部重建。我们的主要贡献是一种新的面部几何建模和反射率合成程序，它显着提高了现有技术。具体来说，给定输入视频，首先采用两阶段帧选择过程来选择一些高质量的帧进行重建。然后应用基于可微渲染器的 3D 可变形模型 (3DMM) 拟合算法从多视图 RGB-D 数据中恢复面部几何形状，它利用了由大量数据生成和扰动构建的强大 3DMM 基础。我们的 3DMM 比传统的 3DMM 具有更大的表达能力，使我们能够仅使用线性基础来恢复更准确的面部几何形状。对于反射率合成，我们提出了一种混合方法，它结合了参数拟合和卷积神经网络 (CNN)用逼真的头发/毛孔/皱纹细节合成高分辨率反照率/法线贴图。结果表明，我们的系统可以生成具有极其逼真细节的忠实 3D 数字人脸。主要代码和新构建的 3DMM 基础是公开的。