Standard registration algorithms need to be independently applied to each surface to register, following careful pre-processing and hand-tuning. Recently, learning-based approaches have emerged that reduce the registration of new scans to running inference with a previously-trained model. The potential benefits are multifold: inference is typically orders of magnitude faster than solving a new instance of a difficult optimization problem, deep learning models can be made robust to noise and corruption, and the trained model may be re-used for other tasks, e.g. through transfer learning. In this paper, we cast the registration task as a surface-to-surface translation problem, and design a model to reliably capture the latent geometric information directly from raw 3D face scans. We introduce Shape-My-Face (SMF), a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model that we smoothly integrate with the mesh convolutions. Compared to the previous state-of-the-art learning algorithms for non-rigid registration of face scans, SMF only requires the raw data to be rigidly aligned (with scaling) with a pre-defined face template. Additionally, our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets. We extensively evaluate the quality of our registrations on diverse data. We demonstrate the robustness and generalizability of our model with in-the-wild face scans across different modalities, sensor types, and resolutions. Finally, we show that, by learning to register scans, SMF produces a hybrid linear and non-linear morphable model. Manipulation of the latent space of SMF allows for shape generation, and morphing applications such as expression transfer in-the-wild. We train SMF on a dataset of human faces comprising 9 large-scale databases on commodity hardware.

标准配准算法需要独立应用于每个表面进行配准，经过仔细的预处理和手动调整。最近，出现了基于学习的方法，可以将新扫描的注册减少到使用先前训练过的模型运行推理。潜在的好处是多方面的：推理通常比解决困难优化问题的新实例快几个数量级，深度学习模型可以对噪声和损坏具有鲁棒性，并且经过训练的模型可以重新用于其他任务，例如通过迁移学习。在本文中，我们将配准任务作为一个表面到表面的转换问题，并设计了一个模型来直接从原始 3D 面部扫描中可靠地捕获潜在的几何信息。我们介绍形状我的脸（SMF），基于改进的点云编码器的强大编码器-解码器架构、新颖的视觉注意机制、具有跳跃连接的图形卷积解码器，以及我们与网格卷积平滑集成的专用嘴模型。与之前用于非刚性人脸扫描配准的最先进学习算法相比，SMF 只需要原始数据与预定义的人脸模板严格对齐（缩放）。此外，我们的模型以最少的监督提供拓扑合理的网格，提供更快的训练时间，可训练参数少几个数量级，对噪声更鲁棒，并且可以推广到以前看不见的数据集。我们根据不同的数据广泛评估我们的注册质量。我们通过不同模式、传感器类型和分辨率的野外人脸扫描证明了我们模型的稳健性和通用性。最后，我们表明，通过学习注册扫描，SMF 产生混合线性和非线性可变形模型。操纵 SMF 的潜在空间允许形状生成和变形应用程序，例如在野外表达转移。我们在人脸数据集上训练 SMF，该数据集包含 9 个商用硬件上的大型数据库。