In this paper, we present iOrthoPredictor, a novel system to visually predict teeth alignment in photographs. Our system takes a frontal face image of a patient with visible malpositioned teeth along with a corresponding 3D teeth model as input, and generates a facial image with aligned teeth, simulating a real orthodontic treatment effect. The key enabler of our method is an effective disentanglement of an explicit representation of the teeth geometry from the in-mouth appearance, where the accuracy of teeth geometry transformation is ensured by the 3D teeth model while the in-mouth appearance is modeled as a latent variable. The disentanglement enables us to achieve fine-scale geometry control over the alignment while retaining the original teeth appearance attributes and lighting conditions. The whole pipeline consists of three deep neural networks: a U-Net architecture to explicitly extract the 2D teeth silhouette maps representing the teeth geometry in the input photo, a novel multilayer perceptron (MLP) based network to predict the aligned 3D teeth model, and an encoder-decoder based generative model to synthesize the in-mouth appearance conditional on the original teeth appearance and the aligned teeth geometry. Extensive experimental results and a user study demonstrate that iOrthoPredictor is effective in qualitatively predicting teeth alignment, and applicable to the orthodontic industry.

中文翻译：

---

iOrthoPredictor

在本文中，我们介绍了 iOrthoPredictor，这是一种新颖的系统，可以在照片中直观地预测牙齿排列。我们的系统以可见牙齿错位的患者的正面图像以及相应的 3D 牙齿模型作为输入，生成牙齿对齐的面部图像，模拟真实的正畸治疗效果。我们方法的关键推动因素是将牙齿几何形状的显式表示与口腔外观有效分离，其中牙齿几何变换的准确性由 3D 牙齿模型确保，而口腔外观被建模为潜在多变的。解开使我们能够在保持原始牙齿外观属性和光照条件的同时实现对对齐的精细几何控制。整个流程由三个深度神经网络组成：一个 U-Net 架构，用于显式提取表示输入照片中牙齿几何形状的 2D 牙齿轮廓图，一个基于多层感知器 (MLP) 的新型网络，用于预测对齐的 3D 牙齿模型，以及一种基于编码器-解码器的生成模型，用于根据原始牙齿外观和对齐的牙齿几何形状合成口腔外观。大量的实验结果和用户研究表明，iOrthoPredictor 在定性预测牙齿排列方面是有效的，适用于正畸行业。以及基于编码器-解码器的生成模型，以根据原始牙齿外观和对齐的牙齿几何形状合成口腔外观。大量的实验结果和用户研究表明，iOrthoPredictor 在定性预测牙齿排列方面是有效的，适用于正畸行业。以及基于编码器-解码器的生成模型，以根据原始牙齿外观和对齐的牙齿几何形状合成口腔外观。大量的实验结果和用户研究表明，iOrthoPredictor 在定性预测牙齿排列方面是有效的，适用于正畸行业。