Face image manipulation via three-dimensional guidance has been widely applied in various interactive scenarios due to its semantically-meaningful understanding and user-friendly controllability. However, existing 3D-morphable-model-based manipulation methods are not directly applicable to out-of-domain faces, such as non-photorealistic paintings, cartoon portraits, or even animals, mainly due to the formidable difficulties in building the model for each specific face domain. To overcome this challenge, we propose, as far as we know, the first method to manipulate faces in arbitrary domains using human 3DMM. This is achieved through two major steps: 1) disentangled mapping from 3DMM parameters to the latent space embedding of a pre-trained StyleGAN2 that guarantees disentangled and precise controls for each semantic attribute; and 2) cross-domain adaptation that bridges domain discrepancies and makes human 3DMM applicable to out-of-domain faces by enforcing a consistent latent space embedding. Experiments and comparisons demonstrate the superiority of our high-quality semantic manipulation method on a variety of face domains with all major 3D facial attributes controllable: pose, expression, shape, albedo, and illumination. Moreover, we develop an intuitive editing interface to support user-friendly control and instant feedback. Our project page is https://cassiepython.github.io/sigasia/cddfm3d.html.

中文翻译：

---

具有3D引导的跨域和解缠结的面部操纵

由于具有语义意义的理解和用户友好的可控性，通过三维导航的面部图像操纵已广泛应用于各种交互式场景中。但是，现有的基于3D变形模型的操作方法无法直接应用于域外的面孔，例如非真实感绘画，卡通肖像甚至动物，这主要是因为在为每个模型建立模型时都遇到了巨大的困难特定的面孔域。为了克服这一挑战，据我们所知，我们提出了使用人类3DMM在任意域中操纵人脸的第一种方法。这可以通过两个主要步骤来实现：1）从3DMM参数到对预训练的StyleGAN2的潜在空间嵌入的纠缠映射，以确保对每个语义属性的纠缠和精确控制；和2）跨域自适应，可通过强制执行一致的潜在空间嵌入来弥合域差异并使人3DMM适用于域外面孔。实验和比较表明，我们的高质量语义处理方法在各种可控制所有主要3D面部属性（姿势，表情，形状，反照率和照明度）的面部区域上的优越性。此外，我们开发了一个直观的编辑界面，以支持用户友好的控制和即时反馈。我们的项目页面是https://cassiepython.github.io/sigasia/cddfm3d.html。实验和比较表明，我们的高质量语义处理方法在各种可控制所有主要3D面部属性（姿势，表情，形状，反照率和照明度）的面部区域上的优越性。此外，我们开发了一个直观的编辑界面，以支持用户友好的控制和即时反馈。我们的项目页面是https://cassiepython.github.io/sigasia/cddfm3d.html。实验和比较表明，我们的高质量语义处理方法在各种可控制所有主要3D面部属性（姿势，表情，形状，反照率和照明度）的面部区域上的优越性。此外，我们开发了一个直观的编辑界面，以支持用户友好的控制和即时反馈。我们的项目页面是https://cassiepython.github.io/sigasia/cddfm3d.html。