We present Face Swapping GAN (FSGAN) for face swapping and reenactment. Unlike previous work, we offer a subject agnostic swapping scheme that can be applied to pairs of faces without requiring training on those faces. We derive a novel iterative deep learning–based approach for face reenactment which adjusts significant pose and expression variations that can be applied to a single image or a video sequence. For video sequences, we introduce a continuous interpolation of the face views based on reenactment, Delaunay Triangulation, and barycentric coordinates. Occluded face regions are handled by a face completion network. Finally, we use a face blending network for seamless blending of the two faces while preserving the target skin color and lighting conditions. This network uses a novel Poisson blending loss combining Poisson optimization with a perceptual loss. We compare our approach to existing state-of-the-art systems and show our results to be both qualitatively and quantitatively superior. This work describes extensions of the FSGAN method, proposed in an earlier conference version of our work (Nirkin et al. 2019), as well as additional experiments and results.

中文翻译：

---

FSGANv2：改进的主题不可知人脸交换和重演

我们提出了用于面部交换和重演的面部交换 GAN (FSGAN)。与以前的工作不同，我们提供了一种主题不可知的交换方案，可以应用于成对的面孔，而不需要对这些面孔进行训练。我们推导出一种新颖的基于迭代深度学习的面部重现方法，该方法可调整可应用于单个图像或视频序列的重要姿势和表情变化。对于视频序列，我们引入了基于重演、Delaunay 三角剖分和重心坐标的面部视图的连续插值。被遮挡的面部区域由面部补全网络处理。最后，我们使用人脸混合网络对两个人脸进行无缝混合，同时保留目标肤色和光照条件。该网络使用了一种新颖的泊松混合损失，将泊松优化与感知损失相结合。我们将我们的方法与现有的最先进系统进行比较，并表明我们的结果在质量和数量上都更优越。这项工作描述了 FSGAN 方法的扩展，在我们工作的早期会议版本中提出（Nirkin等。2019），以及额外的实验和结果。