Face alignment and cross-modal face transfer are two important tasks for automatic face analysis in computer vision. Over the years, they have been extensively studied. Recently, deep neural networks have attracted much research attention for both face alignment and face transfer. With the prevalence of the consumer depth sensor, depth-based face alignment and cross-modal (image and depth) are increasingly important. Different from existing RGB- image based tasks, the main challenge of depth-based tasks is the lack of annotated data. To address the challenge, we observe that these two tasks are closely related and their learning processes may benefit each other. This paper develops a joint multi-task learning algorithm for both depth-based face alignment and face transfer using the deep convolutional neural network (CNN). The proposed approach allows the CNN model to simultaneously share visual knowledge and information between two tasks. We use a dataset of 10,000 face depth images for validation. Our experiments show that the proposed approach outperforms state-of-the-art algorithms. The results also show that learning these two related tasks simultaneously improves the performance of each individual task.

面部对齐和跨模式面部转移是计算机视觉中自动面部分析的两个重要任务。多年来，已经对其进行了广泛的研究。近年来，深度神经网络在面部对齐和面部转移方面都吸引了很多研究关注。随着消费者深度传感器的普及，基于深度的面部对齐和交叉模式（图像和深度）变得越来越重要。与现有的基于RGB图像的任务不同，基于深度的任务的主要挑战是缺少带注释的数据。为了应对这一挑战，我们注意到这两个任务密切相关，它们的学习过程可能会互利。本文使用深度卷积神经网络（CNN）开发了一种基于深度的人脸对齐和人脸转移的联合多任务学习算法。所提出的方法允许CNN模型在两个任务之间同时共享视觉知识和信息。我们使用10,000个面部深度图像的数据集进行验证。我们的实验表明，提出的方法优于最新的算法。结果还表明，学习这两个相关任务可以同时提高每个任务的性能。