Face recognition of realistic visual images (e.g., photos) has been well studied and made significant progress in the recent decade. However, face recognition between realistic visual images/photos and caricatures is still a challenging problem. Unlike the photos, the different artistic styles of caricatures introduce extreme non-rigid distortions of caricatures. The great representational gap between the different modalities of photos and caricatures is a big challenge for photo-caricature face recognition. In this paper, we propose to conduct cross-modal photo-caricature face recognition via multi-task learning, which can learn the features of different modalities with different tasks. Instead of manually setting the task weights as in conventional multi-task learning, this work proposes a dynamic weights learning module which can automatically generate/learn task weights according to the training importance of tasks. The learned task weights enable the network to focus on training the hard tasks instead of being stuck in the overtraining of easy tasks. The experimental results demonstrate the effectiveness of the proposed dynamic multi-task learning for cross-modal photo-caricature face recognition. The performance on the datasets CaVI and WebCaricature show the superiority over the state-of-art methods. The implementation code is provided here. (https://github.com/hengxyz/cari-visual-recognition-via-multitask-learning.git).

人脸识别逼真的视觉图像（例如，照片）已经过充分研究，并且在最近十年中取得了重大进展。然而，现实视觉图像/照片与漫画之间的面部识别仍然是一个具有挑战性的问题。与照片不同，讽刺漫画的不同艺术风格引入了极端非刚性的讽刺漫画变形。在照片和漫画的不同形式之间存在巨大的代表性差距，这对照片漫画的人脸识别是一个巨大的挑战。在本文中，我们建议通过多任务学习进行跨模式的漫画面部识别，以学习不同任务的不同模态的特征。不同于传统的多任务学习中的手动设置任务权重，这项工作提出了一个动态权重学习模块，该模块可以根据任务的培训重要性自动生成/学习任务权重。学习到的任务权重使网络可以专注于训练艰巨的任务，而不会陷入对简单任务的过度训练中。实验结果证明了所提出的动态多任务学习对于交叉模式的漫画面部识别的有效性。CaVI和WebCaricature数据集的性能显示出优于最新方法的优越性。此处提供了实现代码。（https://github.com/hengxyz/cari-visual-recognition-via-multitask-learning.git）。实验结果证明了所提出的动态多任务学习对于交叉模式的漫画面部识别的有效性。CaVI和WebCaricature数据集的性能显示出优于最新方法的优越性。此处提供了实现代码。（https://github.com/hengxyz/cari-visual-recognition-via-multitask-learning.git）。实验结果证明了所提出的动态多任务学习对于交叉模式的漫画面部识别的有效性。CaVI和WebCaricature数据集的性能显示出优于最新方法的优越性。此处提供了实现代码。（https://github.com/hengxyz/cari-visual-recognition-via-multitask-learning.git）。