Abstract In this paper, we propose a novel Deep Kinship Matching and Recognition (DKMR) framework for multi-person kinship matching and recognition, which is a complicated and challenging task with little previous literature. Compared with most existing kinship understanding methods that mainly work on matching kinship in pairwise face images, we target at recognizing the exact kinship in nuclear family photos consisting of multiple persons. The proposed DKMR framework contains three modules. Firstly, we design a deep kinship matching model (termed DKM-TRL) to predict kin-or-not scores by integrating the triple ranking loss into a Siamese CNN model. Secondly, we develop a deep kinship recognition model (named DKR-GA) to predict the exact kinship categories, in which gender and relative age attributes are utilized to learn more discriminative representations. Thirdly, based on the outputs of DKM-TRL and DKR-GA, we propose a reasoning conditional random field (R-CRF) model to infer the corresponding optimal family tree by exploiting the common kinship knowledge of a nuclear family. To evaluate the effectiveness of our DKMR framework, we conduct extensive experiments and the results show that it can gain superior performance on Group-Face dataset, TSKinFace dataset and FIW dataset over state-of-the-arts.

中文翻译：

---

家庭照片的深度多人亲属匹配和识别

摘要在本文中，我们提出了一种新颖的深度亲属匹配和识别（DKMR）框架，用于多人亲属匹配和识别，这是一项复杂且具有挑战性的任务，以前的文献很少。与大多数现有的主要用于匹配成对人脸图像中的亲属关系的亲属关系理解方法相比，我们的目标是识别由多人组成的核心家庭照片中的确切亲属关系。提议的 DKMR 框架包含三个模块。首先，我们设计了一个深度亲属匹配模型（称为 DKM-TRL），通过将三重排序损失集成到 Siamese CNN 模型中来预测亲属与否的分数。其次，我们开发了一个深度亲属关系识别模型（命名为 DKR-GA）来预测确切的亲属关系类别，其中利用性别和相对年龄属性来学习更具辨别力的表示。第三，基于DKM-TRL和DKR-GA的输出，我们提出了一个推理条件随机场（R-CRF）模型，通过利用核心家庭的共同亲属关系知识来推断相应的最优家谱。为了评估我们 DKMR 框架的有效性，我们进行了大量实验，结果表明它可以在 Group-Face 数据集、TSKinFace 数据集和 FIW 数据集上获得优于最先进技术的性能。