当前位置: X-MOL 学术IEEE Trans. Inform. Forensics Secur. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Deep Collaborative Multi-Modal Learning for Unsupervised Kinship Estimation
IEEE Transactions on Information Forensics and Security ( IF 6.8 ) Pub Date : 2021-07-19 , DOI: 10.1109/tifs.2021.3098165
Guan-Nan Dong , Chi-Man Pun , Zheng Zhang

Kinship verification is a long-standing research challenge in computer vision. The visual differences presented to the face have a significant effect on the recognition capabilities of the kinship systems. We argue that aggregating multiple visual knowledge can better describe the characteristics of the subject for precise kinship identification. Typically, the age-invariant features can represent more natural facial details. Such age-related transformations are essential for face recognition due to the biological effects of aging. However, the existing methods mainly focus on employing the single-view image features for kinship identification, while more meaningful visual properties such as race and age are directly ignored in the feature learning step. To this end, we propose a novel deep collaborative multi-modal learning (DCML) to integrate the underlying information presented in facial properties in an adaptive manner to strengthen the facial details for effective unsupervised kinship verification. Specifically, we construct a well-designed adaptive feature fusion mechanism, which can jointly leverage the complementary properties from different visual perspectives to produce composite features and draw greater attention to the most informative components of spatial feature maps. Particularly, an adaptive weighting strategy is developed based on a novel attention mechanism, which can enhance the dependencies between different properties by decreasing the information redundancy in channels in a self-adaptive manner. Moreover, we propose to use self-supervised learning to further explore the intrinsic semantics embedded in raw data and enrich the diversity of samples. As such, we could further improve the representation capabilities of kinship feature learning and mitigate the multiple variations from original visual images. To validate the effectiveness of the proposed method, extensive experimental evaluations conducted on four widely-used datasets show that our DCML method is always superior to some state-of-the-art kinship verification methods.

中文翻译:

用于无监督亲属估计的深度协作多模态学习

亲属关系验证是计算机视觉领域长期存在的研究挑战。呈现在面部的视觉差异对亲属关系系统的识别能力有显着影响。我们认为聚合多个视觉知识可以更好地描述对象的特征以进行精确的亲属关系识别。通常,年龄不变的特征可以代表更自然的面部细节。由于衰老的生物学效应,这种与年龄相关的转换对于人脸识别至关重要。然而,现有的方法主要集中在使用单视图图像特征进行亲属识别,而在特征学习步骤中直接忽略了更有意义的视觉属性,如种族和年龄。为此,我们提出了一种新颖的深度协作多模态学习(DCML),以自适应方式整合面部特征中呈现的基础信息,以加强面部细节,以进行有效的无监督亲属关系验证。具体来说,我们构建了一个精心设计的自适应特征融合机制,它可以联合利用不同视觉角度的互补特性来产生复合特征,并更多地关注空间特征图中信息量最大的部分。特别是,基于新的注意力机制开发了一种自适应加权策略,该策略可以通过以自适应方式减少通道中的信息冗余来增强不同属性之间的依赖性。而且,我们建议使用自监督学习来进一步探索嵌入在原始数据中的内在语义并丰富样本的多样性。因此,我们可以进一步提高亲属特征学习的表示能力,并减轻原始视觉图像的多重变化。为了验证所提出方法的有效性,对四个广泛使用的数据集进行的大量实验评估表明,我们的 DCML 方法始终优于一些最先进的亲属关系验证方法。
更新日期:2021-08-31
down
wechat
bug