Signal Processing: Image Communication ( IF 3.4 ) Pub Date : 2021-01-05 , DOI: 10.1016/j.image.2020.116131 Xitao Zou , Xinzhi Wang , Erwin M. Bakker , Song Wu
Due to the storage and retrieval efficiency of hashing, as well as the highly discriminative feature extraction by deep neural networks, deep cross-modal hashing retrieval has been attracting increasing attention in recent years. However, most of existing deep cross-modal hashing methods simply employ single-label to directly measure the semantic relevance across different modalities, but neglect the potential contributions from multiple category labels. With the aim to improve the accuracy of cross-modal hashing retrieval by fully exploring the semantic relevance based on multiple labels of training data, in this paper, we propose a multi-label semantics preserving based deep cross-modal hashing (MLSPH) method. MLSPH firstly utilizes multi-labels of instances to calculate semantic similarity of the original data. Subsequently, a memory bank mechanism is introduced to preserve the multiple labels semantic similarity constraints and enforce the distinctiveness of learned hash representations over the whole training batch. Extensive experiments on several benchmark datasets reveal that the proposed MLSPH surpasses prominent baselines and reaches the state-of-the-art performance in the field of cross-modal hashing retrieval. Code is available at: https://github.com/SWU-CS-MediaLab/MLSPH.
中文翻译:
基于多标签语义保留的深度跨模态哈希
由于哈希的存储和检索效率以及深度神经网络对特征的高度区分性,近年来,深度交叉模式哈希检索已引起越来越多的关注。但是,大多数现有的深层交叉模式散列方法仅采用单标签直接测量跨不同模式的语义相关性,而忽略了多个类别标签的潜在影响。为了充分利用基于训练数据的多个标签的语义相关性,提高跨模式哈希检索的准确性,本文提出了一种基于多标签语义保留的深度跨模式哈希(MLSPH)方法。MLSPH首先利用实例的多个标签来计算原始数据的语义相似度。后来,引入了一种存储库机制来保留多个标签的语义相似性约束,并在整个训练批次中强制所学习的哈希表示形式的独特性。在几个基准数据集上进行的广泛实验表明,所提出的MLSPH超越了显着的基线,并在跨模态哈希检索领域达到了最先进的性能。可以从以下网址获得代码:https://github.com/SWU-CS-MediaLab/MLSPH。