This paper presents a novel framework, namely Deep Cross-modality Spectral Hashing (DCSH) , to tackle the unsupervised learning problem of binary hash codes for efficient cross-modal retrieval. The framework is a two-step hashing approach which decouples the optimization into (1) binary optimization and (2) hashing function learning. In the first step, we propose a novel spectral embedding-based algorithm to simultaneously learn single-modality and binary cross-modality representations. While the former is capable of well preserving the local structure of each modality, the latter reveals the hidden patterns from all modalities. In the second step, to learn mapping functions from informative data inputs (images and word embeddings) to binary codes obtained from the first step, we leverage the powerful CNN for images and propose a CNN-based deep architecture to learn text modality. Quantitative evaluations on three standard benchmark datasets demonstrate that the proposed DCSH method consistently outperforms other state-of-the-art methods.

中文翻译：

---

无监督深度跨模态频谱哈希。


本文提出了一个新颖的框架，即深度跨模态频谱哈希 (DCSH) ，解决二进制哈希码的无监督学习问题，以实现高效的跨模式检索。该框架是一种两步哈希方法，它将优化分解为 (1) 二进制优化和 (2) 哈希函数学习。第一步，我们提出了一种新颖的基于谱嵌入的算法来同时学习单模态和二进制跨模态表示。前者能够很好地保留每种模态的局部结构，而后者则揭示了所有模态的隐藏模式。第二步，为了学习从信息数据输入（图像和词嵌入）到第一步获得的二进制代码的映射函数，我们利用强大的图像 CNN，并提出一种基于 CNN 的深度架构来学习文本模态。对三个标准基准数据集的定量评估表明，所提出的 DCSH 方法始终优于其他最先进的方法。