Cross-modal retrieval has recently attracted much interest along with the rapid development of multimodal data, and effectively utilizing the complementary relationship of different modal data and eliminating the heterogeneous gap as much as possible are the two key challenges. In this paper, we present a novel network model termed cross-modal Dual Subspace learning with Adversarial Network (DSAN). The main contributions are as follows: (1) Dual subspaces (visual subspace and textual subspace) are proposed, which can better mine the underlying structure information of different modalities as well as modality-specific information. (2) An improved quadruplet loss is proposed, which takes into account the relative distance and absolute distance between positive and negative samples, together with the introduction of the idea of hard sample mining. (3) Intra-modal constrained loss is proposed to maximize the distance of the most similar cross-modal negative samples and their corresponding cross-modal positive samples. In particular, feature preserving and modality classification act as two antagonists. DSAN tries to narrow the heterogeneous gap between different modalities, and distinguish the original modality of random samples in dual subspaces. Comprehensive experimental results demonstrate that, DSAN significantly outperforms 9 state-of-the-art methods on four cross-modal datasets.

中文翻译：

---

具有对抗网络的跨模式双重子空间学习。

随着多模式数据的快速发展，跨模式检索最近引起了人们的极大兴趣，而有效利用不同模式数据的互补关系并尽可能消除异构缺口是两个关键挑战。在本文中，我们提出了一种新型的网络模型，称为带有对抗网络的交叉模式双子空间学习（DSAN）。主要贡献如下：（1）提出了双重子空间（视觉子空间和文本子空间），它可以更好地挖掘不同模态的基础结构信息以及特定于模态的信息。（2）提出了一种改进的四元组损失，该方法考虑了正样本和负样本之间的相对距离和绝对距离，并引入了硬样本挖掘的思想。（3）提出了模态内约束损失，以使最相似的交叉模态负样本和它们对应的交叉模态正样本的距离最大化。特别地，特征保留和形态分类是两个对立物。DSAN试图缩小不同模态之间的异构间隙，并区分双子空间中随机样本的原始模态。全面的实验结果表明，DSAN在四个交叉模式数据集上明显优于9种最新方法。DSAN试图缩小不同模态之间的异构差距，并区分双子空间中随机样本的原始模态。全面的实验结果表明，DSAN在四个交叉模式数据集上明显优于9种最新方法。DSAN试图缩小不同模态之间的异构差距，并区分双子空间中随机样本的原始模态。全面的实验结果表明，在四个交叉模式数据集上，DSAN的性能明显优于9种最新方法。