Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal correlation measurement, and others use adversarial learning technique to abate the heterogeneity of multimodal data. However, very few works combine correlation learning and adversarial learning to bridge the intermodal semantic gap and diminish cross-modal heterogeneity. This article proposes a novel cross-modal retrieval method, named Adversarial Learning based Semantic COrrelation Representation (ALSCOR), which is an end-to-end framework to integrate cross-modal representation learning, correlation learning, and adversarial. Canonical correlation analysis model, combined with VisNet and TxtNet, is proposed to capture cross-modal nonlinear correlation. Besides, intramodal classifier and modality classifier are used to learn intramodal discrimination and minimize the intermodal heterogeneity. Comprehensive experiments are conducted on three benchmark datasets. The results demonstrate that the proposed ALSCOR has better performance than the state of the arts.

中文翻译：

---

基于对抗学习的语义相关表示形式用于跨模态检索

跨模式检索已成为近年来的热门问题。现有的许多工作都关注于相关学习，以生成用于跨模态相关性测量的公共子空间，而其他一些则使用对抗性学习技术来减轻多模态数据的异质性。然而，很少有作品将相关性学习和对抗性学习相结合来弥合跨模态语义鸿沟并减少跨模态异质性。本文提出了一种新颖的跨模式检索方法，称为基于对抗学习的语义相关表示（ALSCOR），它是一个集成了跨模式表示学习，相关学习和对抗的端到端框架。提出了典型的相关分析模型，结合VisNet和TxtNet，以捕获跨模态非线性相关。除了，模态内分类器和模态分类器用于学习模态内歧视并最小化联运异质性。在三个基准数据集上进行了全面的实验。结果表明，提出的ALSCOR具有比现有技术更好的性能。