当前位置: X-MOL 学术J. Visual Commun. Image Represent. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Semi-supervised cross-modal representation learning with GAN-based Asymmetric Transfer Network
Journal of Visual Communication and Image Representation ( IF 2.6 ) Pub Date : 2020-09-15 , DOI: 10.1016/j.jvcir.2020.102899
Lei Zhang , Leiting Chen , Weihua Ou , Chuan Zhou

In this paper, we proposed a semi-supervised common representation learning method with GAN-based Asymmetric Transfer Network (GATN) for cross modality retrieval. GATN utilizes the asymmetric pipeline to guarantee the semantic consistency and adopt (Generative Adversarial Network) GAN to fit the distributions of different modalities. Specifically, the common representation learning across modalities includes two stages: (1) the first stage, GATN trains source mapping network to learn the semantic representation of text modality by supervised method; and (2) the second stage, GAN-based unsupervised modality transfer method is proposed to guide the training of target mapping network, which includes generative network (target mapping network) and discriminative network. Experimental results on three widely-used benchmarks show that GATN have achieved better performance comparing with several existing state-of-the-art methods.



中文翻译:

基于GAN的不对称转移网络的半监督交叉模态表示学习

在本文中,我们提出了一种基于GAN的非对称传输网络(GATN)的半监督通用表示学习方法,用于跨模态检索。GATN利用非对称流水线来保证语义一致性,并采用GAN(通用对抗网络)来适应不同形式的分布。具体来说,跨模式的通用表示学习包括两个阶段:(1)第一阶段,GATN训练源映射网络通过监督方法学习文本模式的语义表示;(2)第二阶段,提出了基于GAN的无监督模态传递方法,指导目标映射网络的训练,包括生成网络(目标映射网络)和判别网络。

更新日期:2020-11-04
down
wechat
bug