Cross-modal retrieval aims to retrieve the related items from various modalities with respect to a query from any type. The key challenge of cross-modal retrieval is to learn more discriminative representations between different category, as well as expand to an unseen class retrieval in the open world retrieval task. To tackle the above problem, in this paper, we propose a prototype learning-based discriminative feature learning (PLDFL) to learn more discriminative representations in a common space. First, we utilize a prototype learning algorithm to cluster these samples labeled with the same semantic class, by jointly taking into consideration the intra-class compactness and inter-class sparsity without discriminative treatments. Second, we use the weight-sharing strategy to model the correlations of cross-modal samples to narrow down the modality gap. Finally, we apply the prototype to achieve class-incremental learning to prove the robustness of our proposed approach. According to our experimental results, significant retrieval performance in terms of mAP can be achieved on average compared to several state-of-the-art approaches.

中文翻译：

---

用于类增量跨模态检索的基于原型的判别特征表示

跨模态检索旨在针对任何类型的查询从各种模态检索相关项目。跨模态检索的关键挑战是学习不同类别之间的更多区分表示，以及在开放世界检索任务中扩展到看不见的类别检索。为了解决上述问题，在本文中，我们提出了一种基于原型学习的判别特征学习（PLDFL），以在公共空间中学习更多的判别表示。首先，我们利用原型学习算法对标记有相同语义类的这些样本进行聚类，通过共同考虑类内紧凑性和类间稀疏性而无需进行区分处理。第二，我们使用权重共享策略对跨模态样本的相关性进行建模，以缩小模态差距。最后，我们应用原型来实现类增量学习，以证明我们提出的方法的鲁棒性。根据我们的实验结果，与几种最先进的方法相比，平均而言可以实现显着的 mAP 检索性能。