Extracting useful knowledge from multimodal data is the core of many multimedia applications, such as recommendation systems, and cross-modal retrieval. In this paper, we propose a label-based multimodal topic (LB-MMT) model to jointly model text and image data tagged with multiple labels. Specifically, we use the labels as supervised information to generate the text and image data. In the LB-MMT model, we assume that the textual words and visual words related to each text and image are drawn from a mixture of latent topics, where each topic is represented as a group of textual words and visual words. Moreover, we introduce multiple topics for each label, to build the top-down relationship from label to text and image. To investigate the effectiveness of the proposed approach, we conduct extensive experiments on a real-world multimodal dataset with labels. The results show the proposed approach obtains superior performances on topic coherence and label prediction compared with previous competitors. In addition, we show that our model yields interesting insights about multimodal topics. The proposed model provides important practical implications, e.g., designing more attractive multimodal contents for marketers.

从多模态数据中提取有用的知识是许多多媒体应用的核心，例如推荐系统和跨模态检索。在本文中，我们提出了一种基于标签的多模态主题（LB-MMT）模型来联合建模带有多个标签的文本和图像数据。具体来说，我们使用标签作为监督信息来生成文本和图像数据。在 LB-MMT 模型中，我们假设与每个文本和图像相关的文本词和视觉词是从潜在主题的混合中提取的，其中每个主题都表示为一组文本词和视觉词。此外，我们为每个标签引入多个主题，以建立从标签到文本和图像的自上而下的关系。为了调查所提出方法的有效性，我们对带有标签的真实多模态数据集进行了广泛的实验。结果表明，与以前的竞争对手相比，所提出的方法在主题连贯性和标签预测方面获得了优越的性能。此外，我们表明我们的模型产生了关于多模式主题的有趣见解。所提出的模型提供了重要的实际意义，例如，为营销人员设计更具吸引力的多模式内容。