Recommendation system is facing the “data sparseness” issue. Additional information, including images, texts, and videos, contributes to alleviating this issue. We propose a new multi-modal visual adversarial Bayesian personalized ranking (MVABPR) model to address the issue. The proposed model takes new features, cross-modal semantics, adversarial learning, and visual interface into account. Two multi-modal datasets are created based on the MovieLens datasets and the correlated images. Besides the shape, texture, color, and deep learning-based features, a set of efficient match kernel features are proposed. More discriminative but low-dimensional cross-modal semantics among these features is mined to characterize each item effectively, which is absorbed into the MVABPR model through a visual interface. A new adversarial learning strategy is employed to optimize the whole training procedure. This makes the MVABPR model more robust and stable. Experimental results demonstrate that the MVABPR model is effective and robust for recommendation. It outperforms other competitive baselines. As another advantage, it can learn visual information and users’ rating jointly, effectively, combined with adversarial learning. And the implicit feeling tone of a recommended item can be accurately captured by the proposed model. More importantly, the model achieves better performance on a large-scale sparser dataset, demonstrating its higher practicality.

推荐系统面临着“数据稀疏”的问题。附加信息，包括图像、文本和视频，有助于缓解此问题。我们提出了一种新的多模态视觉对抗性贝叶斯个性化排名（MVABPR）模型来解决这个问题。所提出的模型考虑了新特征、跨模态语义、对抗性学习和视觉界面。基于MovieLens数据集和相关图像创建了两个多模态数据集。除了形状、纹理、颜色和基于深度学习的特征外，还提出了一组有效的匹配核特征。挖掘这些特征中更具辨别力但低维的跨模态语义，以有效地表征每个项目，并通过视觉界面将其吸收到 MVABPR 模型中。一种新的对抗学习策略被用来优化整个训练过程。这使得 MVABPR 模型更加健壮和稳定。实验结果表明，MVABPR 模型对于推荐是有效且稳健的。它优于其他竞争基准。作为另一个优点，它可以结合对抗性学习，有效地联合学习视觉信息和用户的评分。并且所提出的模型可以准确地捕获推荐项目的隐含感觉基调。更重要的是，该模型在大规模稀疏数据集上取得了更好的性能，展示了其更高的实用性。它优于其他竞争基准。作为另一个优点，它可以结合对抗性学习，有效地联合学习视觉信息和用户的评分。并且所提出的模型可以准确地捕获推荐项目的隐含感觉基调。更重要的是，该模型在大规模稀疏数据集上取得了更好的性能，展示了其更高的实用性。它优于其他竞争基准。作为另一个优点，它可以结合对抗性学习，有效地联合学习视觉信息和用户的评分。所建议的模型可以准确地捕获推荐项目的内在感觉音调。更重要的是，该模型在大规模稀疏数据集上取得了更好的性能，展示了其更高的实用性。