Sarcasm is a sophisticated linguistic phenomenon and commonly manifests on social media platforms, which poses a great challenge for opinion mining systems. Therefore, multimodal sarcasm detection, which aims to understand the implied sentiment in the video, has gained more and more attention. However, previous works mostly focus on multimodal feature fusion without explicitly modeling the incongruity between modalities, such as expressing verbal compliments while rolling eyes, which is an obvious cue for detecting sarcasm. In this article, we propose the incongruity-aware attention network (IWAN), which detects sarcasm by focusing on the word-level incongruity between modalities via a scoring mechanism. This scoring mechanism could assign larger weights to words with incongruent modalities. Experimental results demonstrate the effectiveness of our proposed IWAN model, which not only achieves the state-of-the-art performance on the MUStARD dataset but also offers the advantages of interpretability.

中文翻译：

---

多模态讽刺检测模态之间的建模不一致

讽刺是一种复杂的语言现象，通常出现在社交媒体平台上，这对意见挖掘系统提出了巨大挑战。因此，旨在理解视频中隐含情感的多模态讽刺检测越来越受到关注。然而，以前的工作主要集中在多模态特征融合，而没有明确地对模态之间的不协调进行建模，例如在翻白眼的同时表达口头赞美，这是检测讽刺的明显线索。在本文中，我们提出了不一致感知注意力网络（IWAN），它通过评分机制关注模态之间的词级不一致来检测讽刺。这种评分机制可以为形式不一致的单词分配更大的权重。