During the outbreak of the new Coronavirus (2019-nCoV) in 2020, the spread of fake news has caused serious social panic. Fake news often uses multimedia information such as text and image to mislead readers, spreading and expanding its influence. One of the most important problems in fake news detection based on multimodal data is to extract the general features as well as to fuse the intrinsic characteristics of the fake news, such as mismatch of image and text and image tampering. This paper proposes a Multimodal Consistency Neural Network (MCNN) that considers the consistency of multimodal data and captures the overall characteristics of social media information. Our method consists of five subnetworks: the text feature extraction module, the visual semantic feature extraction module, the visual tampering feature extraction module, the similarity measurement module, and the multimodal fusion module. The text feature extraction module and the visual semantic feature extraction module are responsible for extracting the semantic features of text and vision and mapping them to the same space for a common representation of cross-modal features. The visual tampering feature extraction module is responsible for extracting visual physical and tamper features. The similarity measurement module can directly measure the similarity of multimodal data for the problem of mismatching of image and text. We assess the constructed method in terms of four datasets commonly used for fake news detection. The accuracy of the detection is improved clearly compared to the best available methods.

2020年新型冠状病毒（2019-nCoV）爆发期间，假新闻的传播引发了严重的社会恐慌。假新闻往往利用文字、图片等多媒体信息误导读者，传播扩大影响。基于多模态数据的假新闻检测最重要的问题之一是提取一般特征以及融合假新闻的内在特征，例如图文不匹配和图像篡改。本文提出了一种多模态一致性神经网络（MCNN），它考虑了多模态数据的一致性并捕获了社交媒体信息的整体特征。我们的方法由五个子网络组成：文本特征提取模块，视觉语义特征提取模块，视觉篡改特征提取模块，相似度测量模块和多模态融合模块。文本特征提取模块和视觉语义特征提取模块负责提取文本和视觉的语义特征，并将它们映射到同一空间，用于跨模态特征的共同表示。视觉篡改特征提取模块负责提取视觉物理和篡改特征。相似度度量模块可以针对图文不匹配的问题直接度量多模态数据的相似度。我们根据四个常用于假新闻检测的数据集来评估构建的方法。与现有的最佳方法相比，检测的准确性明显提高。文本特征提取模块和视觉语义特征提取模块负责提取文本和视觉的语义特征，并将它们映射到同一空间，用于跨模态特征的共同表示。视觉篡改特征提取模块负责提取视觉物理和篡改特征。相似度度量模块可以针对图文不匹配的问题直接度量多模态数据的相似度。我们根据四个常用于假新闻检测的数据集来评估构建的方法。与现有的最佳方法相比，检测的准确性明显提高。文本特征提取模块和视觉语义特征提取模块负责提取文本和视觉的语义特征，并将它们映射到同一空间，用于跨模态特征的共同表示。视觉篡改特征提取模块负责提取视觉物理和篡改特征。相似度度量模块可以针对图文不匹配的问题直接度量多模态数据的相似度。我们根据四个常用于假新闻检测的数据集来评估构建的方法。与现有的最佳方法相比，检测的准确性明显提高。