论文标题
多模式匹配感知的共同注意网络，具有相互知识蒸馏以进行虚假新闻检测
Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection
论文作者
论文摘要
假新闻通常涉及多媒体信息，例如文本和图像，以误导读者，从而扩大和扩大其影响力。大多数现有的伪造新闻检测方法将共同注意机制应用于融合多模式的特征，同时忽略了图像和文本的一致性。在本文中，我们提出了具有相互知识蒸馏的多模式匹配感知的共同注意网络，以改善假新闻检测。具体而言，我们设计了一个图像文本匹配感知的共同注意机制，该机制捕获图像和文本的对齐方式，以获得更好的多模态融合。可以通过视觉语言预训练模型获得图像文本匹配表示形式。此外，基于设计的图像文本匹配感知的共同注意机制，我们建议分别建立两个共同注意网络，分别以文本和图像为中心，以进行相互知识蒸馏以改善伪造新闻检测。在三个基准数据集上进行的广泛实验表明，我们提出的模型在多模式假新闻检测方面实现了最先进的性能。
Fake news often involves multimedia information such as text and image to mislead readers, proliferating and expanding its influence. Most existing fake news detection methods apply the co-attention mechanism to fuse multimodal features while ignoring the consistency of image and text in co-attention. In this paper, we propose multimodal matching-aware co-attention networks with mutual knowledge distillation for improving fake news detection. Specifically, we design an image-text matching-aware co-attention mechanism which captures the alignment of image and text for better multimodal fusion. The image-text matching representation can be obtained via a vision-language pre-trained model. Additionally, based on the designed image-text matching-aware co-attention mechanism, we propose to build two co-attention networks respectively centered on text and image for mutual knowledge distillation to improve fake news detection. Extensive experiments on three benchmark datasets demonstrate that our proposed model achieves state-of-the-art performance on multimodal fake news detection.
