用知识增强与情感引导提升多模态假新闻检测准确率
KEN: Knowledge Augmentation and Emotion Guidance Network for Multimodal Fake News Detection
- 结合大视觉语言模型生成图文描述，突破文本信息局限
- 通过情感类型差异化建模，提升真假新闻区分能力
- 在两个真实数据集上表现更优，适合社交媒体内容安全场景
近年来，社交媒体上虚假信息的泛滥使多模态假新闻检测成为关键研究方向。然而，以往研究对图像语义理解不足，且模型在文本信息有限时难以判断新闻真伪。同时，对各类情感新闻采用统一处理方式，导致性能下降。为此，我们提出一种新型知识增强与情感引导网络（KEN）。一方面，利用大视觉语言模型（LVLM）强大的语义理解能力和广泛的世界知识：对图像生成描述，全面理解图像内容与场景；对文本检索证据，打破封闭有限的文本与上下文信息壁垒。另一方面，通过均衡学习考虑不同情感类型新闻的类别差异，实现情感类型与真实性关系的细粒度建模。在两个真实数据集上的大量实验表明，KEN显著优于现有方法。
原文摘要 · Abstract (English)
In recent years, the rampant spread of misinformation on social media has made accurate detection of multimodal fake news a critical research focus. However, previous research has not adequately understood the semantics of images, and models struggle to discern news authenticity with limited textual information. Meanwhile, treating all emotional types of news uniformly without tailored approaches further leads to performance degradation. Therefore, we propose a novel Knowledge Augmentation and Emotion Guidance Network (KEN). On the one hand, we effectively leverage LVLM's powerful semantic understanding and extensive world knowledge. For images, the generated captions provide a comprehensive understanding of image content and scenes, while for text, the retrieved evidence helps break the information silos caused by the closed and limited text and context. On the other hand, we consider inter-class differences between different emotional types of news through balanced learning, achieving fine-grained modeling of the relationship between emotional types and authenticity. Extensive experiments on two real-world datasets demonstrate the superiority of our KEN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。