融合文本、图像与知识图谱，提升虚假新闻检测精度。
KITE: A Tri-Modal Transformer Integrating Text, Images, and Knowledge Graphs for Fake News Detection
- 三模态联合建模：文本、图像与维基数据知识协同分析。
- 在图文不一致场景下，准确率显著超越传统方法。
- 输出各模态置信度，可解释判断依据，适合安全审核场景。
随着多模态虚假信息日益复杂，传统检测方法难以应对融合伪造文本、篡改图像和错误事实的新型骗局。现有工作多聚焦文本-图像融合或将外部知识作为后期处理，无法深入捕捉语义矛盾。本文提出KITE（知识集成文本-图像编码器），一种三模态虚假新闻检测框架，同时建模文本、视觉与结构化知识表示。KITE采用Roberta与CLIP分别编码语言与图像特征，利用图注意力网络（GAT）处理从Wikidata检索的事实信息，并通过多模态Transformer中的跨模态注意力机制实现三者深度融合。模型生成各模态专属置信度分数，辅助决策解释。在基准数据集上的评估表明，KITE显著优于单模态与双模态基线，在图像-文本不匹配或与外部知识冲突的场景中表现尤为突出。
原文摘要 · Abstract (English)
Traditional fake news detection methods are falling behind as multimodal misinformation grows more advanced, seamlessly blending deceptive text, manipulated visuals, and factually incorrect claims. Most prior work focuses on text-image fusion or applies external knowledge only as a post-processing step, limiting their ability to detect deeper semantic inconsistencies. In this paper, we introduce KITE (Knowledge-Integrated Text-Image Encoder), a tri-modal fake news detection framework that jointly models textual, visual, and factual knowledge representations. KITE leverages Roberta [23,14] and CLIP [24] for linguistic and visual encoding, while a Graph Attention Network (GAT) processes structured facts retrieved from Wikidata. KITE uses cross-modal attention [9] within a multimodal transformer to integrate text, visual, and knowledge features, helping it understand how each modality relates to one another. Modality-specific confidence scores are generated alongside the final prediction, offering interpretability by indicating which input type most influenced the decision. Evaluations on benchmark datasets demonstrate that KITE significantly outperforms unimodal and bimodal baselines, particularly in scenarios involving image-text mismatches or contradictions with external knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。