用可解释的概念图推理，识别多模态虚假信息
Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection
- 构建人类可理解的概念图，融合MLLM发现的高阶概念
- 通过分层注意力推理，准确识别虚假信息并应对新手法
- 结果可解释，适合需要透明检测的场景
多模态虚假信息日益严峻，传统检测方法常因黑箱特性与新篡改手段而失效。我们提出概率概念图推理（PCGR），一种可解释且可演化的框架，将多模态虚假信息检测（MMD）重构为基于结构化概念的推理过程。PCGR采用先构建后推理范式：首先利用多模态大语言模型（MLLMs）自动发现并验证新颖的高层次概念，构建人类可理解的概念节点图；随后在该概念图上应用分层注意力机制，推断陈述的真实性。该设计生成从证据到结论的可解释推理链。实验表明，PCGR在多模态虚假信息检测中达到当前最优准确率和对新型操纵手法的鲁棒性，优于先前方法在粗粒度检测与细粒度操纵识别上的表现。
原文摘要 · Abstract (English)
Multimodal misinformation poses an escalating challenge that often evades traditional detectors, which are opaque black boxes and fragile against new manipulation tactics. We present Probabilistic Concept Graph Reasoning (PCGR), an interpretable and evolvable framework that reframes multimodal misinformation detection (MMD) as structured and concept-based reasoning. PCGR follows a build-then-infer paradigm, which first constructs a graph of human-understandable concept nodes, including novel high-level concepts automatically discovered and validated by multimodal large language models (MLLMs), and then applies hierarchical attention over this concept graph to infer claim veracity. This design produces interpretable reasoning chains linking evidence to conclusions. Experiments demonstrate that PCGR achieves state-of-the-art MMD accuracy and robustness to emerging manipulation types, outperforming prior methods in both coarse detection and fine-grained manipulation recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。