用图结构+大模型交互,让复杂说法验证更准更可解释。
Verify-in-the-Graph: Entity Disambiguation Enhancement for Complex Claim Verification with Interactive Graph Representation
- 把说法拆成带图的三元组,融合结构与非结构信息
- 大模型反复查知识库,解决实体歧义问题,提升验证准确率
- 适合需要高可信度和过程可解释的虚假信息检测场景
事实核查是一项长期且具有挑战性的任务,不仅要求高准确率,还需具备可解释性。在大语言模型时代,真实说法常具复杂语义结构或隐藏实体,传统方法通过分解说法并查询知识库来解决隐含实体,但缺乏有效的消歧策略会损害整体验证效果。为此,我们提出 Verify-in-the-Graph(VeGraph)框架,利用大模型代理的推理与理解能力,分三阶段运行:(1) 图表示——将输入说法分解为结构化三元组,构建融合结构与非结构信息的图表示;(2) 实体消歧——迭代式交互知识库,解决图中模糊实体,深化子说法验证;(3) 验证——对剩余三元组进行验证,完成事实核查。基于 Meta-Llama-3-70B(指令版)的实验表明,VeGraph 在 HoVer 与 FEVEROUS 两个基准上表现优于基线,有效应对复杂说法验证挑战。代码与数据已公开。
原文摘要 · Abstract (English)
Claim verification is a long-standing and challenging task that demands not only high accuracy but also explainability of the verification process. This task becomes an emerging research issue in the era of large language models (LLMs) since real-world claims are often complex, featuring intricate semantic structures or obfuscated entities. Traditional approaches typically address this by decomposing claims into sub-claims and querying a knowledge base to resolve hidden or ambiguous entities. However, the absence of effective disambiguation strategies for these entities can compromise the entire verification process. To address these challenges, we propose Verify-in-the-Graph (VeGraph), a novel framework leveraging the reasoning and comprehension abilities of LLM agents. VeGraph operates in three phases: (1) Graph Representation - an input claim is decomposed into structured triplets, forming a graph-based representation that integrates both structured and unstructured information; (2) Entity Disambiguation -VeGraph iteratively interacts with the knowledge base to resolve ambiguous entities within the graph for deeper sub-claim verification; and (3) Verification - remaining triplets are verified to complete the fact-checking process. Experiments using Meta-Llama-3-70B (instruct version) show that VeGraph achieves competitive performance compared to baselines on two benchmarks HoVer and FEVEROUS, effectively addressing claim verification challenges. Our source code and data are available for further exploitation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。