用轻量模型生成伪子图,让大模型更准地验证事实
ClaimPKG: Enhancing Claim Verification via Pseudo-Subgraph Generation with Lightweight Specialized LLM
- 用轻量专用模型将声明转为伪子图,引导检索相关知识图谱片段
- 在FactKG上比现有方法高9%-12%准确率,跨数据集零样本泛化能力强
- 适合做事实核查、多步推理的AI系统开发者参考
将知识图谱(KG)融入大语言模型(LLM)以增强其推理能力,是当前事实验证领域的研究挑战。尽管KG提供结构化语义信息,但多数现有方法依赖非结构化文本,难以有效利用KG。此外,现代LLM虽具备强推理能力,却在多步骤模块化流程和KG推理中表现不佳,需适应。为此,我们提出ClaimPKG,一个端到端框架,实现LLM推理与结构化知识的无缝融合。核心思路是使用轻量专用LLM将输入声明转化为伪子图,指导专门的子图检索模块定位相关KG子图;再由通用LLM处理这些子图,输出最终判断与解释。在FactKG数据集上的大量实验表明,ClaimPKG达到领先性能,相比强基线提升9%-12%准确率点。此外,该方法在未见过的非结构化数据集(如HoVer和FEVEROUS)上展现出零样本泛化能力,能有效结合结构化知识与多种LLM架构的推理能力。
原文摘要 · Abstract (English)
Integrating knowledge graphs (KGs) to enhance the reasoning capabilities of large language models (LLMs) is an emerging research challenge in claim verification. While KGs provide structured, semantically rich representations well-suited for reasoning, most existing verification methods rely on unstructured text corpora, limiting their ability to effectively leverage KGs. Additionally, despite possessing strong reasoning abilities, modern LLMs struggle with multi-step modular pipelines and reasoning over KGs without adaptation. To address these challenges, we propose ClaimPKG, an end-to-end framework that seamlessly integrates LLM reasoning with structured knowledge from KGs. Specifically, the main idea of ClaimPKG is to employ a lightweight, specialized LLM to represent the input claim as pseudo-subgraphs, guiding a dedicated subgraph retrieval module to identify relevant KG subgraphs. These retrieved subgraphs are then processed by a general-purpose LLM to produce the final verdict and justification. Extensive experiments on the FactKG dataset demonstrate that ClaimPKG achieves state-of-the-art performance, outperforming strong baselines in this research field by 9%-12% accuracy points across multiple categories. Furthermore, ClaimPKG exhibits zero-shot generalizability to unstructured datasets such as HoVer and FEVEROUS, effectively combining structured knowledge from KGs with LLM reasoning across various LLM backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。