用三步流程提升复杂声明的真假验证准确率
Retrieve-Refine-Calibrate: A Framework for Complex Claim Fact-Checking
- 先找声明中实体，再检索相关证据，最后精炼并校准结果
- 在HOVER和FEVEROUS-S数据集上超越现有方法
- 适合需要高精度事实核查的应用场景
事实核查旨在根据检索到的证据验证声明的真实性。现有方法通常采用分解范式，将声明拆分为子声明分别验证。但该范式可能引入无关实体或证据，导致验证准确率下降。为此，我们提出基于大语言模型（LLMs）的检索-精炼-校准（RRC）框架。首先识别声明中的实体并检索相关证据；其次基于声明对检索到的证据进行精炼，去除冗余信息；最后通过重新评估低置信度预测来校准验证过程。在两个主流事实核查数据集（HOVER 和 FEVEROUS-S）上的实验表明，该框架显著优于现有基线方法。
原文摘要 · Abstract (English)
Fact-checking aims to verify the truthfulness of a claim based on the retrieved evidence. Existing methods typically follow a decomposition paradigm, in which a claim is broken down into sub-claims that are individually verified. However, the decomposition paradigm may introduce noise to the verification process due to irrelevant entities or evidence, ultimately degrading verification accuracy. To address this problem, we propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs). Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them. Then, it refines the retrieved evidence based on the claim to reduce irrelevant information. Finally, it calibrates the verification process by re-evaluating low-confidence predictions. Experiments on two popular fact-checking datasets (HOVER and FEVEROUS-S) demonstrate that our framework achieves superior performance compared with competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。