arXiv:2605.06006cs.CL2026-05中稿 · LREC 2026被引 1

将新闻事实核查文章中的证据提炼为可复用的独立语句,提升自动验证效果。

From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence

论文配图:From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence
图 1 · 摘自论文原文
  • 用大模型将文章中的超链接句子改写为独立、去上下文的前提句。
  • 提取的证据使检索排名提升30%,判断结论的准确率提高10-20分。
  • 适用于需要高可信证据的自动事实核查系统,尤其适合多粒度判断场景。

事实核查文章包含丰富的支持性证据和推理过程,但因呈现方式非结构化,自动化系统难以利用。本文提出PrimeFacts,一种从完整事实核查文章中提取细粒度证据的方法与资源。我们整理了13,106篇PolitiFact文章,包含待验证声明、裁决结果及所有引用来源,并识别出49,718个文中超链接作为自然锚点,定位关键证据。该框架利用大语言模型(LLMs)将这些锚点句子重写为独立、上下文无关的前提句,并探索额外隐含证据的提取。在跨文章证据检索和声明验证任务中,所提取前提显著提升性能:去上下文化证据的可检索性提升达30%相对增益(以均倒数排名衡量),用于裁决预测时宏平均F1提升10-20分。该效果在不同裁决粒度(2类与5类)及模型架构下均保持一致。定性分析显示,重构的前提忠实于原始来源。本工作展示了复用事实核查员证据的潜力,并提供了一个基于真实核查案例的大规模结构化证据资源。

原文摘要 · Abstract (English)

Fact-checking articles encode rich supporting evidence and reasoning, yet this evidence remains largely inaccessible to automated verification systems due to unstructured presentation. We introduce PrimeFacts, a methodology and resource for extracting fine-grained evidence from full fact-checking articles. We compile 13,106 PolitiFact articles with claims, verdicts, and all referenced sources, and we identify 49,718 in-article hyperlinks as natural anchors to pinpoint key evidence. Our framework leverages large language models (LLMs) to rewrite these anchor sentences into stand-alone, context-independent premises and investigates the extraction of additional implicit evidence. In evaluations on cross-article evidence retrieval and claim verification, the extracted premises substantially improve performance. Decontextualized evidence yields higher retrievability, achieving up to a 30 percent relative gain in Mean Reciprocal Rank over verbatim sentences, and using the evidence for verdict prediction raises Macro-F1 by 10-20 points over the baseline. These gains are consistent across different verdict granularities (2-class vs. 5-class) and model architectures. A qualitative analysis indicates that the decontextualized premises remain faithful to the original sources. Our work highlights the promise of reusing fact-checkers' evidence for automation and provides a large-scale resource of structured evidence from real-world fact-checks.

事实核查证据提取LLM应用结构化数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。