arXiv:2601.21387cs.CL2026-01Conference of the …

让证据按重要性排序,帮用户更快判断信息真假。

User-Centric Evidence Ranking for Attribution and Fact Verification

  • 按重要性排序证据,优先展示关键信息。
  • 增量式排序比一次性排序更少冗余,效果更好。
  • 适合需要高效验证事实的研究者和从业者。

溯源与事实验证是评估信息可信度的关键挑战。现有自动化系统和大语言模型在检索支持或反驳论断的证据时,常提供信息不足或冗余的内容,导致验证效率低且易出错。为此,我们提出证据排序(Evidence Ranking)新任务,旨在尽早呈现充分证据,减少用户阅读负担,同时保留全部可用信息以供后续验证。我们对比了一次性排序与增量式排序两种方法,并引入基于信息检索指标的新评估框架,整合多个现有事实验证数据集构建统一基准。大量实验表明,增量式策略更能捕捉互补证据,基于大语言模型的方法优于浅层基线,但仍面临充分性与冗余性的平衡难题。通过受控用户研究发现,相比证据选择,证据排序能显著降低阅读成本并提升验证准确率。本工作为构建更可解释、高效且用户友好的信息验证系统奠定基础。

原文摘要 · Abstract (English)

Attribution and fact verification are critical challenges in natural language processing for assessing information reliability. While automated systems and Large Language Models (LLMs) aim to retrieve and select concise evidence to support or refute claims, they often present users with either insufficient or overly redundant information, leading to inefficient and error-prone verification. To address this, we propose Evidence Ranking, a novel task that prioritizes presenting sufficient information as early as possible in a ranked list. This minimizes user reading effort while still making all available evidence accessible for sequential verification. We compare two approaches for the new ranking task: one-shot ranking and incremental ranking. We introduce a new evaluation framework, inspired by information retrieval metrics, and construct a unified benchmark by aggregating existing fact verification datasets. Extensive experiments with diverse models show that incremental ranking strategies better capture complementary evidence and that LLM-based methods outperform shallower baselines, while still facing challenges in balancing sufficiency and redundancy. Compared to evidence selection, we conduct a controlled user study and demonstrate that evidence ranking both reduces reading effort and improves verification. This work provides a foundational step toward more interpretable, efficient, and user-aligned information verification systems.

事实验证证据排序LLM应用用户研究

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。