arXiv:2601.02574cs.CLcs.AI2026-01

让大模型自己判断何时该查证,提升事实核查的准确与效率。

Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency

  • 基于概率置信度与推理一致性,动态判断是否需要外部查证。
  • 在三个基准上优于现有方法,不确定性识别更精准。
  • 适用于多种大模型,适合对可靠性要求高的应用。

大语言模型(LLMs)在需事实准确性的场景中日益普及,但其输出常包含幻觉内容。现有核查方法多盲目检索外部证据,忽视模型内部知识,易引入无关噪声。当前系统也缺乏针对特定推理不确定性的处理机制。受人类查证方式启发,我们提出概率置信度与一致性(PCC)框架,通过联合建模模型的概率置信度与推理一致性来估计事实可信度。该置信信号驱动自适应验证策略:高置信时直接作答,不确定或不一致时触发针对性检索,高度模糊时升级为深度搜索。置信引导的路由机制确保仅在必要时调用检索,提升效率与可靠性。跨三个挑战性基准的实验表明,PCC在不确定性量化上优于口头置信度,且持续优于强基线。此外,PCC在多种大模型间具有良好泛化能力。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used in applications requiring factual accuracy, yet their outputs often contain hallucinated responses. While fact-checking can mitigate these errors, existing methods typically retrieve external evidence indiscriminately, overlooking the model's internal knowledge and potentially introducing irrelevant noise. Moreover, current systems lack targeted mechanisms to resolve specific uncertainties in the model's reasoning. Inspired by how humans fact-check, we argue that LLMs should adaptively decide whether to rely on internal knowledge or initiate retrieval based on their confidence in a given claim. We introduce Probabilistic Certainty and Consistency (PCC), a framework that estimates factual confidence by jointly modeling an LLM's probabilistic certainty and reasoning consistency. These confidence signals enable an adaptive verification strategy: the model answers directly when confident, triggers targeted retrieval when uncertain or inconsistent, and escalates to deep search when ambiguity is high. Our confidence-guided routing mechanism ensures that retrieval is invoked only when necessary, improving both efficiency and reliability. Extensive experiments across three challenging benchmarks show that PCC achieves better uncertainty quantification than verbalized confidence and consistently outperforms strong LLM-based fact-checking baselines. Furthermore, we demonstrate that PCC generalizes well across various LLMs.

大模型事实核查置信度自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。