arXiv:2605.29712cs.CLcs.AI2026-05ACL

用考试策略教大模型判断事实真假,省 token 又准

Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies

  • 把事实核查当选择题,用考试策略引导推理
  • 比自由推理少用80%以上token,效果媲美高价方法
  • 小模型也能高效判断,还生成解释,适合落地

基于事实的陈述真实性检查对大语言模型(LLM)应用(如检索增强生成)至关重要,有助于用户评估生成内容的正确性。现有基于蕴含分类器的指标需针对数据集调参,而基于LLM的方法常采用直接提示,未能充分发挥其推理能力。本文将事实核查任务重构为真假阅读理解题,并通过显式引入考试答题策略来引导LLM进行高效推理。该方法相比无指导的开放式推理,减少了超过80%的令牌使用量,在两个事实核查基准上达到与更昂贵方法相当的性能,并在其中一个基准上创下新最佳纪录。为进一步降低推理成本,我们训练小型语言模型(SLMs)以替代原流程中的LLM。通过监督微调(SFT)和自修正机制,这些SLMs学会提升判断质量。实验表明,优化后的SLMs性能媲美强基线,兼具低推理开销与生成支持性推理链的能力,提升可解释性。代码与数据集将在论文录用后公开。

原文摘要 · Abstract (English)

Grounded claim factuality checking is important for large language model (LLM) applications such as retrieval-augmented generation, as it helps users assess the correctness of generated outputs. Existing metrics using entailment classifiers require dataset-specific threshold tuning, while LLM-based approaches often use direct prompting, which underutilises the reasoning capabilities of LLMs. We address this by formulating grounded claim factuality checking as a true/false reading comprehension task and prompting LLMs with explicit test-taking strategies for efficient reasoning. Our method reduces token usage by over 80% compared to unguided open-ended reasoning, and achieves competitive performance to more expensive alternatives across two factuality benchmarks, setting a new state of the art on one. To further reduce inference cost, we train small language models (SLMs) to replace LLMs in the checking pipeline. Using supervised fine-tuning (SFT) and a self-revision mechanism, the SLMs learn to improve their factuality judgements. Experimental results show that the resulting SLMs perform on par with strong baselines, combining low inference costs with generating supporting rationales to support interpretability. Code and datasets will be released upon acceptance.

事实核查推理优化小模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。