arXiv:2505.23912cs.CLcs.AI2025-05ACL被引 16

让大模型生成时实时打分,判断内容真假。

LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations

  • 用强化学习训练模型在长文本生成中实时附加可信度分数。
  • 比传统方法快20倍,且在三个数据集上校准更准确。
  • 适合需要高可信度的长文本生成场景,如报告、论文写作。

幻觉仍是大型语言模型在事实性内容生成中安全可信部署的主要挑战。以往工作通过置信度估计检测幻觉,但多依赖计算成本高的事后自一致性方法。口头置信度提供了更高效替代方案,但现有方法主要局限于短文本问答任务,难以推广至开放式生成。本文提出LoVeC(长文本口头置信度),一种基于强化学习的新方法,训练大模型在长文本生成过程中对每条陈述实时附加数值置信度评分。该评分直接反映生成内容的真实性。我们设计了自由标注和迭代标注两种评估设置,以检验不同口头置信度估计方法。在三个长文本问答数据集上的实验表明,经强化学习训练的模型在置信度校准方面表现更优,并在跨领域任务中具有强泛化能力。此外,本方法效率极高,较传统自一致性方法快20倍,同时实现更好校准效果。

原文摘要 · Abstract (English)

Hallucination remains a major challenge for the safe and trustworthy deployment of large language models (LLMs) in factual content generation. Prior work has explored confidence estimation as an effective approach to hallucination detection, but often relies on post-hoc self-consistency methods that require computationally expensive sampling. Verbalized confidence offers a more efficient alternative, but existing approaches are largely limited to short-form question answering (QA) tasks and do not generalize well to open-ended generation. In this paper, we propose LoVeC (Long-form Verbalized Confidence), a novel reinforcement learning based method that trains LLMs to append an on-the-fly numerical confidence score to each generated statement during long-form generation. The confidence score serves as a direct and interpretable signal of the factuality of generation. We introduce two evaluation settings, free-form tagging and iterative tagging, to assess different verbalized confidence estimation methods. Experiments on three long-form QA datasets show that our RL-trained models achieve better calibration and generalize robustly across domains. Also, our method is highly efficient, being 20 times faster than traditional self-consistency methods while achieving better calibration.

置信度长文本生成强化学习幻觉检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。