arXiv:2510.24020cs.CLcs.AI2025-10被引 2

通过细粒度语义置信度奖励,让大模型更精准地拒绝回答不确定问题。

Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward

  • 基于候选答案的语义聚类,实现每题独立的置信度判断
  • 在域内与域外测试中,拒答可靠性显著提升
  • 适合需要高可信度输出的应用场景

减少大语言模型(LLMs)幻觉是其可靠部署的关键。现有方法通常通过粗粒度信号(如整体置信度或多个采样答案的不确定性)引导模型拒答,但难以精确识别知识边界。为此,我们提出一种新型强化学习框架——细粒度语义置信度奖励(FiSCoR),通过采样多个候选答案并进行语义聚类,训练模型保留高置信度簇中的答案,舍弃低置信度簇中的答案,从而实现精准的事后拒答。此外,我们还提出一个新评估指标,更全面衡量拒答微调任务的可靠性。实验表明,该方法在域内和域外基准上均显著提升了拒答可靠性。

原文摘要 · Abstract (English)

Mitigating hallucinations in Large Language Models (LLMs) is critical for their reliable deployment. Existing methods typically fine-tune LLMs to abstain from answering questions beyond their knowledge scope. However, these methods often rely on coarse-grained signals to guide LLMs to abstain, such as overall confidence or uncertainty scores on multiple sampled answers, which may result in an imprecise awareness of the model's own knowledge boundaries. To this end, we propose a novel reinforcement learning framework built on $\textbf{\underline{Fi}ne-grained \underline{S}emantic \underline{Co}nfidence \underline{Re}ward (\Ours)}$, which guides LLMs to abstain via sample-specific confidence. Specifically, our method operates by sampling multiple candidate answers and conducting semantic clustering, then training the LLM to retain answers within high-confidence clusters and discard those within low-confidence ones, thereby promoting accurate post-hoc abstention. Additionally, we propose a new metric for evaluating the reliability of abstention fine-tuning tasks more comprehensively. Our method significantly enhances reliability in both in-domain and out-of-distribution benchmarks.

大模型幻觉抑制拒答机制强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。