让大模型在检索错误时仍能准确判断自己是否靠谱。
NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systems
- 基于噪声感知规则构建可信度校准框架
- 在热狗问答数据集上微调，提升置信度准确性
- 适合需要可靠答案的高风险应用
准确评估模型置信度对部署大语言模型于关键事实领域至关重要。尽管检索增强生成（RAG）被广泛采用以提高事实依据性，但其下的置信度校准仍不清晰。我们在四个基准上系统研究发现，当检索到含噪声上下文时，大模型表现严重过自信，尤其是存在矛盾或无关证据时。为此，我们提出NOVA规则（噪声感知口语化置信度校准规则），为噪声环境下的过自信问题提供理论基础。进一步设计了NOVA框架，利用约2000个HotpotQA样本，在规则指导下进行监督微调，使模型具备内在噪声感知能力，无需依赖更强教师模型。实验表明，NOVA在域内和域外分别提升ECE分数10.9%和8.0%，有效弥合检索噪声与口语化校准之间的鸿沟，推动更准确且认知可靠的大型语言模型发展。
原文摘要 · Abstract (English)
Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance especially when noisy contexts are retrieved. Specifically, contradictory or irrelevant evidence tends to exacerbate the model's overconfidence issue. To address this, we propose NOVA Rules (NOise-Aware Verbal Confidence CAlibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NOVA, a noise-aware calibration framework that synthesizes supervision from ~2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NOVA equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NOVA yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NOVA paves the way for both accurate and epistemically reliable LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。