让大模型学会判断何时该回答,何时该拒绝,提升可信度。
Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal
- 融合多种不确定信号,生成可校准的正确概率
- 在固定错误率下提升回答覆盖率,优于传统阈值方法
- 无需微调模型,适配各类大模型和应用场景
部署的大语言模型不仅要决定答什么,还要判断何时不该回答。我们提出UniCR,一个统一框架,将序列似然、自一致性差异、检索兼容性及工具/验证器反馈等异构不确定性证据,转化为校准后的正确概率,并通过合理的拒绝策略控制用户指定的错误预算。UniCR采用轻量级校准头,结合温度缩放与恰当评分,支持仅通过API访问的模型(黑盒特征),并利用符合性风险控制实现分布无关的保障。针对长文本生成,通过基于检索证据的原子事实性得分监督,使置信度与语义保真度对齐,减少自信幻觉同时保持覆盖范围。在短问答、带执行测试的代码生成和检索增强型长文本问答任务中,均显示校准指标提升,风险-覆盖曲线下面积更低,在固定风险下覆盖更高,优于熵或逻辑阈值、事后校准器及端到端选择基线。分析表明,证据矛盾、语义分散和工具不一致是拒绝的主要驱动因素,能生成有信息量的用户提示。结果是一个可移植的证据融合—校准概率—风险控制决策流程,提升可信度且无需微调基础模型,对分布偏移也稳健有效。
原文摘要 · Abstract (English)
Deployed language models must decide not only what to answer but also when not to answer. We present UniCR, a unified framework that turns heterogeneous uncertainty evidence including sequence likelihoods, self-consistency dispersion, retrieval compatibility, and tool or verifier feedback into a calibrated probability of correctness and then enforces a user-specified error budget via principled refusal. UniCR learns a lightweight calibration head with temperature scaling and proper scoring, supports API-only models through black-box features, and offers distribution-free guarantees using conformal risk control. For long-form generation, we align confidence with semantic fidelity by supervising on atomic factuality scores derived from retrieved evidence, reducing confident hallucinations while preserving coverage. Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics, lower area under the risk-coverage curve, and higher coverage at fixed risk compared to entropy or logit thresholds, post-hoc calibrators, and end-to-end selective baselines. Analyses reveal that evidence contradiction, semantic dispersion, and tool inconsistency are the dominant drivers of abstention, yielding informative user-facing refusal messages. The result is a portable recipe of evidence fusion to calibrated probability to risk-controlled decision that improves trustworthiness without fine-tuning the base model and remains valid under distribution shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。