让语言模型学会在不确定时主动放弃回答，避免幻觉。
Geometry-Calibrated Conformal Abstention for Language Models
- 基于置信度而非传统非符合性分数做拒答决策，更适用于开放生成。
- 在不确定时拒答，可保证75%条件下回答正确，且有有限样本保障。
- 利用模型内部表示几何结构校准置信度，提升拒答准确性。
当语言模型对某个问题缺乏相关知识时，常会生成看似合理实则虚构的回答，而非承认无知。通过重训练模型以奖励承认无知，可能导致过度保守且泛化能力差，因评估基准稀缺。本文提出一种后处理框架——置信度拒答（Conformal Abstention, CA），源自置信预测（CP），用于判断是否应拒绝回答。CA在有限样本下同时保障参与概率（即不拒答）与回答正确的概率。关键在于，拒答决策基于预测置信度，而非置信预测中难以计算的非符合性分数，后者在开放式生成任务中不可行。为使置信度更好反映模型无知程度，我们引入基于模型内部表示几何的校准策略，衡量知识参与度。实验表明，该方法显著提升选择性回答性能，在75%的条件下确保回答正确。
原文摘要 · Abstract (English)
When language models lack relevant knowledge for a given query, they frequently generate plausible responses that can be hallucinations, rather than admitting being agnostic about the answer. Retraining models to reward admitting ignorance can lead to overly conservative behaviors and poor generalization due to scarce evaluation benchmarks. We propose a post hoc framework, Conformal Abstention (CA), adapted from conformal prediction (CP) to determine whether to abstain from answering a query. CA provides finite-sample guarantees on both the probability of participation (i.e., not abstaining) and the probability that the generated response is correct. Importantly, the abstention decision relies on prediction confidence rather than the non-conformity scores used in CP, which are intractable for open-ended generation. To better align prediction confidence with the model's ignorance, we introduce a calibration strategy using representation geometry within the model to measure knowledge involvement in shaping the response. Experiments demonstrate that we improve selective answering significantly with 75 percent conditional correctness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。