让大模型回答更可信，同时可控调节准确性和具体程度。
Conformal Linguistic Calibration: Trading-off between Factuality and Specificity
- 将语言校准视为答案集预测，统一处理拒答与模糊表达。
- 生成结果在事实准确性上具有可证明的置信保证。
- 适合需要平衡准确与细节的应用，如医疗、法律问答。
语言模型输出并不总是可靠，因此研究如何根据不确定性调整模型响应。现有方法包括：当不确定时拒绝回答（abstention）；以及使用不确定度词汇进行语言校准（linguistic calibration）。但拒答会丢失有价值信息，而校准后的表述难以用于下游任务。本文提出统一视角——共形语言校准（Conformal Linguistic Calibration, CLC），将语言校准重新解释为答案集预测。首先通过语用学视角连接拒答与语言校准；随后提出一种实现方法，可控制模型输出的不精确程度。实验表明，该方法生成的输出在事实准确性上具备共形保证。进一步地，该方法支持微调模型进行不确定性感知的自适应陈述重写，实现准确性和具体性之间的可控权衡。
原文摘要 · Abstract (English)
Language model outputs are not always reliable, thus prompting research into how to adapt model responses based on uncertainty. Common approaches include: \emph{abstention}, where models refrain from generating responses when uncertain; and \emph{linguistic calibration}, where models hedge their statements using uncertainty quantifiers. However, abstention can withhold valuable information, while linguistically calibrated responses are often challenging to leverage in downstream tasks. We propose a unified view, Conformal Linguistic Calibration (CLC), which reinterprets linguistic calibration as \emph{answer set prediction}. First we present a framework connecting abstention and linguistic calibration through the lens of linguistic pragmatics. We then describe an implementation of CLC that allows for controlling the level of imprecision in model responses. Results demonstrate our method produces calibrated outputs with conformal guarantees on factual accuracy. Further, our approach enables fine-tuning models to perform uncertainty-aware adaptive claim rewriting, offering a controllable balance between factuality and specificity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。