仅用一次生成结果，就能让大模型的推理置信度更准。
Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation
- 用无标签数据离线采样生成自一致代理目标，替代真实标签。
- 在5个数学与问答任务上显著优于基线，分布外情况也有效。
- 适合部署时无法重复采样或无标注数据的场景，如实时系统。
推理型语言模型能解决越来越复杂的任务，但难以生成可靠部署所需的校准置信度。现有校准方法通常依赖标签或推理时重复采样，在许多场景下不实用。本文提出一种仅需单次生成即可实现推理大模型无监督置信度校准的方法。该方法利用无标签数据离线采样，构建基于自一致性的代理目标，并将此信号压缩为轻量级部署时置信度预测器。在涵盖5个数学与问答任务、使用9种推理模型的广泛评估中，本方法显著优于基线，即使在分布外情况下仍表现良好，并提升了选择性预测与模拟下游决策的表现。
原文摘要 · Abstract (English)
Reasoning language models can solve increasingly complex tasks, but struggle to produce the calibrated confidence estimates necessary for reliable deployment. Existing calibration methods usually depend on labels or repeated sampling at inference time, making them impractical in many settings. We introduce a method for unsupervised confidence calibration of reasoning LLMs when only a single generation is available at inference time. Our approach uses offline sampling on unlabeled data to derive a self-consistency-based proxy target, then distills this signal into a lightweight deployment-time confidence predictor. In a broad evaluation across 5 math and question-answering tasks using 9 reasoning models, our method substantially outperforms baselines, including under distribution shift, and improves downstream performance in selective prediction and simulated downstream decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。