针对模型置信度校准不均问题,提出分位数自适应温度调整方法。
Quantile Adaptive Temperature Scaling for Confidence Calibration
- 按预测置信度分位数动态调整温度,实现分区域精准校准
- 在多个数据集和场景下显著降低校准误差,尤其改善低置信度预测
- 无需修改模型,适合追求可信预测的实用部署场景
深度神经网络常产生校准不良的置信度估计,即使预测错误也过度自信。温度缩放因简单有效仍是主流后处理校准方法,但其全局统一的对数重缩放无法应对置信度谱中高度异质的校准偏差。尤其在低置信度区域,不确定性最为关键,但标准温度缩放对此类偏差改善有限。本文提出分位数自适应温度缩放(QaTS),一种简单高效的后处理校准方法,将温度设为预测经验置信度分位数的函数。通过将置信度映射至分位数空间,QaTS使校准问题标准化,显化校准偏差结构,并实现跨分位数单调变化的温度函数,同时保持高置信度预测基本不变。该分位数感知形式自然契合重参数化的期望校准误差(ECE)目标,得到样本级温度,在类别不平衡与分布偏移等挑战性场景下仍具鲁棒性。在多种数据集、模型架构、评估场景及任务中,QaTS始终显著优于现有最优后处理校准方法,无需修改模型预测即可提供更可靠、可信的置信度估计。
原文摘要 · Abstract (English)
Deep neural networks often produce poorly calibrated confidence estimates, overstating their certainty even when predictions are incorrect. Temperature Scaling remains the most widely used posthoc calibration method due to its simplicity and effectiveness, yet its global, uniform rescaling of logits fails to correct the highly heterogeneous structure of miscalibration observed across the confidence spectrum. In particular, the largest correctness confidence discrepancies arise in different quantile regions depending on the setting, low confidence predictions, where uncertainty matters most, tend to exhibit the largest correctness confidence discrepancies, which standard TS leaves largely unaddressed. We introduce Quantile Adaptive Temperature Scaling (QaTS), a simple and efficient post hoc calibration method that adapts the temperature as a function of a predictions empirical confidence quantile. By mapping confidences into the quantile space, QaTS normalizes the calibration problem, makes the structure of miscalibration explicit and enables a monotone temperature function that adapts across quantiles while leaving well calibrated high confidence predictions largely unchanged. preserving high confidence behavior. This quantile aware formulation aligns naturally with a reparameterized Expected Calibration Error (ECE) objective and yields a sample wise temperature that is robust across a variety of challenging scenarios, such as class imbalance and distributional shifts. Across a broad range of datasets, architectures, evaluation scenarios and diverse tasks, QaTS consistently, and substantially, outperforms state of the art post hoc calibration methods, delivering more reliable and trustworthy confidence estimates without modifying model predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。