通过分组校准提升模型置信度可靠性,减少盲目自信的错误预测。
Uncertainty-Aware Post-Hoc Calibration: Mitigating Confidently Incorrect Predictions Beyond Calibration Metrics
- 按特征相似性分组预测,区分可信与不可信样本
- 对可信样本用常规校准,对不可信样本降低置信度至均匀分布
- 无需重训练,适合需要可靠置信度的应用场景
尽管神经网络校准研究广泛,现有方法多采用全局变换,忽略个体预测的可靠性差异。本文提出一种后处理校准框架,利用预测可靠性评估,同时提升校准质量与不确定性感知决策能力。该框架基于邻近性等距预测,根据特征空间中的语义相似性将校准样本分为可能正确和可能错误两类。对可能正确的预测使用标准保序回归校准置信度,对可能错误的预测则采用欠自信正则化保序回归,使其置信度趋向均匀分布,便于识别并进一步分析。在CIFAR-10和CIFAR-100上使用BiT和CoAtNet骨干网络进行实验,结果表明该方法显著降低自信错误预测数量,且期望校准误差(ECE)优于保序回归与焦点损失基线。该工作通过实例级自适应实现校准与不确定性量化融合,提供无需模型重训练的实用后处理方案,同时改善概率对齐与不确定性感知决策。
原文摘要 · Abstract (English)
Despite extensive research on neural network calibration, existing methods typically apply global transformations that treat all predictions uniformly, overlooking the heterogeneous reliability of individual predictions. Furthermore, the relationship between improved calibration and effective uncertainty-aware decision-making remains largely unexplored. This paper presents a post-hoc calibration framework that leverages prediction reliability assessment to jointly enhance calibration quality and uncertainty-aware decision-making. The framework employs proximity-based conformal prediction to stratify calibration samples into putatively correct and putatively incorrect groups based on semantic similarity in feature space. A dual calibration strategy is then applied: standard isotonic regression calibrated confidence in putatively correct predictions, while underconfidence-regularized isotonic regression reduces confidence toward uniform distributions for putatively incorrect predictions, facilitating their identification for further investigations. A comprehensive evaluation is conducted using calibration metrics, uncertainty-aware performance measures, and empirical conformal coverage. Experiments on CIFAR-10 and CIFAR-100 with BiT and CoAtNet backbones show that the proposed method achieves lower confidently incorrect predictions, and competitive Expected Calibration Error compared with isotonic and focal-loss baselines. This work bridges calibration and uncertainty quantification through instance-level adaptivity, offering a practical post-hoc solution that requires no model retraining while improving both probability alignment and uncertainty-aware decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。