通过输入扰动的一致性改进模型置信度校准,提升可靠性。
Consistency Calibration: Improving Uncertainty Calibration via Consistency among Perturbed Neighbors
- 基于输入扰动后预测一致性设计校准方法
- 在CIFAR、ImageNet等数据集上达到顶尖校准效果
- 无需额外数据,适用于医疗自动驾驶等高风险场景
校准在深度学习应用中至关重要,尤其在医疗和自动驾驶等领域,准确的置信度估计对决策至关重要。然而,深度神经网络常出现校准不足问题,传统评估方式仅依赖可靠性图与期望校准误差(ECE)。本文提出以一致性为新视角,受大语言模型不确定性研究启发,强调其优于传统可靠性视图的优势。基于此,我们提出一种后处理校准方法——一致性校准(CC),根据模型在扰动输入下的预测一致性调整置信度。该方法在局部不确定性估计中尤为有效,无需额外样本或标签信息,直接从原始数据生成扰动。此外,我们在logit层进行扰动能显著提升计算效率。通过与多种后处理及训练时校准方法对比,验证了CC在标准数据集(如CIFAR-10、CIFAR-100、ImageNet)及长尾数据集(ImageNet-LT)上的卓越表现。
原文摘要 · Abstract (English)
Calibration is crucial in deep learning applications, especially in fields like healthcare and autonomous driving, where accurate confidence estimates are vital for decision-making. However, deep neural networks often suffer from miscalibration, with reliability diagrams and Expected Calibration Error (ECE) being the only standard perspective for evaluating calibration performance. In this paper, we introduce the concept of consistency as an alternative perspective on model calibration, inspired by uncertainty estimation literature in large language models (LLMs). We highlight its advantages over the traditional reliability-based view. Building on this concept, we propose a post-hoc calibration method called Consistency Calibration (CC), which adjusts confidence based on the model's consistency across perturbed inputs. CC is particularly effective in locally uncertainty estimation, as it requires no additional data samples or label information, instead generating input perturbations directly from the source data. Moreover, we show that performing perturbations at the logit level significantly improves computational efficiency. We validate the effectiveness of CC through extensive comparisons with various post-hoc and training-time calibration methods, demonstrating state-of-the-art performance on standard datasets such as CIFAR-10, CIFAR-100, and ImageNet, as well as on long-tailed datasets like ImageNet-LT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。