用更准的不确定性权重改进梯度,让模型预测更可信
Uncertainty Weighted Gradients for Model Calibration
- 从梯度视角重设计优化,聚焦不确定样本
- 采用Brier Score计算权重,提升不确定性估计精度
- 在多个数据集上达到最优校准效果,适合高可靠性场景
模型校准对确保深度神经网络预测概率真实反映实际概率至关重要。然而,深度网络常产生过度自信或不足自信的预测,导致校准偏差。已有方法通过设计有效损失函数(如焦点损失)改善校准,但本文分析发现其优势源于样本级不确定性权重。现有方法存在两个主要问题:优化过程中的不确定性与梯度缩放不匹配,以及单个输出对不确定性的表达能力不足。为此,我们从梯度角度重构优化,聚焦不确定性样本;并提出使用Brier Score作为损失权重因子,通过所有logits实现更精准的不确定性估计。在多种模型和数据集上的大量实验表明,该方法达到当前最优(SOTA)校准性能。
原文摘要 · Abstract (English)
Model calibration is essential for ensuring that the predictions of deep neural networks accurately reflect true probabilities in real-world classification tasks. However, deep networks often produce over-confident or under-confident predictions, leading to miscalibration. Various methods have been proposed to address this issue by designing effective loss functions for calibration, such as focal loss. In this paper, we analyze its effectiveness and provide a unified loss framework of focal loss and its variants, where we mainly attribute their superiority in model calibration to the loss weighting factor that estimates sample-wise uncertainty. Based on our analysis, existing loss functions fail to achieve optimal calibration performance due to two main issues: including misalignment during optimization and insufficient precision in uncertainty estimation. Specifically, focal loss cannot align sample uncertainty with gradient scaling and the single logit cannot indicate the uncertainty. To address these issues, we reformulate the optimization from the perspective of gradients, which focuses on uncertain samples. Meanwhile, we propose using the Brier Score as the loss weight factor, which provides a more accurate uncertainty estimation via all the logits. Extensive experiments on various models and datasets demonstrate that our method achieves state-of-the-art (SOTA) performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。