提出新型校准损失，精准控制深度分类器的置信度偏差。
Focal Calibration Loss: Controlling Posterior Distortion in Deep Neural Classifiers
- 在焦点损失中加入概率误差锚点，实现对置信度校准的精细调控。
- 理论证明偏差受锚点强度λ控制，最大不超过√(log K / λ)。
- 适用于需要高置信度可靠性的场景，如医疗诊断与决策系统。
置信度校准在分类器输出的概率被下游任务使用时至关重要。本文研究焦点校准损失（FCL），其在焦点目标中添加了平方概率误差（多分类Brier）锚点：ℒ_FCL^{γ,λ} = ℒ_focal^γ + λ|ŷ(x) - e_y|₂²。分析表明，FCL 对任意 γ, λ ≥ 0 均保持分类校准性，遵守贝叶斯决策规则，但当 γ > 0 时通常不满足合理性，其最优概率向量会偏离真实后验。主结果量化了该偏移：对所有后验和最小化器，偏差上界为 √(log K / λ)，对内部后验可改进至 O(1/λ)，并给出一阶展开式以识别偏差及对应的总体 ℓ₂ 校准保证。通过在单纯形上直接最小化条件风险（无网络参与）验证了这些理论，后验偏差率与预测中位斜率 -0.994 匹配，总体平方校准误差遵循预测的 λ⁻² 规律（斜率 ≈ -1.99）。在 CIFAR-10/100、Tiny-ImageNet、文本与医学多标签任务中，FCL 表现竞争力而非主导；其优势取决于具体设定与评估指标：在常见验证分割协议下，自适应 AdaFocal 在分箱校准误差上更优，而 FCL 在 NLL、Brier 和错误率上于三组中的两组表现更佳。在 Transformer 上未见校准优势，且从头训练实验不支持预训练解释该现象。论文同时报告了成功与失败情形。
原文摘要 · Abstract (English)
Confidence calibration matters wherever a classifier's probabilities, not just its labels, are consumed downstream. We study Focal Calibration Loss (FCL), which adds a squared probability-error (multiclass Brier) anchor to the focal objective, $\mathcal{L}{\mathrm{FCL}}^{γ,λ} = \mathcal{L}{\mathrm{focal}}^γ + λ|\hat{p}(x) - e_y|_2^2$. Our analysis separates two properties that are easily conflated: FCL is classification-calibrated for every $γ, λ\ge 0$, preserving the Bayes decision rule, yet for $γ> 0$ it is generally not proper, so its Bayes-optimal probability vector is displaced from the true posterior. The main result quantifies that displacement and shows the anchor controls it: bounded by $\sqrt{\log K / λ}$ for every posterior and minimizer without regularity assumptions, improving to $O(1/λ)$ for interior posteriors, with an exact first-order expansion identifying the bias and corresponding population $\ell_2$ calibration guarantees. We verify these population statements directly, minimizing the conditional risk on the simplex with no network involved: the posterior-distortion rate matches its prediction to a median fitted slope of $-0.994$, and exact population squared calibration error follows the predicted $λ^{-2}$ law (slopes $\approx -1.99$). Across CIFAR-10/100, Tiny-ImageNet, text and medical multi-label tasks, FCL is competitive rather than dominant, and the picture is regime- and metric-dependent: under a common validation-split protocol the validation-adaptive AdaFocal attains lower binned calibration error, while FCL attains lower NLL, Brier and error on two of three settings. On transformers its calibration advantage is absent, and a from-scratch experiment tested and did not support the conjecture that pretraining explains this. We report both the gains and the failure regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。