通过调控训练过程中的边缘与曲率，提升模型置信度校准性。
Too Sharp, Too Sure: When Calibration Follows Curvature
- 基于梯度方法的训练中，校准性与曲率、边缘紧密关联。
- 校准误差随优化过程中的曲率变化而同步变化，相关性显著。
- 新目标函数提升校准效果，适用于多种优化器且不损失准确率。
现代神经网络虽能达到高精度，但置信度估计常与实际正确率不匹配，即校准性差。现有研究多将校准视为事后调整属性，本文则从训练过程视角出发，在小型视觉任务上研究校准性如何受训练过程影响。我们发现，在多种基于梯度的方法下，校准性、曲率与分类边缘之间存在紧密耦合。实验证明，期望校准误差（ECE）在整个优化过程中与基于曲率的尖锐度高度相关。数学上，我们证明了ECE和高斯-牛顿曲率均受轨迹上同一类依赖边距的指数尾部泛函控制，仅相差特定问题常数。基于此机制，我们提出一种边缘感知的训练目标，显式优化鲁棒边缘尾部与局部平滑性，可在不牺牲精度的前提下，提升不同优化器下的样本外校准性能。
原文摘要 · Abstract (English)
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。