针对神经网络过拟合与欠拟合问题，提出自适应调节平滑策略提升模型置信度校准。
FedLAS: Feature-Modulated Bidirectional Label Smoothing for Neural Network Calibration
- 基于特征范数构建置信度指标，动态识别样本过/欠自信状态。
- 引入双向校准门控模块，实现对过自信和欠自信的联合修正。
- 可无缝集成现有平滑方法，显著降低校准误差且不损失准确率。
深度神经网络分类器在预测置信度与实际正确率不一致时会出现校准偏差，表现为错误预测过于自信或正确预测过于保守。标签平滑（LS）通过训练中引入熵正则化，将真实标签的概率质量分配给其他类别以改善校准，但传统方法如基于边距的标签平滑（MbLS）依赖预设的均匀平滑规则，仅解决过自信问题。现实中样本存在难易、模糊等差异，且模型训练过程中的置信度状态会动态变化。为此，我们提出FedLAS：一种基于特征调制的双向标签平滑算法。该方法通过特征范数构建置信度指示器（NCI）控制平滑程度，并引入双向校准门控（BCG）模块同时检测过自信与欠自信现象。该算法可与主流的LS及MbLS损失函数结合，在标准与细粒度高分辨率视觉基准上持续提升校准性能，显著降低期望校准误差（ECE）与自适应校准误差（Adaptive ECE），同时保持顶级准确率。代码已开源。
原文摘要 · Abstract (English)
Deep Neural Network (DNN) classifiers suffer from poor calibration when their softmax outputs (predictive confidence) deviate from the empirical likelihoods. This manifests itself as either overconfident incorrect predictions or under-confident correct predictions. Label smoothing (LS) enhances model calibration by introducing entropy regularization during training through redistributing probability mass from the ground-truth label to the remaining classes. LS, including Margin-based LS (MbLS), have restrictive assumptions: they rely on predefined, uniform smoothing rules and only tackle overconfidence. In reality, samples exhibit diverse characteristics, such as difficulty/ambiguity, that interact with the evolving nature of the model being trained. In training, samples may have various degrees of under- or overconfidence. To overcome this, a mechanism that identifies the specific confidence state of each sample and determines the appropriate degree of smoothing in each training step is needed, tailoring the adjustment to the individual sample. We propose FedLAS: Feature-Modulated Bidirectional Label Smoothing, a plug-and-play algorithm for label smoothing-based losses. In FedLAS, we introduce a Feature Norm-based Confidence Indicator (NCI) to control smoothing and a Bidirectional Calibration Gating (BCG) module to detect both over and under-confidence. Our algorithm can be integrated with LS and MbLS based losses when applied to standard DNNs, enhancing performance. Extensive experiments on standard and fine-grained high-resolution vision benchmarks show that FedLAS consistently improves calibration compared to modern baselines, reducing Expected Calibration Error (ECE) and Adaptive ECE while maintaining Top-1 accuracy. Code: github.com/nadarasarbahavan/FEDLAS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。