SAM训练让神经网络更靠谱，减少误判风险。
Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
- 用尖锐度感知优化提升模型预测置信度的合理性
- 在ImageNet-1K等数据集上显著降低校准误差
- 适合对可靠性要求高的医疗与自动驾驶场景
深度神经网络在医疗诊断、自动驾驶等安全关键领域应用日益广泛。然而，多项研究表明其常存在校准不良和过度自信问题，可能导致严重后果。本文指出，与标准训练（如随机梯度下降）不同，近期提出的尖锐度感知最小化（SAM）能有效抑制过度自信倾向。理论分析表明，SAM通过隐式最大化预测分布的熵，使模型天然具备良好校准性。受此启发，我们提出SAM的改进版本CSAM，进一步优化校准效果。在ImageNet-1K等多个数据集上的大量实验验证了SAM在降低校准误差方面的优势，而CSAM表现更优，始终优于其他方法。
原文摘要 · Abstract (English)
Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。