SAM抗标签噪声的机制被揭示，新方法SANER进一步提升鲁棒性。
Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting
- 通过梯度加权分析发现，干净梯度在噪声主导方向上被更强放大，抑制了错误记忆。
- SANER在多个图像分类数据集上显著减少噪声标签记忆，优于SAM和SGD。
- 方法简洁可嵌入SAM类变体，适合噪声数据训练场景。
Sharpness-Aware Minimization (SAM) 被提出以通过寻找平坦极小值来提升泛化能力，但也表现出对标签噪声的鲁棒性，这一现象尚未被完全理解。以往研究主要归因于SAM延长了对干净样本的学习过程。本文从元素级角度分析SAM，发现当噪声梯度主导某参数方向时，其影响因干净梯度的更强放大而被削弱。这减缓了噪声标签的过拟合，同时维持了对干净数据的学习，提供了对SAM鲁棒性的更完整解释。基于此，我们提出SANER（Sharpness-Aware Noise-Explicit Reweighting），一种简单改进的SAM变体，显式增强这种降权效应。在含噪声标签的基准图像分类任务上的实验表明，SANER显著降低噪声标签记忆，且泛化性能优于SAM和SGD。此外，由于SANER源于SAM机制，可无缝集成至SAM类变体中，进一步提升其鲁棒性。
原文摘要 · Abstract (English)
Sharpness-Aware Minimization (SAM) was introduced to improve generalization by seeking flat minima, yet it also exhibits robustness to label noise, a phenomenon that remains only partially understood. Prior work has mainly attributed this effect to SAM's tendency to prolong the learning of clean samples. In this work, we provide a complementary explanation by analyzing SAM at the element-wise level. We show that when noisy gradients dominate a parameter direction, their influence is reduced by the stronger amplification of clean gradients. This slows the memorization of noisy labels while sustaining clean learning, offering a more complete account of SAM's robustness. Building on this insight, we propose SANER (Sharpness-Aware Noise-Explicit Reweighting), a simple variant of SAM that explicitly magnifies this down-weighting effect. Experiments on benchmark image classification tasks with noisy labels demonstrate that SANER significantly mitigates noisy-label memorization and improves generalization over both SAM and SGD. Moreover, since SANER is designed from the mechanism of SAM, it can also be seamlessly integrated into SAM-like variants, further boosting their robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。