arXiv:2508.09697cs.LGcs.CV2025-08被引 1

通过剪枝冗余连接,提升模型在标签噪声下的鲁棒性。

Label-Noise Resistant Learning via Optimal Brain Damage Masking

  • 基于最优脑损伤理论,动态屏蔽全连接层中低激活边。
  • 在合成与真实数据集上显著降低噪声标签导致的性能下降。
  • 可插拔集成至多种抗噪方法,尤其适配KAN等新型网络结构。

真实多媒体应用中标签噪声不可避免。由于深度神经网络具有强记忆能力,噪声标签会导致性能严重退化。现有抗噪方法主要聚焦于鲁棒损失函数和样本选择策略,对动态结构自适应探索有限。本文重新思考分类器连接在标签噪声下的作用:性能下降源于噪声梯度的反向传播。由于最终分类层是误差传播的主要通道,有选择地移除冗余连接可限制噪声梯度的传播路径。为此,我们借鉴模型压缩中的经典最优脑损伤(OBD)理论——若某参数引起的损失扰动可忽略,则可安全移除。基于此,我们证明:屏蔽低激活边能限制估计的损失扰动,同时收紧噪声引发梯度误差的上界。据此提出一种针对全连接层的新型选择性边掩码(SEM)机制,可自适应保留关键信息通路,抑制噪声标签带来的梯度误差。该模块为即插即用设计,可无缝融入多种抗噪方法。此外,我们将其应用于新提出的科莫戈罗夫-阿诺德网络(KAN)作为分类器。在合成与真实世界基准上的大量实验表明,本方法达到当前最佳性能。

原文摘要 · Abstract (English)

Noisy labels are inevitable in real-world multimedia applications. Due to the strong memorization capacity of deep neural networks, these noisy labels cause significant performance degradation. Existing noise-robust methods have mainly focused on robust loss functions and sample selection strategies, with comparatively limited exploration of dynamic architectural adaptation. In this paper, we rethink the role of classifier connectivity under label noise. Intuitively, performance degradation stems from the backpropagation of noisy gradients. Since the final classifier layer acts as the primary gateway for this error propagation, selectively discarding redundant connections can restrict the backpropagation pathways of noisy gradients. Consequently, to identify redundant connections, we leverage the seminal Optimal Brain Damage (OBD) theory from model compression, which posits that parameters causing negligible loss perturbation can be removed. Guided by this principle, we show that masking low-activation edges limits the estimated loss perturbation to preserve the model's fitting capacity, while tightening a derived upper bound on noise-induced gradient error. To bridge this theoretical insight with practical training, we propose a novel Selective Edge Masking (SEM) mechanism for the fully connected (FC) layer to enhance noise robustness. It can adaptively retain critical edges for information propagation while suppressing gradient errors caused by noisy labels. As a plug-and-play component, SEM can be seamlessly integrated into various noise-robust methods. Additionally, we validate SEM's applicability by applying it to the newly emerged Kolmogorov-Arnold Network (KAN) employed as a classifier. Extensive evaluations on synthetic and real-world benchmarks demonstrate that our approach achieves state-of-the-art performance.

标签噪声模型剪枝鲁棒学习KAN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。