无需干净数据集,通过跨层信息一致性检测噪声样本
CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels
- 利用最后一层与倒数第二层特征结构一致性判断样本好坏
- 在合成与真实噪声下均超越现有最优方法
- 适合无干净标签数据的噪声标签学习场景
带噪声标签的学习(LNL)对使用不完美数据训练深度神经网络至关重要。元学习方法虽成功依赖于干净无偏的标签数据集来训练鲁棒模型,但该方法严重依赖可获得的干净标签元数据集,在实际中难以实现。本文针对无干净标签数据集的噪声标签场景提出元学习新策略。我们的方法基于数据自身特性,不依赖标签信息。核心思想是:干净样本能保持最后隐藏层与最终层之间相关数据结构的一致性,而噪声样本会破坏这种一致性。为此,我们设计了基于跨层信息分歧的元更新策略(CLID-MU),通过分析不同特征空间间的数据结构对齐程度评估模型性能,并以此指导训练过程。在多个基准数据集上,针对不同噪声比例的合成与真实噪声场景进行实验,结果表明CLID-MU优于当前最先进方法。代码已公开于https://github.com/ruofanhu/CLID-MU。
原文摘要 · Abstract (English)
Learning with noisy labels (LNL) is essential for training deep neural networks with imperfect data. Meta-learning approaches have achieved success by using a clean unbiased labeled set to train a robust model. However, this approach heavily depends on the availability of a clean labeled meta-dataset, which is difficult to obtain in practice. In this work, we thus tackle the challenge of meta-learning for noisy label scenarios without relying on a clean labeled dataset. Our approach leverages the data itself while bypassing the need for labels. Building on the insight that clean samples effectively preserve the consistency of related data structures across the last hidden and the final layer, whereas noisy samples disrupt this consistency, we design the Cross-layer Information Divergence-based Meta Update Strategy (CLID-MU). CLID-MU leverages the alignment of data structures across these diverse feature spaces to evaluate model performance and use this alignment to guide training. Experiments on benchmark datasets with varying amounts of labels under both synthetic and real-world noise demonstrate that CLID-MU outperforms state-of-the-art methods. The code is released at https://github.com/ruofanhu/CLID-MU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。