通过自适应软标签优化,高效提升噪声标签下的模型性能
Efficient Adaptive Label Refinement for Label Noise Learning
- 将纠错与学习解耦,用模型预测动态生成软标签
- 在CIFAR-10/100等数据集上准确率提升3.2%以上
- 无需先验噪声信息,适合实际噪声场景应用
深度神经网络易受噪声标签过拟合影响,导致性能下降。现有方法依赖人工设定准则,在每轮迭代中尝试最优划分以避免错误标签拟合,同时充分学习干净样本,但常导致模型过于复杂且难训练。为此,本文将避免错误拟合与充分学习解耦,提出一种简单高效的自适应标签精炼方法(ALR)。首先,受标签修复技术启发,利用模型预测将原始硬标签转化为软标签,降低错误拟合风险;随后引入熵损失,逐步将高置信度软标签“硬化”,引导模型更好学习干净样本。该方法无需噪声先验知识或辅助数据集,更易于部署。我们在含人工噪声的基准数据集(CIFAR-10/100)及含真实噪声的数据集(ANIMAL-10N、Clothing1M、WebVision)上验证了ALR的有效性,结果表明其性能优于当前最先进方法。
原文摘要 · Abstract (English)
Deep neural networks are highly susceptible to overfitting noisy labels, which leads to degraded performance. Existing methods address this issue by employing manually defined criteria, aiming to achieve optimal partitioning in each iteration to avoid fitting noisy labels while thoroughly learning clean samples. However, this often results in overly complex and difficult-to-train models. To address this issue, we decouple the tasks of avoiding fitting incorrect labels and thoroughly learning clean samples and propose a simple yet highly applicable method called Adaptive Label Refinement (ALR). First, inspired by label refurbishment techniques, we update the original hard labels to soft labels using the model's predictions to reduce the risk of fitting incorrect labels. Then, by introducing the entropy loss, we gradually `harden' the high-confidence soft labels, guiding the model to better learn from clean samples. This approach is simple and efficient, requiring no prior knowledge of noise or auxiliary datasets, making it more accessible compared to existing methods. We validate ALR's effectiveness through experiments on benchmark datasets with artificial label noise (CIFAR-10/100) and real-world datasets with inherent noise (ANIMAL-10N, Clothing1M, WebVision). The results show that ALR outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。