arXiv:2412.04898cs.CVcs.LG2024-12被引 1

用自监督预训练+迭代伪标签优化,提升噪声数据下的模型准确率

Mitigating Instance-Dependent Label Noise: Integrating Self-Supervised Pretraining with Pseudo-Label Refinement

  • 先用SimCLR自监督学习提取鲁棒特征,避开初始噪声标签
  • 通过多阶段筛选高置信样本,逐步修正伪标签提升质量
  • 在高噪声下表现显著优于现有方法,适合真实标注有误的场景

深度学习依赖大量标注数据实现高性能,但现实数据集常因人工误差、歧义或资源限制导致标签噪声。实例相关标签噪声(IDN)更普遍且难以处理,因其错误概率随输入特征变化。本文提出一种结合SimCLR自监督预训练与迭代伪标签精炼的混合框架:首先利用自监督学习建立对噪声不敏感的特征表示基础;随后采用迭代训练,通过多阶段策略识别高置信预测样本,并更新其标签以逐步提升标签质量。在含合成实例相关噪声的CIFAR-10和CIFAR-100数据集上评估,结果表明该方法在高噪声条件下显著优于多个先进方法,大幅提高分类准确率与鲁棒性。研究显示,自监督学习与迭代伪标签精炼的融合是应对实例相关标签噪声的有效策略。

原文摘要 · Abstract (English)

Deep learning models rely heavily on large volumes of labeled data to achieve high performance. However, real-world datasets often contain noisy labels due to human error, ambiguity, or resource constraints during the annotation process. Instance-dependent label noise (IDN), where the probability of a label being corrupted depends on the input features, poses a significant challenge because it is more prevalent and harder to address than instance-independent noise. In this paper, we propose a novel hybrid framework that combines self-supervised learning using SimCLR with iterative pseudo-label refinement to mitigate the effects of IDN. The self-supervised pre-training phase enables the model to learn robust feature representations without relying on potentially noisy labels, establishing a noise-agnostic foundation. Subsequently, we employ an iterative training process with pseudo-label refinement, where confidently predicted samples are identified through a multistage approach and their labels are updated to improve label quality progressively. We evaluate our method on the CIFAR-10 and CIFAR-100 datasets augmented with synthetic instance-dependent noise at varying noise levels. Experimental results demonstrate that our approach significantly outperforms several state-of-the-art methods, particularly under high noise conditions, achieving notable improvements in classification accuracy and robustness. Our findings suggest that integrating self-supervised learning with iterative pseudo-label refinement offers an effective strategy for training deep neural networks on noisy datasets afflicted by instance-dependent label noise.

标签噪声自监督学习伪标签模型鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。