用对比学习提升Transformer在噪声标签下的鲁棒性
Learning from Noisy Labels with Contrastive Co-Transformer
- 设计对比共变换框架，融合对比损失与分类损失
- 在Clothing1M等6个数据集上显著超越现有方法
- 可有效利用含噪声样本，适合弱监督场景
带有噪声标签的深度学习是弱监督学习中的重要挑战。尽管卷积神经网络具备强大学习能力，但在噪声标签样本存在时易过拟合。本文以经典的协同训练框架为基础，提出一种简单快速的对比共变换（Contrastive Co-Transformer）框架，显著提升性能。我们论证了变压器模型在处理标签噪声时的鲁棒性。该方法通过对比损失与分类损失联合训练，能够充分利用数据集中所有样本，无论其标签是否干净。在包含Clothing1M在内的六个标准基准数据集上的大量实验表明，所提方法优于现有最先进方法。
原文摘要 · Abstract (English)
Deep learning with noisy labels is an interesting challenge in weakly supervised learning. Despite their significant learning capacity, CNNs have a tendency to overfit in the presence of samples with noisy labels. Alleviating this issue, the well known Co-Training framework is used as a fundamental basis for our work. In this paper, we introduce a Contrastive Co-Transformer framework, which is simple and fast, yet able to improve the performance by a large margin compared to the state-of-the-art approaches. We argue the robustness of transformers when dealing with label noise. Our Contrastive Co-Transformer approach is able to utilize all samples in the dataset, irrespective of whether they are clean or noisy. Transformers are trained by a combination of contrastive loss and classification loss. Extensive experimental results on corrupted data from six standard benchmark datasets including Clothing1M, demonstrate that our Contrastive Co-Transformer is superior to existing state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。