arXiv:2412.00244cs.LG2024-12被引 1

提出模拟人类标注噪声的新方法,让深度学习模型测试更真实。

Robust Testing for Deep Learning using Human Label Noise

  • 基于特征聚类生成与数据相关的仿真人类噪声
  • 现有抗噪方法在新测试下性能显著下降
  • 适合评估模型鲁棒性及改进抗噪算法的研究者

深度学习模型常因训练数据中的标签噪声而性能下降,现有学习带噪声标签(LNL)方法多在合成噪声下测试,但实际中人类标注噪声更复杂且更具破坏性。本文通过分析CIFAR-10N数据集的标签记忆现象,提出基于聚类的噪声生成方法(CBN),模拟真实人类标注错误的特征依赖性。实验表明,当前LNL方法在CBN生成的噪声下表现明显恶化,证明其测试强度更高。为此,我们进一步提出软邻近标签采样(SNLS)方法,有效应对此类更复杂的噪声,在多个基准上优于现有技术。

原文摘要 · Abstract (English)

In deep learning (DL) systems, label noise in training datasets often degrades model performance, as models may learn incorrect patterns from mislabeled data. The area of Learning with Noisy Labels (LNL) has introduced methods to effectively train DL models in the presence of noisily-labeled datasets. Traditionally, these methods are tested using synthetic label noise, where ground truth labels are randomly (and automatically) flipped. However, recent findings highlight that models perform substantially worse under human label noise than synthetic label noise, indicating a need for more realistic test scenarios that reflect noise introduced due to imperfect human labeling. This underscores the need for generating realistic noisy labels that simulate human label noise, enabling rigorous testing of deep neural networks without the need to collect new human-labeled datasets. To address this gap, we present Cluster-Based Noise (CBN), a method for generating feature-dependent noise that simulates human-like label noise. Using insights from our case study of label memorization in the CIFAR-10N dataset, we design CBN to create more realistic tests for evaluating LNL methods. Our experiments demonstrate that current LNL methods perform worse when tested using CBN, highlighting its use as a rigorous approach to testing neural networks. Next, we propose Soft Neighbor Label Sampling (SNLS), a method designed to handle CBN, demonstrating its improvement over existing techniques in tackling this more challenging type of noise.

标签噪声模型鲁棒性数据生成深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。