忽略标签噪声反而更优，理论证明无须纠正噪声即可实现最佳性能。
Label Noise: Ignorance Is Bliss
- 将标签噪声建模为后验漂移的域适应问题，提出相对信号强度度量。
- 理论证明忽略噪声的最小化经验风险在多类别场景下接近最优。
- 实践中用自监督特征提取+线性分类器，在CIFAR-N上达到顶尖表现。
我们建立了一个新的多类别、实例依赖型标签噪声学习的理论框架。该框架将带标签噪声的学习视为一种域适应，特别是后验漂移下的域适应。引入了相对信号强度（RSS）这一逐点度量，用于量化从含噪后验到清洁后验的可迁移性。基于RSS，我们建立了过剩风险的几乎匹配上下界。理论结果支持简单的‘噪声无关经验风险最小化’（NI-ERM）原则，即在忽略标签噪声的前提下最小化经验风险。最后，我们将这一理论洞见转化为实践：通过使用NI-ERM在自监督特征提取器之上拟合线性分类器，在CIFAR-N数据挑战中取得了当前最优性能。
原文摘要 · Abstract (English)
We establish a new theoretical framework for learning under multi-class, instance-dependent label noise. This framework casts learning with label noise as a form of domain adaptation, in particular, domain adaptation under posterior drift. We introduce the concept of \emph{relative signal strength} (RSS), a pointwise measure that quantifies the transferability from noisy to clean posterior. Using RSS, we establish nearly matching upper and lower bounds on the excess risk. Our theoretical findings support the simple \emph{Noise Ignorant Empirical Risk Minimization (NI-ERM)} principle, which minimizes empirical risk while ignoring label noise. Finally, we translate this theoretical insight into practice: by using NI-ERM to fit a linear classifier on top of a self-supervised feature extractor, we achieve state-of-the-art performance on the CIFAR-N data challenge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。