用互信息检测数据中的错误标签和损坏样本，提升模型训练质量。
Detecting Mislabeled and Corrupted Data via Pointwise Mutual Information
- 通过点互信息衡量样本输入与标签的统计依赖关系
- 低互信息样本多为噪声或错标，过滤后准确率最高提升15%
- 对输入扰动鲁棒，能区分语义有效与真实损坏数据
深度神经网络会记忆错误标签，因此数据质量对模型性能至关重要，但真实数据集常同时存在标签噪声和输入噪声。本文提出一种基于互信息的数据选择框架，用于混合噪声场景，量化输入与标签间的统计依赖关系。计算每个样本对整体互信息的点贡献，发现贡献较低的样本通常为噪声或错标。在不同合成噪声设置下的MNIST实验表明，该方法能有效过滤低质量样本。在标签污染情况下，仅使用高互信息样本训练，分类准确率相比随机采样最高提升15%。此外，该方法对良性输入修改具有鲁棒性，可保留语义有效的数据，同时过滤真正损坏的样本。
原文摘要 · Abstract (English)
Deep neural networks can memorize corrupted labels, making data quality critical for model performance, yet real-world datasets are frequently compromised by both label noise and input noise. This paper proposes a mutual information-based framework for data selection under hybrid noise scenarios that quantifies statistical dependencies between inputs and labels. We compute each sample's pointwise contribution to the overall mutual information and find that lower contributions indicate noisy or mislabeled instances. Empirical validation on MNIST with different synthetic noise settings demonstrates that the method effectively filters low-quality samples. Under label corruption, training on high-MI samples improves classification accuracy by up to 15\% compared to random sampling. Furthermore, the method exhibits robustness to benign input modifications, preserving semantically valid data while filtering truly corrupted samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。