arXiv:2608.30046cs.LGcs.AI2026-08中稿 · SIBGRAPI'26

对比五种机器遗忘方法,发现纠错效果取决于噪声类型。

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

  • 按噪声类型选择遗忘策略,简单微调在多数场景有效
  • 随机重标与显著性遗忘在实例相关噪声下接近重训精度
  • 开放集噪声下重训反而更差,需重新评估目标

噪声标签是深度神经网络训练中的关键挑战,因错误标签的过拟合会损害泛化能力。模型训练后识别出噪声样本时,传统方案是基于清洗数据集从头重训,但随着数据集和模型规模扩大,成本日益高昂。机器遗忘(MU)作为计算高效的替代方案近年兴起,但不同MU策略在噪声修正中的相对有效性仍不明确。本文在CIFAR-10、CIFAR-100及真实世界噪声数据集Food-101N上,对五种MU方法(NegGrad、Fine-Tuning(FT)、Random Labeling(RL)、SalUn、MUNBa)在对称、非对称、实例相关和开放集噪声下进行对比实证研究。核心发现:合适遗忘策略依赖于噪声结构。在多数封闭集场景中,简单微调表现强劲;在实例相关噪声下,随机重标与显著性遗忘可接近重训精度,且计算成本仅为其十分之一;极端对称噪声下,MUNBa更具优势。然而在开放集噪声中,仅对清洗子集重训会导致准确率低于原始噪声基线,因此逼近重训模型不再是合理目标。在Food-101N上,所有MU方法均保持竞争力,尽管运行时间缩短一个数量级,准确率仍接近重训结果。这些发现为训练后噪声标签修正提供了实用指导。

原文摘要 · Abstract (English)

Noisy labels remain a critical challenge for training deep neural networks, since memorizing incorrect labels degrades generalization. Once noisy samples are identified after training, the standard solution is to retrain the model from scratch on the cleaned dataset, which is increasingly expensive as datasets and models grow. Machine Unlearning (MU) has recently emerged as a computationally efficient alternative, but the relative effectiveness of different MU strategies for noisy-label correction remains poorly understood. In this work, we conduct a comparative empirical study of five MU methods (NegGrad, Fine-Tuning (FT), Random Labeling (RL), SalUn, and MUNBa) across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N. Our central finding is that the appropriate unlearning strategy is conditioned on the noise structure. Simple FT is a strong baseline across most closed-set scenarios; RL and SalUn are the most consistently robust methods and, under instance-dependent noise, approach retraining accuracy at a fraction of the computational cost; MUNBa shows advantages mainly under extreme symmetric noise. Under open-set noise, in contrast, we show that retraining on the cleaned subset degrades accuracy relative to the noisy baseline, so approximating the retrained model is not an adequate objective in this regime. On Food-101N, all MU methods remain competitive and achieve accuracies close to retraining despite reducing runtime by an order of magnitude. These findings provide practical guidelines for selecting MU strategies for post-training noisy-label correction.

机器遗忘噪声标签模型优化实证研究

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。