arXiv:2506.11615cs.LG2025-06被引 2

通过归因引导的去噪与神经元剪枝,让深度模型在噪声数据中更鲁棒且省时。

Machine Unlearning for Robust DNNs: Attribution-Guided Partitioning and Neuron Pruning in Noisy Environments

  • 用梯度归因区分优质与污染数据,不依赖噪声假设。
  • 通过敏感性分析识别并剪掉易受噪声影响的神经元。
  • 仅在高质量数据上微调,准确率提升10%,训练时间减少47%。

深度神经网络在多个领域表现卓越,但其性能易受噪声或损坏训练数据的影响。传统去噪方法常依赖对噪声分布的明确假设,或需大量重训练,对大规模模型不实用。受机器遗忘思想启发,我们提出一种新框架,融合归因引导的数据划分、判别性神经元剪枝与针对性微调,以缓解噪声样本的影响。该方法利用基于梯度的归因概率性区分高质量样本与可能被污染的样本,无需对噪声施加严格假设;随后通过回归敏感性分析识别并剪除最易受噪声干扰的神经元;最后在高质量数据子集上微调模型,高效恢复并增强泛化能力。该集成式框架相比传统噪声鲁棒学习方法具多项优势:结合数据级遗忘与模型级适应,避免全模型重训练或显式建模噪声。我们在不同噪声水平下评估了该方法在典型任务(如CIFAR-10图像分类与语音识别)上的表现,结果表明准确率与效率均有显著提升。例如,在引入标签噪声的CIFAR-10上,本方法比标准重训练高出约10%的绝对准确率,且在某些场景下将重训练时间减少高达47%。这些结果证明了该方法在噪声环境中实现鲁棒泛化的有效性与可扩展性。

原文摘要 · Abstract (English)

Deep neural networks (DNNs) have achieved remarkable success across diverse domains, but their performance can be severely degraded by noisy or corrupted training data. Conventional noise mitigation methods often rely on explicit assumptions about noise distributions or require extensive retraining, which can be impractical for large-scale models. Inspired by the principles of machine unlearning, we propose a novel framework that integrates attribution-guided data partitioning, discriminative neuron pruning, and targeted fine-tuning to mitigate the impact of noisy samples. Our approach employs gradient-based attribution to probabilistically distinguish high-quality examples from potentially corrupted ones without imposing restrictive assumptions on the noise. It then applies regression-based sensitivity analysis to identify and prune neurons that are most vulnerable to noise. Finally, the resulting network is fine-tuned on the high-quality data subset to efficiently recover and enhance its generalization performance. This integrated unlearning-inspired framework provides several advantages over conventional noise-robust learning approaches. Notably, it combines data-level unlearning with model-level adaptation, thereby avoiding the need for full model retraining or explicit noise modeling. We evaluate our method on representative tasks (e.g., CIFAR-10 image classification and speech recognition) under various noise levels and observe substantial gains in both accuracy and efficiency. For example, our framework achieves approximately a 10% absolute accuracy improvement over standard retraining on CIFAR-10 with injected label noise, while reducing retraining time by up to 47% in some settings. These results demonstrate the effectiveness and scalability of the proposed approach for achieving robust generalization in noisy environments.

深度学习噪声鲁棒模型压缩机器遗忘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。