arXiv:2511.18660cs.LGstat.ML2025-11

无需原始数据,用少量污染样本修复模型被污染的影响。

Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic

  • 将干净与污染信号视为不同任务,通过代理样本微调生成修正向量。
  • 在标签噪声下恢复大部分性能,在后门攻击中几乎消除威胁。
  • 适合无法获取训练数据的现实场景,如隐私敏感应用。

训练数据中的污染现象普遍存在。纠正性机器遗忘(CMU)旨在训练后消除污染影响。以往方法通常假设可访问已识别的污染样本(即‘遗忘集’),但在许多真实场景中原始训练数据已不可用。本文提出源无依赖式CMU,即原始训练数据不可获取,无法指定遗忘集。我们假设存在少量反映污染类型的代理样本,无需与原数据一致。在此更严格设定下,依赖遗忘集的方法无效或范围受限。为此,我们提出任务空间中的纠正性遗忘(CUTS),一种基于任务算术原理的轻量级权重空间修正方法。CUTS将干净与污染信号视为独立任务:先在代理样本上对污染模型进行短时微调,放大权重空间中的污染机制;计算污染模型与微调后模型的权重差,作为代理任务向量;再减去该向量的校准倍数,以抵消污染影响。在无清洁数据或遗忘集的情况下,CUTS在标签噪声下恢复了大部分性能,在后门触发器攻击中近乎完全消除攻击效果,且对模型效用损伤极小,优于现有最先进专用CMU方法在源无依赖设置下的表现。

原文摘要 · Abstract (English)

Corrupted training data are ubiquitous. Corrective Machine Unlearning (CMU) seeks to remove the influence of such corruption post-training. Prior CMU typically assumes access to identified corrupted training samples (a "forget set"). However, in many real-world scenarios the training data are no longer accessible. We formalize source-free CMU, where the original training data are unavailable and, consequently, no forget set of identified corrupted training samples can be specified. Instead, we assume a small proxy (surrogate) set of corrupted samples that reflect the suspected corruption type without needing to be the original training samples. In this stricter setting, methods relying on forget set are ineffective or narrow in scope. We introduce Corrective Unlearning in Task Space (CUTS), a lightweight weight space correction method guided by the proxy set using task arithmetic principles. CUTS treats the clean and the corruption signal as distinct tasks. Specifically, we briefly fine-tune the corrupted model on the proxy to amplify the corruption mechanism in the weight space, compute the difference between the corrupted and fine-tuned weights as a proxy task vector, and subtract a calibrated multiple of this vector to cancel the corruption. Without access to clean data or a forget set, CUTS recovers a large fraction of the lost utility under label noise and, for backdoor triggers, nearly eliminates the attack with minimal damage to utility, outperforming state-of-the-art specialized CMU methods in source-free setting.

机器遗忘数据污染任务算术后门防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。