提出新方法实现更可靠的机器遗忘,提升安全性与准确性。
Toward Reliable Machine Unlearning: Theory, Algorithms, and Evaluation
- 通过对抗样本微调降低遗忘样本置信度,提升遗忘效果。
- 引入FastClip技术控制模型平滑性,增强鲁棒性。
- 针对类别遗忘设计新目标函数,有效避免信息泄露,适合安全敏感场景。
本文提出了针对随机样本遗忘和类别遗忘的新方法,均优于现有技术。核心思路是使模型在遗忘和保留样本上的预测结果与重新训练后的模型保持一致。提出对抗机器遗忘(AMUN)方法,通过在遗忘样本的对抗样本上微调,降低其置信度,在图像分类任务中基于最先进的成员推断攻击(MIA)得分超越现有最优方法。理论分析揭示模型平滑性是影响性能的关键因素,为此提出FastClip方法,对仿射层进行逐层谱范数裁剪,实现可扩展的平滑模型训练。进一步研究发现,更高平滑性自然提升对抗样本迁移能力,支持上述结论。针对类别遗忘,提出近邻成员推断攻击(MIA-NN),证明现有方法无法准确复现重新训练模型的行为,存在信息泄漏。为此设计一种微调目标,通过估计类间相似性并调整目标分布,使遗忘类输入的输出分布逼近从头训练模型的剩余类别分布。该分布称为倾斜重加权(TRW)分布,在多个基准测试中,TRW在原有指标上达到或超过现有方法性能。
原文摘要 · Abstract (English)
We propose new methodologies for both unlearning random set of samples and class unlearning and show that they outperform existing methods. The main driver of our unlearning methods is the similarity of predictions to a retrained model on both the forget and remain samples. We introduce Adversarial Machine UNlearning (AMUN), which surpasses prior state-of-the-art methods for image classification based on SOTA MIA scores. AMUN lowers the model's confidence on forget samples by fine-tuning on their corresponding adversarial examples. Through theoretical analysis, we identify factors governing AMUN's performance, including smoothness. To facilitate training of smooth models with a controlled Lipschitz constant, we propose FastClip, a scalable method that performs layer-wise spectral-norm clipping of affine layers. In a separate study, we show that increased smoothness naturally improves adversarial example transfer, thereby supporting the second factor above. Following the same principles for class unlearning, we show that existing methods fail in replicating a retrained model's behavior by introducing a nearest-neighbor membership inference attack (MIA-NN) that uses the probabilities assigned to neighboring classes to detect unlearned samples and demonstrate the vulnerability of such methods. We then propose a fine-tuning objective that mitigates this leakage by approximating, for forget-class inputs, the distribution over remaining classes that a model retrained from scratch would produce. To construct this approximation, we estimate inter-class similarity and tilt the target model's distribution accordingly. The resulting Tilted ReWeighting(TRW) distribution serves as the desired target during fine-tuning. Across multiple benchmarks, TRW matches or surpasses existing unlearning methods on prior metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。