arXiv:2506.13715cs.LG2025-06中稿 · ICLR被引 5

SAM在模型遗忘中可提升性能,通过分离保留与遗忘信号增强鲁棒性。

Sharpness-Aware Machine Unlearning

  • 将SAM拆分为两部分:保留信号用SAM优化,遗忘信号用尖锐度最大化
  • 相比SGD,SAM在较少保留数据下仍保持性能,且降低成员推理攻击风险
  • 适用于严格样本级遗忘场景,尤其适合对抗过拟合的高噪声数据

我们分析了尖锐度感知最小化(SAM)在机器遗忘中的有效性,发现遗忘信号会干扰保留信号。尽管以往研究证明SAM能通过防止噪声记忆提升泛化能力,但本文指出,在拟合遗忘集时SAM会丧失这种去噪特性,导致泛化性能依赖于信号强度。我们进一步刻画了不同信号强度下的信号盈余,使得模型能在较少保留数据下维持性能,并更侧重遗忘操作。实验证明,SAM在保留数据要求更低的情况下仍优于SGD,且可作为预训练或遗忘算法提升多种方法表现。基于对SAM遗忘机制的细化理解,并观察到过拟合有助于更严格的样本级遗忘,我们提出Sharp MinMax:将模型分为两部分,分别用SAM学习保留信号、用尖锐度最大化实现遗忘,取得最优效果。大量实验显示,SAM在不同遗忘难度下均有效,减少保留与遗忘集间的特征纠缠,增强对成员推理攻击的抵抗力,且使损失曲面更平坦。该结论在更高噪声数据、不同优化器及架构下也成立。

原文摘要 · Abstract (English)

We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. While previous work prove that SAM improves generalization with noise memorization prevention, we show that SAM abandons such denoising property when fitting the forget set, leading to altered generalization depending on signal strength. We further characterize the signal surplus of SAM in the order of signal strength, which enables learning from less retain signals to maintain model performance and putting more weight on unlearning the forget set. Empirical studies show that SAM outperforms SGD with relaxed requirement for retain signals and can enhance various unlearning methods either as pretrain or unlearn algorithm. Motivated by our refined characterization of SAM unlearning and observing that overfitting can benefit more stringent sample-specific unlearning, we propose Sharp MinMax, which splits the model into two to learn retain signals with SAM and unlearn forget signals with sharpness maximization, achieving best performance. Extensive experiments show that SAM enhances unlearning across varying difficulties measured by memorization, yielding decreased feature entanglement between retain and forget sets, stronger resistance to membership inference attacks, and a flatter loss landscape. Our observations generalize to more noised data, different optimizers, and different architectures.

机器遗忘SAM泛化成员推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。