PURGE通过保留引导遗忘,实现高效隐私保护的模型删减。
PURGE: Projected Unlearning via Retain-Guided Erasure

- 利用梯度投影约束遗忘过程,确保保留数据性能不下降。
- 多层表征擦除使隐藏层特征贴近保留分布,深度消除敏感信息。
- 自动停止机制无需调参,适合对隐私安全要求高的场景。
我们提出PURGE,一种基于持续学习与机器遗忘本质对偶性的遗忘算法。该方法借鉴A-GEM的梯度投影机制,确保每一步遗忘均不增加保留集损失;同时在多层中间表示上执行擦除,将遗忘集激活推向保留集分布,从隐藏表征层面彻底移除信息而非仅抑制输出。关键设计是保留混淆目标:不将遗忘输出推向均匀分布(易被成员推断攻击识别),而是模仿模型在保留数据上的自然混淆模式,使被遗忘模型难以与从头训练的模型区分。两个自调节停止条件(保留损失预算和遗忘准确率目标)使算法自主决定终止,避免人工调参。在五个数据集(CIFAR-10、MNIST、SVHN、STL10、PathMNIST)上22个类别级遗忘任务中,PURGE始终维持保留准确率高于96%,且成员推断攻击AUROC接近0.5(理想值),优于梯度上升、KL-均匀及多个已有基线,在隐私-效用权衡上表现更优。
原文摘要 · Abstract (English)
We propose PURGE, a machine unlearning algorithm built on a simple but an under-exploited observation: continual learning (CL) and machine unlearning (MU) which are fundamentally dual problems. CL tries to learn new tasks without forgetting old ones; MU tries to erase specific data without hurting retained performance representing the same underlying tension in opposite directions. PURGE leverages this duality by adapting gradient projection from A-GEM (Chaudhry et al., 2019) so that every unlearning step is constrained to not increase the retain-set loss. On top of this, it performs multi-layer representation erasure, pushing forget-set activations in intermediate layers towards the retain distribution to remove information from hidden representations rather than just suppressing it at the output. A key design choice is the retain-confusion target: rather than pushing forget outputs toward the uniform distribution, which we found to be surprisingly easy for membership inference attacks to detect, we instead target the model's natural confusion pattern on retain data. This makes the unlearned model hard to distinguish from one retrained from scratch. Two self-regulating stopping criteria (a retain-loss budget and a forget-accuracy target) let the algorithm decide on its own when to stop, removing the need for manual epoch tuning. In experiments on five datasets (CIFAR-10, MNIST, SVHN, STL10, PathMNIST) across 22 class-level forgetting tasks, PURGE consistently keeps retain accuracy above 96% while achieving MIA AUROC close to 0.5 (the ideal), outperforming gradient ascent, KL-uniform, and several published baselines on the privacy-utility frontier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。