发现模型遗忘有邻近效应，提出局部教师蒸馏缓解此问题。
Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning
- 通过对比删除后重训练与未重训模型预测差异，发现遗忘影响具局部聚集性。
- 在CIFAR-100部分类别删除任务中，邻近遗忘样本处的预测误差显著升高。
- 新方法用邻近保留样本训练小教师生成软标签，提升局部还原精度。
机器遗忘旨在移除特定训练样本的影响而无需完整重训练。标准评估常使用聚合指标（如准确率和遗忘得分），可能掩盖局部失效问题。本文通过逐样本比较遗忘模型与删除后重训练模型的预测结果，揭示点级差异高度非均匀：梯度上升与随机标签方法（无论是否使用保留集微调）的误差随距遗忘集的几何距离增大而增加。我们称此现象为局部旁生遗忘。分析表明，其机制在于遗忘过程中使用的代理目标与重训练所形成的局部预测结构不一致，该不一致性通过共享表示传播至邻近样本。为此，我们提出局部教师蒸馏：用仅在遗忘集邻近保留样本上训练的小教师生成软标签，替代随机目标。在CIFAR-100部分类别删除任务中，该方法使遗忘模型在遗忘集附近更接近重训练结果，同时保持良好的整体遗忘指标。
原文摘要 · Abstract (English)
Machine unlearning aims to remove the influence of selected training examples without full retraining. Standard evaluations often summarize unlearning quality with aggregate metrics, such as accuracy- and forgetting-based scores, which can hide localized failures. We study this failure mode at the example level by comparing the predictions of an unlearned model to those of the model retrained after deletion. We show that this pointwise discrepancy can be highly non-uniform: for gradient-ascent and random-labeling methods, with and without retain-set fine-tuning, it grows with geometric proximity to the forget set. We call this phenomenon localized collateral forgetting. Our analysis identifies a mechanism behind the effect: surrogate targets used during unlearning can be inconsistent with the local prediction structure induced by retraining, and this inconsistency propagates through shared representations to nearby examples. Motivated by this mechanism, we propose Local Teacher Distillation, a simple mitigation strategy that replaces random targets with soft labels from a small teacher trained only on retained neighbors of the forget set. On CIFAR-100 partial-class deletion, this local teacher brings the unlearned model substantially closer to retraining, especially near the forget set, while maintaining competitive aggregate unlearning metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。