arXiv:2604.08271cs.LG2026-04中稿 · AISTATS 2026被引 2

现有机器遗忘方法看似有效,实则存在表象欺骗,需从内部表征评估真伪。

An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representations

  • 通过分析模型内部表征,发现多数方法仅表面抹除数据影响。
  • 未遗忘数据的隐藏特征仍高度可区分,线性探测可恢复近原始准确率。
  • 提出基于类别均值特征的分类器,确保表征与分类器对齐,提升可信度。

尽管近期许多机器遗忘(MU)方法在擦除遗忘数据、类别或概念方面表现良好,但它们仍极易受干扰——例如简单微调可能意外重新引入被遗忘的概念。本文通过考察已遗忘模型的内部表征,而非以往主要关注输出行为的方法,揭示了这一矛盾根源:多数先进方法的成功主要源于最后一层特征与分类器之间的错位,我们称之为‘特征-分类器错位’。事实上,隐藏特征依然高度判别性,简单线性探测即可恢复接近原始准确率。假设原模型满足神经坍缩(neural collapse),我们进一步证明仅调整分类器即可实现极低遗忘准确率,同时保持保留准确率;实验验证了仅分类器微调的有效性。基于此,我们提出基于类别均值特征(CMF)分类器的遗忘方法,显式强制特征与分类器对齐。标准基准测试表明,基于CMF的遗忘方法能有效减少表示中的遗忘信息,同时保持高保留准确率,凸显了对机器遗忘进行真实表征级评估的必要性。

原文摘要 · Abstract (English)

While numerous machine unlearning (MU) methods have recently been developed with promising results in erasing the influence of forgotten data, classes, or concepts, they are also highly vulnerable-for example, simple fine-tuning can inadvertently reintroduce erased concepts. In this paper, we address this contradiction by examining the internal representations of unlearned models, in contrast to prior work that focuses primarily on output-level behavior. Our analysis shows that many state-of-the-art MU methods appear successful mainly due to a misalignment between last-layer features and the classifier, a phenomenon we call feature-classifier misalignment. In fact, hidden features remain highly discriminative, and simple linear probing can recover near-original accuracy. Assuming neural collapse in the original model, we further demonstrate that adjusting only the classifier can achieve negligible forget accuracy while preserving retain accuracy, and we corroborate this with experiments using classifier-only fine-tuning. Motivated by these findings, we propose MU methods based on a class-mean features (CMF) classifier, which explicitly enforces alignment between features and classifiers. Experiments on standard benchmarks show that CMF-based unlearning reduces forgotten information in representations while maintaining high retain accuracy, highlighting the need for faithful representation-level evaluation of MU.

机器遗忘表征分析模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。