提出一种无需重训练的删类方法,精准移除特定类别信息。
Class Unlearning via Depth-Aware Removal of Forget-Specific Directions

- 通过投影法在各层移除与遗忘类相关的特征方向
- 在多个数据集和模型上接近重训练效果,保留原类性能
- 适用于单类或多类遗忘,且不依赖梯度优化
机器遗忘旨在不重新训练的前提下移除模型中的目标知识。然而,在类别遗忘中,降低遗忘类的准确率并不等于真正遗忘:遗忘信息可能仍编码在内部表示中,表面遗忘可能是分类头抑制所致而非表征删除。现有方法常表现出弱或负选择性,保留深层表示中遗忘类结构,或严重依赖最终层偏置调整。我们提出DAMP(深度感知投影调制),一种一次性、闭式权重手术方法,可在不使用梯度优化的情况下从预训练网络中移除遗忘特定方向。每层中,DAMP在下一可学习算子的输入空间计算类别原型,将遗忘方向作为相对于保留类原型的残差提取,并通过投影更新降低下游对这些方向的敏感性。为保持实用性,DAMP采用基于探测可分性的无参数深度感知缩放规则,早期层编辑幅度小,深层编辑幅度大。该方法可通过低秩子空间移除自然扩展至多类遗忘。在MNIST、CIFAR-10、CIFAR-100和Tiny ImageNet上,涵盖卷积与Transformer架构,DAMP表现更接近重训练标准,提升选择性遗忘,更好保留保留类性能,并减少深层中残留的遗忘类结构。
原文摘要 · Abstract (English)
Machine unlearning aims to remove targeted knowledge from a trained model without the cost of retraining from scratch. In class unlearning, however, reducing accuracy on forget classes does not necessarily imply true forgetting: forgotten information can remain encoded in internal representations, and apparent forgetting may arise from classifier-head suppression rather than representational removal. We show that existing class-unlearning methods often exhibit weak or negative selectivity, preserve forget-class structure in deep representations, or rely heavily on final-layer bias shifts. We then introduce DAMP (Depth-Aware Modulation by Projection), a one-shot, closed-form weight-surgery method that removes forget-specific directions from a pretrained network without gradient-based optimization. At each stage, DAMP computes class prototypes in the input space of the next learnable operator, extracts forget directions as residuals relative to retain-class prototypes, and applies a projection-based update to reduce downstream sensitivity to those directions. To preserve utility, DAMP uses a parameter-free depth-aware scaling rule derived from probe separability, applying smaller edits in early layers and larger edits in deeper layers. The method naturally extends to multi-class forgetting through low-rank subspace removal. Across MNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet, and across convolutional and transformer architectures, DAMP more closely resembles the retraining gold standard than some of the prior methods, improving selective forgetting while better preserving retain-class performance and reducing residual forget-class structure in deep layers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。