通过激活签名实现实体级遗忘，让模型既不说出目标又抹除内部表征。
Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
- 挖掘特定实体的激活签名，抑制其对应激活方向。
- 在TOFU数据集上遗忘率99%、效用损失仅3%，且内部激活几乎消失。
- 适合关注模型内部表征安全的开发者与研究者。
现有实体级遗忘评估依赖输出表现：模型是否停止提及目标、拒绝查询或改变真实率分布。但这些测试无法反映内部表征是否被削弱。本文提出实体表征遗忘框架（ERUF），通过挖掘特定实体的激活签名，抑制其激活方向，并将行为蒸馏至LoRA参数中。在多个基线中，唯有ERUF同时实现表面抑制、内部衰减与性能保留。在TOFU forget10上，ERUF达成FQ=0.99、MU=0.62，接近理想遗忘效果且保持高可用性。在多数主流大模型设置下，泄漏极低，内部目标激活维持在SMR 0.00%~1.10%、EL10<0.06、效用漂移<3%。在Llama-3.1-8B上，对抗性实体恢复率从63.89%降至20.15%，无名恢复率下降72.7%至77.4%。联合诊断揭示推理优先模型存在尺度依赖行为，仅表面指标无法捕捉。结果表明表征层面衰减，非形式化保证删除。
原文摘要 · Abstract (English)
Entity-level unlearning is usually evaluated by what a model says: whether it stops naming the target, refuses a query, or shifts a Truth Ratio distribution. These output-level tests, however, do not show whether a subject's internal representation has been attenuated. We introduce the Entity Representation Unlearning Framework (ERUF), a representation-aware framework that mines subject-specific activation signatures, suppresses the corresponding activation direction, and distills the behavior into LoRA parameters. Among evaluated baselines, ERUF is the only method that jointly achieves surface-level suppression, internal attenuation, and utility preservation. On TOFU forget10, ERUF achieves FQ = 0.99 and MU = 0.62, matching reported oracle utility while approaching oracle forget quality. Across most standard foundation-model settings, ERUF maintains low leakage and low internal target activation, with SMR between 0.00% and 1.10%, EL10 below 0.06, and utility drift below 3%. On Llama-3.1-8B, adversarial entity recovery falls from 63.89% to 20.15%, while name-agnostic recovery decreases by 72.7% to 77.4%. Joint surface/internal diagnostics further reveal scale-dependent behavior in reasoning-prior models that surface metrics alone would miss. We interpret these results as operational evidence of representation-level attenuation, not as a formal guarantee of irreversible deletion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。