提出新方法检测模型是否真正遗忘数据，避免虚假安全
RULER: Representation-Level Verification of Machine Unlearning
- 从表示层验证遗忘效果，不依赖重训练
- 12种条件下10组测试发现残留信息显著存在
- 适用于图像、文本等多场景，可提前发现遗忘漏洞
机器遗忘旨在移除特定训练样本对已部署模型的影响，而无需从头训练。现有方法仅在输出层面通过成员推断、保留准确率和遗忘集准确率验证，但模型可能满足这三项指标却仍在其中间表示中编码被遗忘记录。本文提出RULER，一套表示层验证指标：M2通过与重新训练模型的表示位置对比，衡量遗忘记录是否仍在原位置；M4为无须重训练的自检指标，仅依赖模型内部相似性结构。四种近似遗忘方法均通过输出层评估，但在线性混合效应模型下，M2在12种条件中的10种检测到显著残留（p<0.05），效应量随遗忘比例增加而增大。第五种方法Bad Teacher虽机制不同，仍出现相同残留。M4可在表格、图像、临床文本及人脸身份识别场景中作为预遗忘诊断工具，成功检测出人脸识别模型中的身份级记忆残留，而当前方法均未完全消除该信号。
原文摘要 · Abstract (English)
Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy, but a model can satisfy all three whilst still encoding forgotten records in its intermediate representations. We introduce RULER, a set of representation-level verification metrics. The oracle-comparative metric M2 measures whether forget-set records occupy the same representational position as in a model retrained without them. The oracle-free metric M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining. Four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the forget fraction increases. A fifth method, Bad Teacher, shows the same residuals despite a different forgetting mechanism. M4 acts as a pre-unlearning diagnostic across tabular, image, clinical text, and face-identity settings: it detects identity-level memorisation in face recognition models where no tested method fully erases the signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。