检验解题痕迹记忆的审计方法，发现其结论可能误判模型是否真正遗忘。
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
- 用条件化蜜罐头测试解题过程中的记忆残留，验证遗忘效果。
- 更换解码模板后答案正确率骤降，表明原有结果可能受提示影响。
- 建议增加解码模板替换作为低成本验证手段，避免误判。
评估推理模型的遗忘效果时常出现绕过现象：答案看似已遗忘，但模型自身的思考轨迹仍会泄露被遗忘内容，该差异常被视为权重仍存记忆的证据。本文以 DeepSeek-R1-Distill-Qwen-7B 模型为基础，采用 LoRA 记忆虚构作者并进行 NPO 遗忘训练，通过六标记蜜罐头进行条件化测试。在第一个种子下，将思考轨迹替换为短非蜜罐前缀，答案正确率下降幅度与绕过差距相当，无论前缀是否模仿训练模板。在第二个种子下，绕过差距缩小而非消失，且前缀替换使正确率升至顶点。正向的解析器分叉差距并不能单独确认隐藏权重记忆，也无法排除。在另一蒸馏版本中，同一指标反转，因解析器无法找到闭合标签。建议在标准审计外增加解码时模板替换作为低成本校验。
原文摘要 · Abstract (English)
Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace keeps emitting the forgotten content, and the gap is taken as evidence that the weights still remember. We audit this reading on DeepSeek-R1-Distill-Qwen-7B with LoRA-memorized fictional authors and NPO unlearning, conditioned on a six-token canary head. On one seed, swapping the thinking trace for a short non-canary prefill on the same weights drops the answer rate by as much as the bypass gap itself, whether the prefill mimics the training template or not. On a second seed the bypass gap shrinks rather than vanishing, and the prefill swap reverses direction and brings the answer rate to ceiling. A positive parser-split bypass gap thus does not by itself identify hidden weight-level memorization, and does not rule it out either. On a different distillate the same metric flips sign because the parser cannot find the closing tag. We recommend a decode-time template swap as a cheap sanity check alongside the canonical audit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。