针对推理模型中的隐私泄露问题，提出首个专用评估基准与有效遗忘方法。
R-TOFU: Unlearning in Large Reasoning Models
- 构建包含多步推理链的评估基准，精准检测隐藏知识残留。
- 发现传统方法仅看答案会遗漏大量遗忘痕迹，导致隐私泄露风险。
- 新方法在保持推理连贯性的同时实现更强遗忘效果，适合需要安全推理的场景。
大型推理模型不仅在最终答案中嵌入隐私或受版权保护的信息，还在多步推理链（CoT）过程中保留这些内容，使得可靠遗忘比标准大模型更为困难。本文提出首个专为此场景设计的基准测试 R-TOFU，通过添加真实推理链标注并提供逐步骤评估指标，揭示了仅靠答案层面检查无法发现的残留知识。基于 R-TOFU，我们系统比较了基于梯度和偏好优化的多种基线方法，发现仅以答案为目标的训练仍会留下显著的遗忘痕迹。为此，我们提出一种新的偏好优化方法 Reasoned IDK，可在保持推理连贯性的同时呈现不确定结论，实现遗忘效率与模型可用性的更好平衡。此外，我们识别出一种失效模式：如 ZeroThink、LessThink 等解码策略即便在表面看来已成功遗忘，仍可能暴露被遗忘内容，凸显需在多样解码设置下评估模型。该基准、分析与新基线共同为研究与改进推理模型的遗忘能力提供了系统性基础。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) embed private or copyrighted information not only in their final answers but also throughout multi-step chain-of-thought (CoT) traces, making reliable unlearning far more demanding than in standard LLMs. We introduce Reasoning-TOFU (R-TOFU), the first benchmark tailored to this setting. R-TOFU augments existing unlearning tasks with realistic CoT annotations and provides step-wise metrics that expose residual knowledge invisible to answer-level checks. Using R-TOFU, we carry out a comprehensive comparison of gradient-based and preference-optimization baselines and show that conventional answer-only objectives leave substantial forget traces in reasoning. We further propose Reasoned IDK, a preference-optimization variant that preserves coherent yet inconclusive reasoning, achieving a stronger balance between forgetting efficacy and model utility than earlier refusal styles. Finally, we identify a failure mode: decoding variants such as ZeroThink and LessThink can still reveal forgotten content despite seemingly successful unlearning, emphasizing the need to evaluate models under diverse decoding settings. Together, the benchmark, analysis, and new baseline establish a systematic foundation for studying and improving unlearning in LRMs while preserving their reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。