让大模型学会区分哪些思考该保留、哪些可丢弃，提升推理可靠性。
State commitment learning: training language models to distinguish computation from memory
- 提出状态承诺学习，训练模型识别需保留的长期状态与可丢弃的临时计算。
- 在数学、逻辑推理等任务中，消除隐藏思考依赖后仍保持高准确率。
- 适合需要可信、可解释推理的场景，如科学问答与多轮工具使用。
推理型语言模型无法区分用于计算的标记与构成持久状态的标记：一旦生成，所有隐藏思维都会保留在上下文中并影响后续预测。这导致下游推理可能依赖于失败尝试、死胡同和私有草稿，这些不应被安全地依赖。本文将此现象重构为新训练目标——状态承诺学习：训练模型显式区分应作为持久状态保留的信息与可丢弃的临时计算。定义反事实标准‘持久状态充分性’，可衡量答案在擦除隐藏思维后是否仍可用。提出反事实擦除强化学习（CERL），在同一前缀下评估保留隐藏思维与擦除它们两种路径，并仅当擦除路径仍正确时给予奖励。引入擦除依赖协议，在数学、长链逻辑、科学问答及多轮工具使用任务中验证，CERL显著降低答案对隐藏思维的依赖，且不牺牲准确率，持续优于仅关注正确性的强化学习与长答案监督微调基线。
原文摘要 · Abstract (English)
Reasoning language models do not distinguish tokens used for computation from tokens that constitute persistent state: once generated, all hidden thoughts remain in context and influence future predictions. As a result, downstream reasoning may depend on failed attempts, dead ends, and private scratch work that should not be safely relied on later. We recast this phenomenon as a new training objective, state commitment learning: training models to explicitly distinguish information that should be committed as persistent state from temporary computation that can be discarded. We define a counterfactual criterion, persistent-state sufficiency, which makes it trainable and measurable whether an answer remains usable after hidden thoughts are erased. We then propose Counterfactual Erasure RL (CERL), which evaluates, under the same prefix, both a path that keeps hidden thoughts and a path that erases them, and gives reward only when the erasure path remains correct. We also introduce the Erasure Dependence Protocol and show across mathematics, long-chain logic, scientific QA, and multi-turn tool-use evaluation that CERL substantially reduces answer dependence on hidden thoughts without sacrificing accuracy, consistently outperforming correctness-only RL and long-answer SFT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。