模型学不会依赖搜索的推理，因为无法生成可验证的逐步推导。
A Verifiable Search Is Not a Learnable Chain-of-Thought
- 用程序反推解题步骤并微调模型，但搜索类任务无法有效学习。
- 在密码算术题上，不同方法下准确率仅0.01-0.07，远低于真实解法的71%。
- 只有将搜索转为查表+验证，才能让模型真正学会，适合结构化任务优化场景。
人们常认为只要任务能用短程序解决，就能通过链式思维训练模型掌握：写出步骤，微调即可。本文证明该假设在特定任务中失效。实验基于九个确定性生成器的任务，公开与隐藏数据共享生成器，以留出集测试精度。我们逆向工程生成器为Python求解器，转化为链式思维，并蒸馏为30B（3.5B活跃参数）Nemotron模型上的≤32秩LoRA。可正向计算的任务（如查表、算术、8位布尔任务）迁移成功（准确率≥0.99和0.68）。但密码算术任务失败：无论采用十一种链式设计、可验证奖励的强化学习或自训练，准确率均维持在0.01-0.07，尽管搜索求解器可答对71%。这并非能力不足：模型在97%-100%行上完成算术，71%情况下将正确密码排在前八名，却无法将其作为左到右推导延续。微调使模型学习到可验证的消除步骤，但其判断变为无条件模板，仅16%-57%正确（‘判决即令牌’）。此上限在3B至671B模型及微调与提示中均成立。控制实验表明：揭示密钥可使推理向前推进，准确率从0.03升至0.57。当任务唯一解法是遍历信息无关结构的搜索时，不存在忠实的前向链式思维可供模仿。任务仅可通过移除搜索、将组合核心预计算为目录，将过程简化为回忆加验证而变得可学，第一名方案因此达私有排行榜0.92。真正被蒸馏的是记忆与验证，而非搜索本身。
原文摘要 · Abstract (English)
It is tempting to assume any task solvable by a short program can be taught to a model as its chain-of-thought: write the steps out, fine-tune, and the model follows. This paper shows the assumption fails for an identifiable class of procedures. The testbed is nine reasoning tasks, each from a deterministic generator; public and hidden splits share generators, so held-out data proxies test accuracy. I reverse-engineer the generators into Python solvers, render them as chain-of-thought, and distill into a rank-<= 32 LoRA over a 30B (3.5B-active) Nemotron model. Forward-computable tasks install readily: lookup/arithmetic and an 8-bit boolean task transfer (>= 0.99 and 0.68). Cryptarithm does not: distilling its backtracking search holds at 0.01-0.07 across eleven chain-of-thought designs, RL from verifiable rewards, and self-training, even though a search solver answers 71% of instances. This is not a capability gap. The model does the arithmetic on 97-100% of lines and ranks the correct cipher in its top eight on 71%; it cannot carry the search forward as a left-to-right derivation. Fine-tuning learns the shape of a verifiable elimination step while its verdicts become unconditional templates, correct only 16-57% of the time ("verdict-as-token"). The ceiling holds across backbones from 3B to 671B and across fine-tuning and prompting; a controlled intervention isolates the cause: revealing the cipher key, which turns the derivation forward, lifts the same instances from 0.03 to 0.57. When a procedure's only solution is search over information-free structure, no faithful forward chain-of-thought exists to imitate. The task becomes learnable only by removing the search, precomputing its combinatorial core into a catalog and reducing the trace to recall plus verification; the 1st-place solution reaches Private LB 0.92 this way. What distills is memorization and verification, not search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。