通过迭代自推理蒸馏,让大模型推理更简洁高效。
Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

- 先用简短指令生成优质解题路径,再蒸馏成紧凑模型。
- 在MATH-500上准确率从86.2%提至92.0%,文本长度减半。
- 适合想压缩推理成本又不牺牲精度的开发者使用。
大型推理模型常依赖冗长的思维链(CoT)解决问题,但其中大量计算浪费在重复验证和无效路径上。现有方法通过让学生模型模仿自身采样的简洁前缀来降低开销,但存在初始化瓶颈:监督仅作用于已访问前缀,而初始模型生成的上下文常嘈杂冗余,导致教师难以纠正错误轨迹。本文提出BIRD(Bootstrapped Iterative Self-Reasoning Distillation),分两阶段优化:首先在简短指令下从基模型采样正确解题路径,仅保留答案正确的序列,并进行轻量级提示切换微调;随后在此预热模型基础上,采用简洁自教师进行反KL蒸馏,训练时使用更干净、信息量更高的前缀。在Qwen3系列模型上,BIRD在MATH-500与AIME基准上均优于提示工程与冷启动蒸馏。以Qwen3-8B为例,其在MATH-500上的准确率由86.2%提升至92.0%,平均响应长度从3,099降至1,115个词元。结果表明,前缀质量是高效推理蒸馏的核心因素。
原文摘要 · Abstract (English)
Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student's own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。