对比推理蒸馏与标准微调，发现多数情况下后者更高效
Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation
- 用同一教师模型切换推理模式生成对比数据，控制变量研究
- 相同算力下，标准微调在18个基准上多数优于推理蒸馏
- 仅用25%-50%推理数据混合微调，即可接近高阶推理效果
从强教师模型中蒸馏推理路径已成为构建高效小模型的标准方法。然而推理路径比标准指令微调输出长5-20倍，这意味着选择推理蒸馏的实践者实际上放弃了在同一算力预算下训练更大规模的指令微调模型。这一权衡是否值得尚未得到解答。我们通过受控实验研究该问题：单一教师模型通过切换推理模式，对相同提示生成配对的指令微调与推理输出，仅将监督格式作为唯一变量。在五个模型规模（0.5B至14B）上训练学生模型，并在18个基准上评估。结果表明，在匹配浮点运算量（FLOPs）下，指令微调在多数配置中位于或接近帕累托前沿。推理仅在7B及以上规模的开放任务中达到帕累托前沿。即便如此，仅用25%-50%的推理数据与指令微调混合的顺序课程，即可获得大部分精度提升，且显著降低计算成本。
原文摘要 · Abstract (English)
Distilling reasoning traces from strong teacher models has become the standard recipe for building capable small language models. Yet reasoning traces are 5-20$\times$ longer than standard instruction fine-tuning (IFT) outputs, meaning every practitioner who chooses reasoning distillation implicitly forgoes training a larger IFT model on the same compute budget. Whether this trade-off is worthwhile remains unaddressed. We study it with a controlled experiment: a single teacher generates paired IFT and reasoning outputs for identical prompts by toggling only its reasoning mode, isolating supervision format as the sole variable. Training students at five scales (0.5B to 14B) and evaluating on 18 benchmarks, we find that at matched FLOPs, IFT lies on or near the Pareto frontier across the majority of configurations. Reasoning reaches the Pareto frontier only on open-ended tasks at 7B and above. Even there, a sequential curriculum mixing just 25-50\% reasoning data with IFT captures most of the accuracy benefit at far lower compute cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。