用自蒸馏让大模型学会专家推理，只需千条高质量解题步骤。
Making Expert Reasoning Learnable with Self-Distillation
- 先将专家解法转化为模型可学的详细推理链，再用对比学习聚焦核心思路。
- 仅用不到1000条专家解法，推理准确率提升31%，效率翻倍。
- 适合想低成本提升模型逻辑能力的研究者和工程落地团队。
提升大语言模型的推理能力通常依赖模型自身采样正确解进行强化，或依赖更强模型提供监督信号。然而，许多难题对当前前沿模型仍难以求解，导致有效训练信号缺失。一个有前景的替代方案是利用高质量的专家人类解法，但直接模仿这类数据效果不佳，因为其本质是面向人类的、包含隐含推理空白的讲解式文本，与模型分布不一致。此外，高质量专家解法成本高昂，亟需通用且高效的样本利用方法。本文提出分布对齐模仿学习（DAIL），一种两阶段自蒸馏方法：首先将专家解法转化为详细、符合模型分布的推理轨迹，再通过对比损失聚焦于专家的洞察与方法论。实验表明，DAIL仅需少于1000条高质量专家解法，即可在Qwen2.5-Instruct和Qwen3上实现最高31%的pass@128提升，推理效率翻倍，并支持跨领域泛化。
原文摘要 · Abstract (English)
Improving the reasoning capabilities of large language models (LLMs) typically relies either on the model's ability to sample a correct solution to be reinforced or the existence of a stronger model able to solve the problem. However, many difficult problems remain intractable for even current frontier models, preventing the extraction of valid training signals. A promising alternative is to leverage high-quality expert human solutions, yet naive imitation of this data fails because it is fundamentally out-of-distribution: expert solutions are typically didactic, containing implicit reasoning gaps intended for human readers rather than computational models. Furthermore, high-quality expert solutions are expensive, necessitating generalizable, sample-efficient training methods. We propose Distribution Aligned Imitation Learning (DAIL), a two-step self-distillation method that bridges the distributional gap by first transforming expert solutions into detailed, in-distribution reasoning traces and then applying a contrastive objective to focus learning on expert insights and methodologies. We find that DAIL can leverage fewer than 1000 high-quality expert solutions to achieve up to 31% pass@128 gains on Qwen2.5-Instruct and Qwen3, double reasoning efficiency, and enable out-of-domain generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。