让小模型自己生成任务路径，大模型只纠正最早错误，显著提升小模型推理能力。
Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs
- 小模型生成推理过程，大模型仅修正首个错误，匹配小模型实际水平。
- 7B小模型经此方法后，性能接近72B大模型在12个基准上的表现。
- 适合想用小模型实现强推理能力的研究者和开发者。
大型语言模型代理在多步推理和工具使用任务中表现优异，但其能力通常依赖于极庞大的模型结构。现有蒸馏方法让小模型模仿大模型的完整推理轨迹，但因师生间存在推理与知识差距，导致误差不断累积。本文提出SCoRe框架：由小模型生成训练轨迹，大模型仅纠正最早的错误，生成符合小模型能力的训练数据并暴露其具体弱点。小模型先在修正后的轨迹上微调，随后从验证过的前缀开始进行短时程强化学习，目标奖励设置在最早错误处。该设计使小模型通过无约束的强化学习探索解决问题，短时程机制则提升了训练稳定性。在12个挑战性基准上，一个7B参数的小模型经SCoRe蒸馏后，其代理性能与72B参数教师模型相当。
原文摘要 · Abstract (English)
Large Language Model agents achieve strong performance on multi-step reasoning and tool-use tasks, but their impressive capabilities typically rely on extremely large backbones. Existing distillation approaches train smaller students to imitate full teacher trajectories, yet reasoning and knowledge gaps between the teacher and student can cause compounding errors. We propose SCoRe, a student-centered framework in which the student generates training trajectories and the teacher corrects only the earliest error, producing training data matched to the student's abilities and exposing specific weaknesses. The student is first fine-tuned on corrected trajectories. Subsequently, short-horizon reinforcement learning starts from the verified prefix preceding the earliest error, with target rewards assigned at that step. This design enables the student to solve problems through unconstrained RL exploration rather than teacher imitation, while the short-horizon setup improves training stability. On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe closes the agentic performance gap with a 72B-parameter teacher.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。