LARK通过学习性评估精选推理轨迹,提升学生模型训练效率。
LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation

- 基于学生模型的学习速率设计轨迹选择策略
- 在保持分布覆盖的前提下,使损失下降速度提升30%以上
- 适合需要高效推理蒸馏的AI模型开发者
本文研究推理蒸馏中的轨迹选择问题,即从教师生成的推理轨迹中选择部分作为学生模型的监督信号。现有方法依赖轨迹质量或模型置信度等启发式规则,但常忽略学生是否能有效学习这些轨迹。为此,本文提出LARK,一种基于学习性的轨迹选择方法。其核心是学习性因子ρ,表征学生训练损失下降速率。为高效估计该速率并维持泛化能力,LARK引入学习性代理与χ²正则化选择策略,两者均具有强理论保证的估计误差。实验表明,LARK在多个基础模型和推理任务上持续优于数据选择基线。诊断分析显示,LARK得分可预测下游训练效用,且所选轨迹能加速监督微调过程中的损失下降。代码已开源。
原文摘要 · Abstract (English)
We study trajectory selection for reasoning distillation, where teacher-generated reasoning trajectories are selectively used as supervision for a student model. Existing methods rely on heuristics such as trajectory quality or model confidence, but they often overlook whether a trajectory is learnable by the student. In this paper, we present LARK, a learnability-grounded method for reasoning trajectory selection. LARK selects trajectories that the student can learn efficiently while preserving the generalization of the full training distribution. At the core of LARK is a learnability factor $ρ$, which characterizes the rate at which the student's training loss decreases. To estimate this rate efficiently and maintain generalization, we introduce a learnability proxy and a $χ^2$-regularized selection policy that balances learnability and distributional coverage, both with strong theoretical guarantees on their estimation error. Empirically, LARK consistently outperforms data selection baselines across multiple base models and reasoning tasks. Diagnostic analyses show that the LARK score predicts downstream training utility and that LARK-selected trajectories induce faster supervised fine-tuning loss reduction. Our code is available at https://github.com/Tianrun-Yu/LARK.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。