让大模型训练聚焦在最能提升的难题上，效率更高更稳定。
PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
- 根据学生答题率动态加权，优先训练掌握度适中的题目。
- 在多个数学竞赛数据集上比传统方法提升8.2分，遗忘率降至1.4%。
- 无需改模型结构，自动适应不同任务，适合高效训练大模型。
标准大模型蒸馏对所有问题一视同仁，浪费算力在已掌握或无法解决的问题上。我们实证发现，这种低效在梯度层面有明确特征：跨问题梯度信噪比随学生通过率呈钟形分布，两端趋近于零。为此提出PACED，以权重w(p) = p(1−p)动态分配训练重点，聚焦于‘最近发展区’。仅需学生采样结果，无需架构改动或超参调整。理论上证明，形式为w(p) = p^α(1−p)^β的Beta核是信噪比崩溃结构下的最优权重族，且对模型偏差具有最小最大鲁棒性（最坏情况效率损失仅O(δ²)）。在Qwen3、Qwen2.5和Llama-3系列模型上，PACED在MATH-500、AIME~2024和AIME~2025任务中达到新SOTA，相比无权重蒸馏最高提升+8.2，较AKL基线最高提升+3.6，同时将遗忘率控制在1.4%和0.6%。采用两阶段前向-反向KL调度进一步提升至+5.8，超越标准前向KL。
原文摘要 · Abstract (English)
Standard LLM distillation treats all training problems equally -- wasting compute on problems the student has already mastered or cannot yet solve. We empirically show that this inefficiency has a precise gradient-level signature: the cross-problem gradient signal-to-noise ratio (SNR) follows a bell curve over student pass rate, collapsing at both extremes. We propose PACED, which weights each problem by $w(p) = p(1{-}p)$ where $p$ is the student's empirical pass rate -- concentrating training on the zone of proximal development. This requires only student rollouts, no architectural changes, and no hyperparameters. We prove the Beta kernel $w(p) = p^α(1{-}p)^β$ is the leading-order optimal weight family arising from the SNR boundary-collapse structure, and is minimax-robust under misspecification (worst-case efficiency loss $O(δ^2)$). Across Qwen3, Qwen2.5, and Llama-3 families, PACED sets a new state of the art in our experimental setting on MATH-500, AIME~2024, and AIME~2025, improving over unweighted distillation by up to $\mathbf{+8.2}$ and over the strong AKL baseline by up to $\mathbf{+3.6}$, while reducing forgetting to $\mathbf{1.4\%}$ and $\mathbf{0.6\%}$ in distillation and self-distillation. A two-stage forward-then-reverse KL schedule pushes gains further to $\mathbf{+5.8}$ over standard forward KL on the hardest benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。