arXiv:2607.13124cs.LGcs.AI2026-07被引 5

用短到长的策略恢复剪枝大模型生成能力,提升显著。

ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation

  • 先短后长滚动训练,动态分配资源避免重复冗余
  • 恢复后生成得分达未恢复时9倍,超传统方法1.6至4.4倍
  • 适合需要高效部署的剪枝大模型生成任务

结构化剪枝是硬件友好的大模型压缩方式,但主要在选择题任务上验证,实际部署所需的自由文本生成能力却会严重退化。两个观察揭示此差距:其一,剪枝后贪婪解码准确率近乎归零,但多次采样下高阶准确率显著回升,说明有效生成并未消失;其二,失败主因是后缀重复。因此,应在剪枝模型自身生成状态上进行密集的词元级监督训练,即采用基于预压缩模型为冻结教师的在线策略蒸馏(OPD)。然而,长序列滚动生成早期会浪费预算在低信息重复后缀上,延缓损失下降。为此提出【ShortOPD】:通过检测教师确认的重复后缀,将剩余前缀视为每轮有效长度,并将后续预算分配给当前可用的有效长度。在数学、代码及开放式生成任务中,ShortOPD使剪枝模型得分提升至原始未恢复值的约9倍,达标准恢复方法(无知识蒸馏的SFT、KD、SeqKD)的1.6–4.4倍,且在仅8.5小时训练时间(仅为35.9小时的24%)和71%更少的回滚令牌数下,达到固定8192标记序列长度的性能水平。希望该方法推动结构化剪枝从困惑度与选择题基准的边际提升迈向真正可用的生成质量。

原文摘要 · Abstract (English)

Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires. Two observations trace this gap. First, greedy \textsc{pass}@$1$ nearly vanishes after compression, yet \textsc{pass}@$k$ recovers substantially under repeated sampling: useful generations are demoted, not erased. Second, the recoverable regime fails mainly through suffix repetition. Recovery should therefore train on the compressed model's own on-policy states with dense token-level supervision, which On-Policy Distillation (OPD) provides by reusing the pre-compression model as a frozen teacher. However, long on-policy rollouts spend early recovery budget on low-information repetitive suffixes, delaying loss descent. To mitigate this waste, we propose \textbf{\shortopd}, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes, treats the surviving prefix as each rollout's effective length, and allocates future rollout budgets to the effective lengths the policy can currently use. Across math, code, and open-ended generation, \shortopd\ raises the compressed model's score to about $9\times$ its unrecovered value and $1.6$--$4.4\times$ standard recovery recipes (SFT w/o KD, KD, and SeqKD), and it matches a fixed $8192$-token rollout horizon within two points using a quarter of the training time ($8.5$ vs.\ $35.9$ hours) and $71\%$ fewer rollout tokens. We hope this recipe helps move structured pruning beyond marginal gains on perplexity and multiple-choice benchmarks, a step closer to deployment-ready generation quality.

大模型压缩生成恢复蒸馏方法剪枝优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。