直接优化生成接受率，让小模型更高效地猜中大模型答案。
LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding
- 设计新型损失函数LK，直接提升候选词被接受的概率。
- 在8B到685B模型上，平均接受长度提升8%-10%。
- 无需额外计算开销，可无缝接入现有训练框架。
推测解码通过轻量级草稿模型提出候选词，由目标模型并行验证以加速自回归大语言模型推理。速度提升高度依赖接受率，但标准训练以KL散度为代理目标。尽管KL与接受率全局最优解一致，但容量有限的小型草稿模型常收敛至次优解，最小化KL并不保证最大化接受率。为此，我们提出LK损失，一种直接针对接受率的训练目标。在四种草稿架构和六种目标模型（参数量8B至685B）上的全面实验表明，相比标准KL训练，所有配置下接受率指标均有稳定提升。我们在通用、编程和数学领域评估，平均接受长度最高提升8%-10%。LK损失实现简单，无计算开销，可直接集成至任意现有推测器训练框架，是现有训练目标的有力替代方案。
原文摘要 · Abstract (English)
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose LK losses, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 8-10% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。