提出动态融合验证框架，提升大模型推理速度
DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification
- 用动态加权融合草案与目标模型分布，放松严格验证约束
- 实测推理效率显著优于传统推测解码方法
- 适合追求高速推理且对生成质量要求高的场景
推测解码通过并行生成多个词元加速大语言模型推理。实践中，其加速效果常受限于严格的验证步骤——要求接受的词元分布必须与目标模型完全一致。这一约束导致大量合理词元被拒绝，降低接受率并限制整体速度提升。为此，我们提出动态宽松验证推测解码（DIVERSED），一种通过学习集成验证器来融合草案与目标模型分布的松弛验证框架，权重根据任务和上下文动态调整。我们提供了理论依据，并实证表明DIVERSED在保持生成质量的同时，显著提升了推理效率。代码已公开于：https://github.com/comeusr/diversed。
原文摘要 · Abstract (English)
Speculative decoding is an effective technique for accelerating large language model inference by drafting multiple tokens in parallel. In practice, its speedup is often bottlenecked by a rigid verification step that strictly enforces the accepted token distribution to exactly match the target model. This constraint leads to the rejection of many plausible tokens, lowering the acceptance rate and limiting overall time speedup. To overcome this limitation, we propose Dynamic Verification Relaxed Speculative Decoding (DIVERSED), a relaxed verification framework that improves time efficiency while preserving generation quality. DIVERSED learns an ensemble-based verifier that blends the draft and target model distributions with a task-dependent and context-dependent weight. We provide theoretical justification for our approach and demonstrate empirically that DIVERSED achieves substantially higher inference efficiency compared to standard speculative decoding methods. Code is available at: https://github.com/comeusr/diversed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。