动态调整词汇表与参数，让大模型推理更快更准。
EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation
- 根据验证反馈实时优化词汇表和轻量参数。
- 在Qwen3-8B上提速2.18倍，比EAGLE-2快20%。
- 适合需要高效推理的部署场景，如边缘计算。
推测解码通过先生成后验证的方式加速大语言模型推理，但轻量级草案模型存在效率与质量难以兼顾的问题：大词汇量输出投影开销高，而有限的草案容量和静态参数导致在专业或变化输入下接受率下降。词汇剪枝可降低投影成本，但静态版本忽略局部重要长尾词，动态版本仍受预设策略和预算限制。此外，草案容量不足会导致目标词虽被覆盖却分布不匹配。在线对齐可提升草案质量，但全参数更新带来显著内存与延迟开销。本文提出EvoSpec，基于验证反馈联合自适应调整活跃词汇表与轻量草案参数。它异步检索语义与统计邻近词，并进行课程加权的在线LoRA对齐，同时保持与目标模型完全一致的验证过程。在Qwen3-8B/EAGLE-2上，EvoSpec相比原始解码实现2.18倍加速，相较EAGLE-2提升20%，提升专业领域覆盖率，且辅助GPU适配内存减少27%。
原文摘要 · Abstract (English)
Speculative decoding accelerates Large Language Model inference through draft-then-verify generation, yet lightweight draft models face coupled efficiency and quality limitations: large-vocabulary output projection is costly, while limited draft capacity and static parameters reduce acceptance under specialized or shifting inputs. Vocabulary pruning lowers projection cost, but static variants miss locally important long-tail tokens, while dynamic variants remain sensitive to preset selection policies and budgets. Moreover, limited draft capacity can leave the draft distribution misaligned even when the target token is covered. Online alignment improves draft quality, but full-parameter updates introduce substantial memory and latency overhead. We introduce EvoSpec, which jointly adapts the active vocabulary and lightweight draft parameters from verification feedback. EvoSpec asynchronously retrieves semantic and statistical token neighbors and performs curriculum-weighted online LoRA alignment while preserving exact target-model verification. On Qwen3-8B/EAGLE-2, EvoSpec reaches a $2.18\times$ speedup over vanilla decoding and a $1.20\times$ gain over EAGLE-2, while improving specialized-domain coverage and using $27\%$ less auxiliary GPU adaptation memory than full-parameter online adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。