让每个验证的令牌都有效，提升MoE模型的推测解码速度
Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding
- 通过截断草稿树只保留高性价比前缀，实现自适应验证
- 在多个MoE模型上最高提速2.35倍，平均比基线快1.21倍
- 无需训练和调参，兼容主流服务框架，适合部署优化场景
基于树结构的推测解码通过并行验证多个草稿候选来加速自回归生成，但在稀疏的专家混合（MoE）模型中该优势减弱。随着草稿树增长，不同分支激活不同专家，导致目标侧验证时激活的专家集合扩大，显著增加验证开销。我们提出EVICT，一种无需训练、无需超参数调优且无损失的MoE推测解码自适应验证方法。EVICT通过在目标验证前截断草稿树，仅保留成本效益高的前缀，利用细粒度草稿信号估算候选收益，并结合离线预估的验证开销。该方法与高性能图式服务框架SGLang高度兼容。在多种MoE骨干模型和基准测试上的实验表明，相比自回归解码最高提速2.35倍，平均比先进基线EAGLE-3快1.21倍，同时显著减少验证阶段的无效专家激活。
原文摘要 · Abstract (English)
Tree-based speculative decoding accelerates autoregressive generation by verifying multiple draft candidates in parallel, but this advantage weakens for sparse Mixture-of-Experts (MoE) models. As the draft tree grows, different branches activate different experts, expanding the union of activated experts and substantially increasing target-side verification cost. We propose EVICT, a training-free, hyperparameter-free, and lossless adaptive verification method for MoE speculative decoding. EVICT makes every verified token count by truncating the draft tree before target verification and retaining only the cost-effective prefix. It leverages fine-grained drafter signals to estimate candidate benefit, combines them with offline-profiled verification cost, and remains highly compatible with the high-performance graph-based serving framework SGLang. Extensive experiments on diverse MoE backbones and benchmarks show that EVICT achieves up to 2.35x speedup over autoregressive decoding and an average 1.21x speedup over the state-of-the-art baseline EAGLE-3, while significantly reducing unnecessary expert activations during verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。