开源框架SpecForge加速大模型推测解码训练，提升推理速度。
SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
- 采用目标-草稿解耦与混合并行，优化训练效率
- 使Qwen3-235B-A22B训练速度提升9.9倍
- 配套发布生产级草稿模型，支持实际部署
大语言模型因自回归解码导致高延迟。推测解码通过轻量草稿模型批量预生成多个词元并验证，缓解此瓶颈。然而其应用受限于高质量草稿模型稀缺及可扩展训练基础设施不足。我们提出SpecForge，一个面向生产环境的开源训练框架，全面支持EAGLE-3。该框架引入目标-草稿解耦、混合并行、优化训练内核，并集成生产级推理引擎，实现Qwen3-235B-A22B模型上高达9.9倍的EAGLE-3训练加速。此外，我们发布SpecBundle，一套基于SpecForge训练的生产级EAGLE-3草稿模型，覆盖主流开源LLM。系统研究推测解码训练策略后，这些草稿模型在SGLang上实现最高4.48倍端到端推理加速，确立SpecForge作为真实场景推测解码部署的实用基础。
原文摘要 · Abstract (English)
Large language models incur high inference latency due to sequential autoregressive decoding. Speculative decoding alleviates this bottleneck by using a lightweight draft model to propose multiple tokens for batched verification. However, its adoption has been limited by the lack of high-quality draft models and scalable training infrastructure. We introduce SpecForge, an open-source, production-oriented framework for training speculative decoding models with full support for EAGLE-3. SpecForge incorporates target-draft decoupling, hybrid parallelism, optimized training kernels, and integration with production-grade inference engines, enabling up to 9.9x faster EAGLE-3 training for Qwen3-235B-A22B. In addition, we release SpecBundle, a suite of production-grade EAGLE-3 draft models trained with SpecForge for mainstream open-source LLMs. Through a systematic study of speculative decoding training recipes, SpecBundle addresses the scarcity of high-quality drafts in the community, and our draft models achieve up to 4.48x end-to-end inference speedup on SGLang, establishing SpecForge as a practical foundation for real-world speculative decoding deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。