用树结构恢复生成依赖，让快速推测解码更高效。
Trees from Marginals: Autoregressive drafting with factorized priors
- 基于因子化草案模型的前缀边际，构建带条件依赖的生成树。
- 在不增加全词表投影的情况下，实现4.37倍推理加速。
- 适合追求低延迟生成的系统级部署场景。
推测解码通过以计算换更多生成词的方式，显著提升自回归语言模型的交互性。因子化草案模型因并行预测未来词的边际分布而特别高效，但其独立性假设导致推测预算增大时接受率急剧下降。本文分析该限制，提出Weaver——一种轻量级自回归适配器，从因子化草案模型的前K个边际中构建提案树，恢复生成词间的条件依赖，同时避免全词表投影。为支持带有门控增量网络层的模型，推导出无需回滚的树验证算法，并在SGLang中实现优化的CUDA内核。结合模型与系统改进，相比自回归解码获得4.37倍加速，优于高度优化的DFlash基线24.7%。
原文摘要 · Abstract (English)
Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter. Weaver restores conditional dependencies between proposed tokens while avoiding a full-vocabulary projection. To support fast verification for models with Gated Delta Net layers, we derive a rollback-free tree-verification algorithm and implement optimized CUDA kernels in SGLang. By combining these model and systems contributions we achieve a 4.37-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 24.7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。