让大模型生成更快:拆分因果建模与逐字起草,提升推理效率。
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding

- 先并行生成候选词,再用轻量头补全上下文依赖关系
- 在Qwen3上实现最高5.8倍吞吐加速
- 适合追求高并发推理的部署场景
推测解码通过并行验证多个候选词来加速大模型推理。但其实际提速受限于草稿质量与起草成本之间的权衡:自回归草稿器虽能建模词间依赖,却有串行开销;并行草稿器虽快,但弱化了块内依赖关系。本文提出Domino框架,将因果建模与昂贵的自回归起草过程解耦。首先用并行草稿主干生成整块的初步分布,再通过轻量级Domino头引入前缀相关的因果信息进行修正。为稳定教师强制下的因果编码,进一步设计基线锚定训练策略,先强化并行主干,再逐步优化至最终的因果校正分布。在Qwen3模型上的实验表明,Domino在Transformers后端实现最高5.49倍端到端加速,在SGLang服务中达到最高5.8倍吞吐加速。
原文摘要 · Abstract (English)
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical speedup is constrained by the trade-off between draft quality and drafting cost: autoregressive drafters model causal dependencies among draft tokens but incur sequential overhead, while parallel drafters reduce drafting cost but weaken intra-block dependency modeling. In this paper, we propose Domino, a speculative decoding framework that decouples causal dependency modeling from expensive autoregressive draft execution. Domino first uses a parallel draft backbone to produce preliminary draft distributions for the entire block, and then applies a lightweight Domino head to refine them with prefix-dependent causal information. To stabilize teacher-forced causal encoding, we further introduce a base-anchored training curriculum that first strengthens the parallel backbone and then gradually shifts optimization toward the causally corrected final distribution. Experiments on Qwen3 models show that Domino achieves up to \(5.49\times\) end-to-end speedup under the Transformers backend and up to \(5.8\times\) throughput speedup under SGLang serving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。