提出无需训练的树状生成方法，提升大模型推理速度与效率。
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
- 基于Domino的条件校正机制构建路径依赖的树结构草案
- 在8个基准上实现最高平均通过长度，最长达32K上下文
- 适合追求高吞吐、低延迟推理的部署场景
推测解码通过并行生成候选词并验证来加速大模型推理。现有块扩散类生成器如DFlash仅建模位置独立的边缘分布，而树形方法如DDTree则基于这些边缘分布扩展候选树。释放的Domino生成器引入基于GRU的因果校正，使每个生成词的分布依赖于历史路径，这是DDTree因子化形式无法表达的。本文提出DominoTree，一种无需训练的最佳优先草案树，沿每条根到节点路径使用Domino的条件（非因子化）校正，并通过限制每节点校正仅作用于候选Top-M使其可实用。我们在单流环境和SGLang中评估该方法，对比AR、DFlash、EAGLE-3和Domino，在相同参数设置下，DominoTree在所有服务单元中均达到最高平均通过长度——涵盖两种模型尺寸、单请求与并发负载、上下文长度至32K；在研究测试框架中，24个数据集/温度组合中有21个取得最高总体通过长度。三路分解显示：应用校正本身带来+10.1%的通过长度增益，沿实际路径重新计算校正再带来+4.7%提升，即本文贡献的核心部分。当验证占主导时，吞吐随之提升：相比AR最高达7.3倍，超越已发布Domino在CUDA图最优下的表现，且在SGLang中单请求吞吐领先Domino 12%（Qwen3-8B）。在HELMET长上下文任务中，其通过长度领先Domino 29–36%，吞吐领先10–34%。在显存受限设备的准入阈值之上，链式生成展现更优吞吐性能；在最长上下文（预填充主导）下，对EAGLE-3的领先缩小至持平。
原文摘要 · Abstract (English)
Speculative decoding accelerates LLM inference by drafting tokens and verifying them in parallel. Block-diffusion drafters such as DFlash model only per-position marginals, and tree methods such as DDTree expand candidate trees from those marginals. The released Domino drafter adds a GRU-based causal correction making each draft token's distribution path-dependent, a structure DDTree's factorized formulation cannot represent. We introduce DominoTree, a training-free best-first draft tree scored by Domino's conditional (non-factorized) correction along each root-to-node path, made practical by restricting the per-node correction to a candidate top-M. We evaluate it on eight benchmarks in a single-stream harness, and in SGLang, where it runs as an out-of-tree plugin against AR, DFlash, EAGLE-3 and Domino under identical flags. DominoTree attains the highest mean accepted length in every serving cell - two model sizes, single-request and concurrent load, context to 32K - and the highest Overall accepted length at every temperature in the research harness (21 of 24 per-dataset cells). A three-arm decomposition holding drafter, budget and verifier fixed separates the gain from applying the correction at all (+10.1% accepted length) from that of recomputing it along each candidate's realized path (+4.7% more), the part this paper adds. Where the round is verify-dominated, throughput follows: up to 7.3x over AR on Qwen3-8B, beating the released Domino decoder at its CUDA-graph best at every temperature, and inside SGLang winning single-request throughput by +12% over Domino on Qwen3-8B. On HELMET long context it beats Domino by +29-36% accepted length and +10-34% throughput at every length and both model sizes. Past a memory-constrained card's admission cap the chain wins goodput, and at our longest context, where prefill dominates, our lead over EAGLE-3 narrows to a tie.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。