arXiv:2608.02123cs.CL2026-08被引 1

让大模型推理更快:通过树状结构提升并行生成效率

From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decoding

  • 用树形结构代替线性生成,基于已有模型条件能力实现多路径并行
  • 在7个验证预算下,速度提升3.1%到29.5%,4B模型上加速比达6.6倍
  • 无需重新训练,仅调整解码策略,适合追求推理速度的部署场景

推测解码可加速大语言模型推理,但仅当生成内容通过目标模型验证时才有效。半自回归生成器如DSpark通过一次前向计算预测一整块令牌,并用轻量马尔可夫头进行修正。然而,DSpark将该块解码为单一链式结构,早期不匹配即导致后续全部无效,限制了大块生成的优势。我们发现,DSpark已学习的条件结构可支持无须重训练或额外主干计算的多父节点一致生成。为此提出父条件树(PCTree),利用预训练马尔可夫头对每个具体父节点独立评分不同子节点,并将固定验证预算分配给最可能路径。这使DSpark的线性生成变为树状结构,同时保持单次并行主干计算。在Qwen3-{4B,8B,14B}和九个基准测试中,当验证预算B=7时,相比匹配的DSpark,相对自回归解码的速度提升为3.1%至29.5%。在Qwen3-4B GSM8K任务中,当B=16时,平均接受长度从9.41提升至11.16,三轮平均自回归加速比从6.14×提升至6.60×。结果表明,通过仅修改推理过程,即可将半自回归生成器中已有的条件容量转化为端到端推理收益。

原文摘要 · Abstract (English)

Speculative decoding accelerates LLM inference only when drafted continuations survive target-model verification. Semi-autoregressive drafters such as DSpark predict an entire token block with one backbone forward and refine it with a lightweight Markov head. However, DSpark decodes this block as a single chain, so an early mismatch invalidates the remaining suffix and limits the benefit of large draft blocks. We show that the conditional structure already learned by DSpark can support multiple parent-consistent continuations without retraining or additional backbone passes. We introduce Parent-Conditioned Drafting Tree (PCTree), which uses the pretrained Markov head to score alternative children separately for each concrete parent and allocates a fixed verification budget to the most probable paths. This converts DSpark's linear draft into a tree while preserving its one-pass parallel backbone. Across Qwen3-{4B,8B,14B} and nine benchmarks, at $B{=}7$, measured speedup gains over autoregressive (AR) decoding, relative to matched DSpark, range from $3.1\%$ to $29.5\%$. On Qwen3-4B GSM8K at $B{=}16$, PCTree increases mean acceptance length from $9.41$ to $11.16$ and three-run mean AR speedup from $6.14{\times}$ to $6.60{\times}$. These show that parent-conditioned branching can turn conditional capacity already present in a semi-autoregressive drafter into end-to-end inference gains through an inference-only change.

推理加速生成优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。