无需训练，让大模型一次生成多个候选路径，提速翻倍
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
- 用渐进树结构引导模型并行探索多个语义路径
- 单次前向传播实现多路径生成，最高提速2倍
- 不依赖额外模型，适配任意自回归语言模型
推测解码通过缓解内存瓶颈显著加速了大语言模型的推理。然而，传统方法通常依赖辅助草稿模块，带来显著的训练和通信开销。尽管近期方法尝试在目标模型内部生成草稿，但因缺乏结构化协调，难以充分挖掘其潜在并行能力。本文提出「渐进树草稿（Progressive Tree Drafting, PTD）」，采用结构化、有指导的并行草稿策略，激发模型的并行潜力。通过结合渐进树结构与逐步剪枝机制，PTD 在单次前向传播中主动引导模型探索多个语义路径，确保草稿的多样性与连贯性。实验表明，PTD 在多个基准上实现了最高2倍的解码加速，且无需训练、对模型无特定要求。代码已开源：https://github.com/MINE-USTC/PTD。
原文摘要 · Abstract (English)
Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically relies on auxiliary draft modules, incurring significant training and communication overhead. Although recent methods attempt to generate drafts within the target model itself, they often fail to fully exploit its latent parallel capacity due to a lack of structural coordination. In this paper, we propose \textbf{Progressive Tree Drafting (PTD)}, which employs a structured, guided parallel drafting strategy to harness the model's parallel potential. By coupling a progressive tree structure with a stepwise pruning mechanism, PTD actively guides the LLM to explore multiple semantic paths in a single forward pass, ensuring both draft diversity and coherence. Experiments demonstrate that PTD achieves up to $2\times$ decoding speedup across various benchmarks while remaining training-free and model-agnostic. Our code is available at: https://github.com/MINE-USTC/PTD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。