提出无需训练的Double机制，突破生成加速瓶颈，显著提升大模型推理速度。
Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism
- 通过双阶段检索并行，让草稿模型迭代推测，打破理论加速上限。
- 在LLaMA3.3-70B上实现5.3倍加速，在Qwen3-32B上达2.8倍，优于需训练的方法。
- 适合追求高效推理、不希望额外训练的大模型部署场景。
并行推测解码（PSD）通过重叠草稿生成与验证过程，提升了传统推测解码（SD）的速度。然而，其仍受制于两大根本挑战：（1）由草稿模型与目标模型速度比决定的理论加速上限；（2）因早期错误导致的中段序列令牌拒绝所引发的高计算浪费与流水线停顿。为此，我们提出 extsc{Double}（双检索推测并行）。该框架通过新颖的同步机制，弥合了SD与PSD之间的差距，解决检索中的精度-效率困境。具体而言，草稿模型可执行迭代检索推测以突破理论加速极限；为避免回滚带来的拒绝，目标模型则进行权威检索，生成多标记引导。 extsc{Double} 完全无需训练且无损失。大量实验表明，在 LLaMA3.3-70B 上实现 5.3× 的最先进加速，在 Qwen3-32B 上达到 2.8×，显著优于需大量训练的先进方法 EAGLE-3。
原文摘要 · Abstract (English)
Parallel Speculative Decoding (PSD) accelerates traditional Speculative Decoding (SD) by overlapping draft generation with verification. However, it remains hampered by two fundamental challenges: (1) a theoretical speedup ceiling dictated by the speed ratio between the draft and target models, and (2) high computational waste and pipeline stall due to mid-sequence token rejections of early errors. To address these limitations, we introduce \textsc{Double} (Double Retrieval Speculative Parallelism). By bridging the gap between SD and PSD, our framework resolves the Retrieval \emph{Precision-Efficiency Dilemma} through a novel synchronous mechanism. Specifically, we enable the draft model to execute iterative retrieval speculations to break the theoretical speedup limits; to alleviate rejections without rollback, the target model performs authoritative retrieval to generate multi-token guidance. \textsc{Double} is entirely training-free and lossless. Extensive experiments demonstrate state-of-the-art speedup of $\textbf{5.3}\times$ on LLaMA3.3-70B and $\textbf{2.8}\times$ on Qwen3-32B, significantly outperforming the advanced method EAGLE-3 that requires extensive model training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。