提出一种无需训练的推测解码方法，提升生成效率
Approximate Speculative Decoding
- 用预算控制的最长前缀选择替代首次不匹配即终止的策略
- 在7个任务上平均提升7.78%吞吐量，最高达15.26%
- 适合追求高效推理且不想调模型的研究者和工程师
推测解码通过并行验证草稿块加速自回归生成。标准贪婪验证在首个草稿词与目标模型最大概率词不同时停止，丢弃后续目标得分的连续后缀。尽管此类不匹配会改变解码路径，但若其后续词在实际前缀下仍为贪婪选择，仍可复用。本文提出训练无关的近似推测解码（ASD），将二元首不匹配截断替换为带预算的最长前缀选择。ASD允许在局部目标对数几率门控、每块例外上限及请求级持续后悔预算约束下接受部分不匹配，并在无需额外近似决策或目标模型前向传播的前提下复用连续的目标贪婪后缀。ASD无需新草稿模型或微调，当预算为零时精确退化为标准贪婪验证。实验表明，相较于严格匹配验证，ASD在固定工作负载下提升3.05%至15.26%，在七个Qwen3-14B + DSpark-14B任务中平均提升7.78%；在DeepSeek-V4-Flash（284B）+ DSpark的FP4至FP8兼容设置下，于GSM8K和MATH-500任务上使验证器侧接受率提升约10%至16%。源代码公开于：https://github.com/Kissmetothemoon/ASD
原文摘要 · Abstract (English)
Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the first draft token that differs from the target argmax, discarding the remaining target-scored suffix. Although accepting such a mismatch changes the decoding trajectory, it can make a contiguous suffix reusable when its tokens remain target-greedy under the realized prefix. In this paper, we introduce \textbf{Approximate Speculative Decoding (ASD)}, a training-free verifier that replaces binary first-mismatch truncation with budgeted longest-prefix selection. ASD accepts selected mismatches subject to a local target-logit regret gate, a per-block exception cap, and a persistent request-level regret budget, then reuses the contiguous target-greedy suffix without additional approximate decisions or target-model forward passes. ASD requires neither a new draft model nor fine-tuning, and exactly reduces to standard greedy verification when the budget is zero. Experiments show that ASD improves fixed-workload throughput by $3.05\%$--$15.26\%$ over matched strict verification and averages a $7.78\%$ gain across seven Qwen3-14B + DSpark-14B tasks. On DeepSeek-V4-Flash (284B) with DSpark it also raises verifier-side acceptance by roughly $10\%$--$16\%$ on GSM8K and MATH-500 in an FP4-to-FP8 compatibility setting. The source code is publicly available at: https://github.com/Kissmetothemoon/ASD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。