通过重塑残差分布，提升多候选推测采样的效率与准确性。
ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping
- 在验证阶段动态调整候选分布，缓解残差漂移问题。
- 相比当前最优方法，最高实现1.92倍加速。
- 适合追求推理效率的LLM服务部署场景。
大型语言模型（LLM）服务的效率受自回归解码顺序性的根本限制。推测解码（SD）通过轻量级草稿模型推测未来标记，并在单次并行前向传播中由主模型验证。为进一步提升效率，多候选方案提出多样化候选集以提高接受概率。然而，我们发现这些方案受限于残差漂移现象：初始候选被拒绝后，残差目标分布偏离草稿模型预测，导致后续候选失效，迫使系统进行昂贵的重采样。为此，我们提出ResiSpec框架，在验证过程中战略性地重构提议分布，将残差目标质量锚定在草稿模型高置信区域。通过数学上对齐验证过程而不牺牲输出精确性，ResiSpec防止候选过时，相比现有最先进多候选方法实现高达1.92×的加速。代码已公开于https://github.com/Czzzk/Resispec。
原文摘要 · Abstract (English)
The efficiency of Large Language Model (LLM) serving is fundamentally limited by the sequential nature of autoregressive decoding. Speculative Decoding (SD) mitigates this by using a lightweight draft model to speculate future tokens, which are then validated by the LLM in a single parallel forward pass. To further boost efficiency, multi-candidate schemes propose diverse candidate sets to increase the likelihood of token acceptance. However, we show that these schemes are bottlenecked by Residual Drift: a phenomenon where the rejection of initial candidates causes the residual target distribution to diverge from the draft model's predictions. This shift renders subsequent candidates ineffective and forces the system into expensive resampling. To resolve this, we propose ResiSpec, a framework that strategically reforms the proposal distribution during verification to anchor the residual target mass within the draft model's high-confidence regions. By mathematically re-aligning the verification process without compromising output exactness, ResiSpec prevents candidate obsolescence and achieves up to 1.92$\times$ speedup over state-of-the-art multi-candidate methods. Code is available at https://github.com/Czzzk/Resispec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。