用检索+概率联合生成，让大模型推理快一倍以上
RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding
- 从上下文检索精确片段，结合概率预测未来内容
- 在多个测试中实现超2倍于传统生成的加速
- 无需训练，可直接插入现有模型加速推理
大型语言模型的自回归解码每步生成一个词元，导致推理延迟高。推测性解码（SD）通过猜测-验证策略缓解此问题，但现有免训练方法存在权衡：基于检索的草稿在无完全匹配时失效，而基于逻辑值的草稿缺乏结构引导。本文提出RACER（检索增强的上下文快速推测解码），一种轻量级、免训练的方法，将检索到的精确模式与基于逻辑值的未来线索相结合。该统一机制既提供可靠锚点，又支持灵活外推，生成更丰富的推测草稿。在Spec-Bench、HumanEval和MGSM-ZH上的实验表明，RACER持续加速推理，在多数场景下实现超过2倍于自回归解码的速度提升，并优于先前免训练方法，为高效LLM解码提供可扩展、即插即用的解决方案。代码已开源。
原文摘要 · Abstract (English)
Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigates this through a guess-and-verify strategy, but existing training-free variants face trade-offs: retrieval-based drafts break when no exact match exists, while logits-based drafts lack structural guidance. We propose $\textbf{RACER}$ ($\textbf{R}$etrieval-$\textbf{A}$ugmented $\textbf{C}$ont$\textbf{e}$xtual $\textbf{R}$apid Speculative Decoding), a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues. This unification supplies both reliable anchors and flexible extrapolation, yielding richer speculative drafts. Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than $2\times$ speedup over autoregressive decoding, and outperforms prior training-free methods, offering a scalable, plug-and-play solution for efficient LLM decoding. Our source code is available at $\href{https://github.com/hkr04/RACER}{https://github.com/hkr04/RACER}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。