通过自推测提升推理模型生成速度,无需训练即可加速复杂任务。
Self-Speculation for Faster Reasoning Models

- 用部分推理路径作为草稿,全路径作为验证,实现无训练的自推测解码。
- 在Qwen3.5和Gemma-4上最高降低24.1%生成延迟,适用于长文本任务。
- 创新引入后缀解码,利用草稿种子恢复非连续有用内容,进一步降延时。
大语言模型(LLMs)被用于越来越多涉及规划与多步决策的复杂任务,但高质量表现通常需要生成长推理链,这与语音助手、代码代理等低延迟、交互式应用的需求不匹配。现有加速方法多聚焦于词元级生成,未利用推理流程结构。本文提出SSR:自推测推理模型,一种无需训练的自推测解码方法,以思维链(CoT)作为推测依据。SSR使用部分推理路径分布作为草稿,全推理路径分布作为验证,两者均来自同一模型在不同推理预算下生成。基于观察——后期部分推理结果常与完整响应有较高语义与词汇重叠——可一次性接受长草稿前缀,显著提速结构化与长文本生成任务。为进一步挖掘草稿与最终响应间的重叠,超越标准推测解码仅接受连续前缀的限制,SSR还引入后缀解码机制,利用草稿种子构建后缀缓存,恢复超出接受前缀的有用片段,进一步降低延迟。我们在多个结构化与长文本生成任务上评估了SSR,证明其在Qwen3.5和Gemma-4等主流开源模型上可实现最高24.1%的总生成延迟相对提升。
原文摘要 · Abstract (English)
Large language models (LLMs) are deployed for increasingly complex tasks involving planning and multi-step decision making, but high-quality performance on these tasks often requires generating long reasoning traces. This is a poor fit for latency-sensitive and interactive applications like voice assistants or coding agents, where generation latency can strongly affect user experience. Existing acceleration methods typically focus on token-level generation, without utilizing the structure of reasoning workflows. We introduce SSR: Self-Speculation for Reasoning Models, a training-free self-speculative decoding method that leverages the chain-of-thought (CoT) as a source of speculation. SSR uses the partial-CoT answer distribution as the drafter and the full-CoT distribution as the verifier, deriving both from the same model at different reasoning budgets. This builds on the observation that later partial-CoT responses often exhibit greater semantic and lexical overlap with the full-budget response. Due to this overlap, SSR can accept long draft prefixes at once, leading to large speedups on structured and long-form generation tasks. To further exploit draft-response overlap beyond the contiguous prefix accepted by standard speculative decoding, SSR also incorporates suffix decoding, using the draft to seed a suffix cache and recover useful spans beyond the accepted prefix, further reducing latency on tasks with high lexical overlap between the draft and the final response. We evaluate SSR on multiple structured and long-form generation tasks where it is most useful, and demonstrate a relative improvement of up to 24.1% on total generation latency for popular open-source models such as Qwen3.5 and Gemma-4.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。