通过部分验证提升长文本生成速度，加速推理同时保持精度。
SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification
- 用部分键值状态快速验证候选词，减少计算开销
- 在长上下文场景下实现最高6倍的生成速度提升
- 适合需要高效长文本生成的研究者和开发者
代码生成、深度推理和长文档理解等任务对大语言模型的长上下文生成能力提出更高要求。自推测解码是一种直接有效的加速生成方法，采用草稿-验证范式：轻量级草稿模型生成多个候选词，目标模型进行验证。然而我们发现，随着上下文长度增加，验证阶段成为主要瓶颈。为此，我们提出SpecPV，一种自推测解码方法，通过使用部分键值状态（KV）实现快速验证，并定期进行完整验证以消除累积误差。我们在多个长上下文基准和模型（包括LLaMA-3.1-8B-Instruct和Qwen3系列）上验证了该方法。实验结果表明，SpecPV相比标准自回归解码最高可实现6倍的速度提升，且性能损失极小。
原文摘要 · Abstract (English)
Growing demands from tasks like code generation, deep reasoning, and long-document understanding have made long-context generation a crucial capability for large language models (LLMs). Speculative decoding is one of the most direct and effective approaches for accelerating generation. It follows a draft-verify paradigm, where a lightweight draft model proposes several candidate tokens and the target model verifies them. However, we find that as the context length grows, verification becomes the dominant bottleneck. To further accelerate speculative decoding in long-context generation, we introduce SpecPV, a self-speculative decoding approach that performs fast verification using partial key-value states (KV) and periodically applies full verification to eliminate accumulated errors. We validate SpecPV across multiple long-context benchmarks and models, including LLaMA-3.1-8B-Instruct and Qwen3-series. Experimental results show that SpecPV achieves up to 6x decoding speedup over standard autoregressive decoding with minor degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。