无需训练的推测解码新方法，提升长文本生成效率。
Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computes
- 用验证器已计算的隐藏状态重键池中上下文，实现语义匹配
- 在相同预算下，接受长度提升24%-29%，最高达4.4倍加速
- 特别适合工具调用等重复性高、需精准续写的场景
提出一种无需训练的推测解码方法Oilbird，通过匹配上下文精确后缀从历史池中获取草稿。分析发现，错误主要源于索引方式而非内容覆盖不足：在最密集的工具调用基准上，约一半正确草稿存在于池中但无法通过精确匹配访问。为此引入第二类语义草稿源——使用验证器已计算的隐藏状态作为新密钥，并与现有词法草稿器树结构融合。在三个已发表的草稿器中，相同池大小和预算下，接受长度提升24%-29%。在API-Bank测试中，Oilbird达到4.4倍自回归解码速度，优于最强训练自由基线（3.9x）和EAGLE-3（2.0x）。
原文摘要 · Abstract (English)
Training-free speculative decoding drafts by matching an exact suffix of the context against a pool of earlier context. That lookup misses correct drafts already in the pool, most visibly on tool-calling traffic, where a request repeats almost everything but the few values minted for it, and where one rejected token discards the correct continuation behind it. We diagnose the failure position by position across ten benchmarks and find it to be a problem of addressing rather than of coverage: on our densest tool-calling benchmark, about half of what the strongest exact-match drafter misses is present in the pool yet unreachable by exact matching. We therefore propose a second, semantic draft source: the same pool, re-keyed by the hidden state the verifier has already computed at each committed token, together with a merge that lets it ride inside an existing lexical drafter's tree. In three published drafters, at matched pool and budget, it lifts accepted length by 24-29%. Oilbird reaches 4.4x autoregressive decoding speed on API-Bank, against 3.9x for the strongest training-free baseline in our harness and 2.0x for EAGLE-3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。