提出SpecLA，让线性注意力模型实现更快的推测解码。
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
- 设计拓扑感知核，支持链与树结构的推测验证。
- 通过紧凑因子存储，高效恢复被接受的状态轨迹。
- 适合使用状态化线性注意力模型的推理加速场景。
线性注意力模型用递推状态替代增长的键值缓存，但自回归解码仍需逐个读取、更新和写入这些状态。推测解码可通过一次目标阶段验证多个草稿标记来降低此开销，但现有系统专为Transformer键值缓存设计。针对有状态的线性注意力目标，验证必须遵循跨链与分支的递推依赖，接受需仅更新已接受的状态轨迹，且草稿生成器必须避免提交浪费状态验证工作的情况。本文提出SpecLA，一种面向有状态线性注意力模型的推测解码运行时。SpecLA采用拓扑感知核验证链与树结构，以紧凑因子存储验证过程产物以恢复接受状态，并结合置信度剪枝与目标对齐的EAGLE风格草稿生成器，向验证器输入有效候选。在NVIDIA H100上，使用公开GDN-1.3B目标模型，SpecLA相比自回归解码最高实现1.70倍端到端加速。
原文摘要 · Abstract (English)
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculative decoding can reduce this cost by verifying several draft tokens in one target pass, yet existing speculative systems are designed for Transformer KV caches. For stateful linear-attention targets, verification must follow recurrent dependencies across chains and branches, acceptance must update only the accepted state trajectory, and the drafter must avoid submitting candidates that waste stateful verification work. This paper presents SpecLA, a speculative decoding runtime for stateful linear-attention models. SpecLA verifies chains and trees with topology-aware kernels, stores compact factors produced during verification to recover accepted states, and uses confidence pruning plus a target-aligned EAGLE-style drafter to feed useful candidates to the verifier. On an NVIDIA H100 with a public GDN-1.3B target, SpecLA achieves up to 1.70x end-to-end speedup over autoregressive decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。