arXiv:2608.15533cs.DCcs.LG2026-08

通过延迟更新状态,显著降低线性注意力解码的内存开销。

DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding

  • 用基础状态+紧凑更新日志表示递归状态,减少频繁写入。
  • 相比原方法,状态写入流量最多降低7.83倍,解码速度提升1.2倍。
  • 适合大模型、多头结构的在线服务场景,尤其关注内存效率。

线性注意力模型通过递归状态更新替代成对的标记交互,消除了传统softmax注意力中的二次前缀计算和上下文增长的键值缓存问题。然而,现有解码实现通常在每个生成标记后都完整写出递归状态,导致状态维护成为主要内存流量来源,尤其在状态规模大、头数多的模型中更为严重。本文提出DeltaLog,一种不改变模型语义的递归状态解码方案。具体而言,DeltaLog将递归状态表示为密集的基础状态与有限长度的紧凑更新日志之和:大多数解码步骤仅向日志追加小型更新因子,周期性合并步骤则将累积更新回填至密集基础状态。因此,模型观察到的状态与立即更新一致,但大部分全状态写回被轻量级追加操作取代。我们在GDN、KDA和RWKV6上实现了DeltaLog,并集成至原型服务栈。实验表明,该方法使递归状态更新内核加速达1.86倍,有损状态写入流量最多降低7.83倍,端到端服务速度相较密集基线提升1.05–1.20倍。

原文摘要 · Abstract (English)

Linear attention models eliminate the quadratic prefix computation and context-growing KV cache of softmax attention by replacing pairwise token interactions with recurrent state updates. However, existing decoding implementations often materialize and write back the full recurrent state after every generated token, making state maintenance a major source of memory traffic, especially for models with large states and many heads. This paper presents DeltaLog, a recurrent-state decoding scheme that reduces this overhead without changing the model semantics. Specifically, DeltaLog represents the recurrent state as a dense base state together with a bounded log of recent compact updates. Most decode steps append only compact update factors to this log, while periodic merge steps fold the accumulated updates back into the dense base state. Thus, the model observes the same dense state as in eager decoding, but most full-state write-backs are replaced by lightweight append operations. We implement DeltaLog for GDN, KDA, and RWKV6 and integrate it into a prototype serving stack. Across these models, DeltaLog accelerates the recurrent-state update kernel by up to $1.86\times$, reduces profiled recurrent-state write traffic by up to $7.83\times$, and achieves $1.05$--$1.20\times$ end-to-end serving speedups over dense recurrent baselines.

线性注意力状态压缩推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。