用谱柯普曼机制实现无缓存的高效记忆检索,突破传统模型长序列记忆瓶颈。
Echo: KV-Cache-Free Associative Recall with Spectral Koopman Operators

- 引入谱柯普曼注意力,以常数内存动态建模历史键值对,无需存储缓存。
- 在5000万参数下,对4096跨度、32组干扰项均达100%召回准确率。
- 适合长链推理与工具调用场景,特别适用于资源受限设备部署。
长链思维推理和代理工具调用会产生长达数万标记的痕迹,但Transformer的键值缓存随序列长度线性增长,在普通硬件上造成内存瓶颈。状态空间模型虽能实现常数内存循环,却存在记忆悬崖:当存储事实与查询之间的距离超过递归状态的有效范围时,检索准确率急剧下降。我们提出Echo,一种基于谱柯普曼注意力(SKA)的无键值缓存关联记忆架构;该方法可作为注意力层的即插即用替代方案,为状态空间模型块增加一个闭式动力学算子,其充分统计量可在恒定内存中累积,无需键值缓存。Echo通过核岭回归拟合键值历史的谱线性系统,并通过学习的幂迭代滤波器进行检索,整个过程仅需$O(r^{2})$流式状态,其中$r$为小投影秩。在多查询关联记忆基准测试中,纯Mamba-2 SSM在所有跨度和键值对数量下均未超过随机水平(~3%),而5000万参数规模的SKA增强模型在所有测试配置中均达到100%召回率,包括4096标记跨度、32组键值对的干扰情形。在五个额外迁移基准(如针堆找针、工具调用轨迹、多跳检索)中,SKA持续优于纯SSM及SSM+Attention混合模型,同时保持常数推理内存。消融实验表明,性能提升源于谱算子本身,而非前缀掩码策略。
原文摘要 · Abstract (English)
Long chain-of-thought reasoning and agentic tool-calling produce traces spanning tens of thousands of tokens, yet Transformer KV caches grow linearly with sequence length, creating a memory bottleneck on commodity hardware. State-space models offer constant-memory recurrence but suffer a memory cliff: retrieval accuracy collapses once the gap between a stored fact and its query exceeds the effective horizon of the recurrent state. We introduce Echo, a KV-cache-free associative recall architecture built around Spectral Koopman Attention (SKA); a drop-in replacement for attention layers that augments SSM blocks with a closed-form dynamical operator whose sufficient statistics are accumulated in constant memory with no KV cache. Echo fits a spectral linear system to the key and value history via kernel ridge regression and retrieves through a learned power-iterated filter, all from $O(r^{2})$ streaming state where $r$ is a small projection rank. On the Multi-Query Associative Recall benchmark, a pure Mamba-2 SSM fails to exceed chance accuracy (${\sim}3\%$) across all gap lengths and KV-pair counts, while at the 50M parameter scale SKA-augmented models achieve $100\%$ retrieval accuracy on every configuration tested, including distractor gaps of $4{,}096$ tokens with $32$ KV pairs. Across five additional transfer benchmarks including needle-in-a-haystack, tool-trace, and multi-hop retrieval, SKA consistently outperforms both pure SSM and SSM+Attention hybrids while maintaining constant inference memory. Ablations confirm that the spectral operator, not the prefix masking strategy, drives the retrieval gain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。