arXiv:2609.04971cs.LGcs.CL2026-09

通过锚点查询压缩记忆,大幅提升长推理模型的效率。

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

论文配图:BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
图 1 · 摘自论文原文
  • 用锚点查询代表全局查询聚类,预判哪些记忆会被重用。
  • 在多个模型上实现最高5.8倍内存压缩,精度几乎不变。
  • 无需训练,适合部署长序列推理任务的工程师使用。

大型推理模型(LRMs)通过扩展的思维链(CoT)生成实现优秀的问题求解能力,但由此产生的键值(KV)缓存随序列长度线性增长,造成严重的内存瓶颈,常超出GPU容量。现有KV缓存压缩方法依赖近期查询来估计未来标记的重要性,隐含假设这些查询可作为未来注意力模式的可靠代理。我们发现,在长时程推理中,这一假设失效:某些解码步骤会产生回溯思考标记(TRT),重新关注远距离之前的上下文,例如早期形成的任务解决计划。通过系统分析,我们发现对应于TRT的查询在嵌入空间中聚集为少量相似组。基于此洞察,我们提出BeaconKV,一种无需训练的KV缓存压缩方法,通过保留每个全局查询簇的锚点查询,预测哪些KV对将被重访,而无需存储完整查询历史。在四个开源LRM和多样化的推理基准测试中,BeaconKV普遍优于现有压缩方法,实现最高5.8倍内存压缩,几乎保持全缓存精度,并使吞吐量提升超过4.3倍。

原文摘要 · Abstract (English)

Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cache compression methods rely on recent queries to estimate future token importance, implicitly assuming these serve as reliable proxies for future attention patterns. We demonstrate that this assumption fails in long-horizon reasoning: certain decoding steps generate Thought Revisiting Tokens (TRT) that re-attend to distant previous context, such as task-solving plans formulated early in the trace. Through systematic analysis, we discover that queries corresponding to the TRT cluster into a small number of similarity groups in the embedding space. Based on this insight, we propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster, to anticipate which KV pairs will be revisited without storing the entire query history. Across four open-source LRMs and diverse reasoning benchmarks, BeaconKV generally outperforms existing compression methods, achieving up to $5.8\times$ memory reduction while nearly preserving full cache accuracy and improving throughput by over $4.3\times$.

推理优化缓存压缩大模型记忆管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。