用图神经网络动态更新关键词保留策略，提升长文本生成效率
GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction
- 将token建模为节点，通过图结构传播信号动态调整重要性
- 在LLM推理中实现更优的上下文保留，减少关键信息丢失
- 可直接接入现有缓存机制，适合需要长序列处理的研究者
高效的关键值（KV）缓存管理对大语言模型处理长文本序列至关重要，但内存限制常制约性能。传统基于注意力分数的top-k静态选择策略无法捕捉推理过程中词元间动态演化的隐含依赖关系。为此，我们提出GraphKV，一种基于图结构的KV缓存淘汰框架，重新定义词元选择机制。在GraphKV中，词元作为带重要性评分的节点，边表示其相似性关系。通过衰减-信号传播机制，重要性在图中动态更新，实现对最具上下文意义词元的自适应保留。GraphKV可无缝集成至SnapKV、PyramidKV等现有方法中，支持即插即用。代码将开源于Github。
原文摘要 · Abstract (English)
Efficient Key-Value (KV) cache management is essential for processing long text sequences in large language models (LLMs), where memory constraints often limit performance. Conventional KV eviction strategies, such as top-k selection based on attention scores, depend on static heuristics that fail to capture the evolving implicit dependencies among tokens during inference. To overcome this, we propose GraphKV, a graph-based framework that redefines token selection for KV cache compression. In GraphKV, tokens are modeled as nodes with importance scores, and edges represent their similarity relationships. Through a decay-signal-propagation mechanism, token importance is dynamically updated by propagating information across the graph, enabling adaptive retention of the most contextually significant tokens. GraphKV can be seamlessly utilized in existing KV cache eviction methods such as SnapKV and PyramidKV in a plug-and-play manner. Codes will be released on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。