通过提交状态区分信息生命周期,实现多轮智能体的高效缓存压缩。
CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents
- 基于提交前后的对比判断缓存页是否暂时休眠或已结束任务。
- 在多个基准测试中,内存占用降低37.6%,推理速度提升28.4%。
- 适合需要长对话记忆的复杂多轮智能体系统使用。
多轮推理与行动(ReAct)智能体在运行过程中会累积越来越多的推理轨迹、工具调用和观测数据,导致其键值(KV)缓存持续增长,增加推理时的内存占用和注意力计算开销。现有缓存压缩方法通常依据当前注意力分数低来淘汰状态,但当前低关注度并不意味着未来无用——某些暂时休眠的信息可能在后续步骤中变得重要。基于快照的淘汰策略未能明确区分此类临时休眠信息与已完成角色的信息。本文提出CommitKV,通过检测提交(commit)转换来识别KV缓存的生命周期。具体而言,CommitKV将已完成的智能体事件划分为标记页,比较每个候选页在工具调用提交前与提交后观测结果融入后的删除影响。基于这些成对测量,区分出休眠页与高到低完成度候选页。随后采用贪心联合测试,仅当组合后的提交后影响仍在阈值内时才接受淘汰。最后,在后续压缩检查点,被接受的页面被移除,一组等待提交后测量的页面被保护,其余状态则在缓存预算内保留,且使用相同的标记索引作为键、值及绝对位置。实验表明,CommitKV在多个基准上显著降低内存使用,加速端到端推理,并实现更高准确率,优于现有压缩方法。
原文摘要 · Abstract (English)
Multi-turn Reasoning-and-Acting (ReAct) agents accumulate growing trajectories of reasoning, tool calls, and observations. Their key-value (KV) caches grow accordingly, increasing memory use and attention cost during model inference. Existing KV cache compression methods reduce these costs by evicting states with low attention scores. However, low attention in the current turn does not imply future irrelevance, as temporarily inactive information may become important later. Snapshot-based eviction methods therefore do not explicitly distinguish temporarily dormant information from information that appears to have completed its role. In this paper, we present CommitKV, which identifies KV lifecycles through commit transitions. Specifically, CommitKV first divides completed agent events into token pages and compares each eligible page's deletion effect before a tool-call commit and after the commit's returned observation has been incorporated. Based on these paired measurements, CommitKV distinguishes dormant pages from high-to-low completion candidates. It then applies a greedy joint test, accepting candidates for retirement only when their combined post-commit effect remains bounded. Finally, at a later compression checkpoint, accepted pages are excluded, a bounded set of pages awaiting post-commit measurement is protected, and the remaining KV states are retained within the cache budget using the same token indices for keys, values, and absolute positions. These mechanisms ensure that CommitKV can distinguish dormant information from information that has completed its observed role and can be safely removed. Experiments on various benchmarks show that CommitKV reduces agent memory use, accelerates end-to-end inference, and achieves higher accuracy than existing KV cache compression methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。