通过感知多轮意图动态压缩键值缓存，显著降低大模型代理推理的内存与带宽开销。
IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference
- 基于跨轮次意图构建查询记忆，用注意力机制评分历史令牌并保留关键信息。
- 在8k缓存预算下，峰值请求令牌减少23.9%~30.7%，最坏情况下降77.8%。
- 无需修改原有结构，适配现有前缀缓存，适合长周期智能体部署场景。
多轮大模型代理将短查询扩展为包含工具调用、搜索结果和中间推理的长轨迹，导致键值（KV）缓存与读取带宽增长数个数量级，成为长时序代理服务的主要瓶颈。我们提出IntentKV，一种无需微调基础模型的可学习缓存剪枝方法。它维护会话级的跨轮次意图记忆，通过记忆注意力规则对活跃历史令牌打分，并引入零初始化残差头，对当前查询的K向量进行交叉注意力。为保持与前缀缓存的兼容性，淘汰操作采用槽位映射重定向：被移除的位置指向哨兵死槽，而保留的K/V行、RoPE相位和槽位标识保持不变。在严格缓存预算下，IntentKV几乎无精度损失地达到全缓存基线性能：在8k KV预算下，Qwen3-8B的平均峰值请求令牌下降23.9%，Qwen2.5-14B下降30.7%。在所有方法均能完成的100个最长BCP查询上，IntentKV-8k将最坏情况下的峰值请求令牌从92.3k降至20.5k（减少77.8%），最坏情况原始KV读取从411M降至31M（减少92.6%）。
原文摘要 · Abstract (English)
Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning. Both KV memory and KV read bandwidth grow by orders of magnitude across a single trajectory, making the key-value (KV) cache, not parameter compute, the dominant serving bottleneck for long-horizon agents. We introduce IntentKV, learned KV pruning that keeps the base LLM frozen. IntentKV maintains a session-level QueryMemory of cross-turn intent, scores live history tokens with a memory-attention rule, and adds a zero-initialized residual head with cross-attention over current-query K-vectors. To stay composable with prefix caches, eviction is a slot-map redirection: dropped positions route to a sentinel dead slot while surviving K/V rows, RoPE phases, and slot identities stay in place. IntentKV matches the no-pruning full-cache baseline with almost no accuracy drop under tight KV budgets: at an 8k KV budget, mean peak request tokens drop 23.9% on Qwen3-8B and 30.7% on Qwen2.5-14B. On the 100 longest BCP queries that all methods complete on Qwen2.5-14B, IntentKV-8k further cuts worst-case peak request tokens from 92.3k to 20.5k, a 77.8% reduction, and worst-case raw KV reads from 411M to 31M, a 92.6% reduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。