让工具型大模型重复使用工具知识，提速并省内存。
ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
- 按资源独立缓存，避免重复编码工具信息。
- 比标准方法快3.655倍，内存减少92.43%。
- 适合需要频繁调用工具的智能体系统。
智能体语言模型在不同请求中反复编码工具和技能模板，导致传统前缀缓存无法复用其键值（KV）状态。我们提出ReCache框架，独立缓存资源表示，降低推理时的计算与内存开销。资源注意力消除跨资源交互，并分配局部位置，生成与组合无关的KV块。ReCache随后仅保留贡献关键的层-头-组路径，并通过结构与语义剪枝保留调用必要字段。我们在七个公开工具与技能使用数据集构建的基准上评估，包括资源无交集测试。资源注意力达到82.3%对82.4%的调用准确率（Inv-F1），同时实现3.655倍的时间到首个词元加速。完整框架将分配的KV张量内存减少92.43%，注意力加速1.423倍。结果表明，分离可重用模板编码与选择性资源访问，能显著降低智能体推理成本，且效果损失有限。代码已开源：https://github.com/EIT-NLP/ReCache。
原文摘要 · Abstract (English)
Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing their key--value (KV) states. We introduce \textbf{ReCache}, a framework for independently caching resource representations while reducing their inference-time computational and memory overhead. Resource-wise attention removes cross-resource interactions and assigns resource-local positions, producing composition-invariant KV blocks. ReCache then restricts resource visibility to contribution-selected layer--KV-head-group routes and retains only invocation-critical fields through structural and semantic pruning. We evaluate ReCache on a benchmark assembled from seven public tool- and skill-use datasets, including resource-disjoint tests. Resource-wise attention matches dense invocation performance (82.3\% versus 82.4\% Inv-F1) while providing a 3.655$\times$ time-to-first-token speedup. The complete framework reduces allocated KV-tensor memory by 92.43\% and accelerates attention by 1.423$\times$. These results show that separating reusable schema encoding from selective resource access substantially reduces agentic inference costs with limited effectiveness loss. The code is available at https://github.com/EIT-NLP/ReCache.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。