arXiv:2609.04852cs.LG2026-09

让消费级显卡运行百万级上下文的长时智能体,保持历史细节不丢失。

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU

论文配图:KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
图 1 · 摘自论文原文
  • 用分页键值状态虚拟化超长工作区,跨显存/内存/固态硬盘管理历史
  • 在百万级上下文任务中提升成功率至48.4%,推理效率优于传统压缩方法
  • 适合本地部署长时智能体,支持交互式响应,可扩展至超大工作区

现代大模型智能体依赖持久化工作区,其累积历史可能远超GPU键值缓存和模型原生上下文窗口。现有系统通常将旧历史压缩为摘要或后期检索文本,导致细粒度执行证据丢失或重复预填充已处理内容。本文提出KVMem,一种键值上下文虚拟化系统,将溢出的工作区历史以分页键值状态形式存储于GPU内存、主机内存与NVMe之间。KVMem利用轻量级、模型原生注意力空间索引,按查询选择相关历史块,并生成受模型原生上下文窗口限制的动态执行视图。在涵盖百万级历史的LongMemEval、MemoryAgentBench与AgentLongBench等长上下文智能体基准测试中,KVMem普遍优于基于压缩的方法,在DeepSWE测试中,Qwen3.8-27B的任务成功率从43.8%提升至48.4%。本地部署评估显示,KVMem可在配备24GB RTX 5090笔记本显卡的普通电脑上,以四倍于模型原生256K-token上下文窗口的能力,虚拟化最大100万令牌的工作区;单会话下生成速度约50 tokens/s,实现本地智能体的交互式响应。更广泛地,通过解耦可访问工作区大小与模型原生上下文窗口,KVMem为构建长期运行的智能体提供了实用路径。

原文摘要 · Abstract (English)

Modern LLM agents operate in persistent workspaces whose accumulated history can exceed both GPU KV capacity and the model's native context window. Existing systems typically compact older context into summaries or retrieve it later as text, either losing fine-grained execution evidence or repeatedly prefilling content that the model has already processed. We present KVMem, a KV-context virtualization system that preserves overflowed workspace history as paged KV state across GPU memory, host memory, and NVMe. KVMem uses lightweight, model-native attention-space indexes to select relevant historical blocks and materializes a query-dependent execution view bounded by the model's native context window. Extensive evaluations on long-context agent benchmarks spanning histories up to one million tokens, including LongMemEval, MemoryAgentBench, and AgentLongBench, show that KVMem generally achieves higher task utility and greater inference efficiency than compaction-based approaches, the de facto standard for handling context overflow. In the DeepSWE long-context test with Qwen3.8-27B, KVMem improves task success from 43.8% with compaction-only context management to 48.4%. In our local-deployment evaluation, KVMem runs Qwen3.6/3.8-27B NVFP4 with MTP on an off-the-shelf laptop equipped with a 24\,GB RTX 5090 Laptop GPU, virtualizing agent workspaces of up to 1M tokens-four times the model's native 256K-token context window. In a single-session setting, KVMem generates $\sim$50 tokens/s, providing interactive responsiveness for local agent execution. More broadly, by decoupling addressable workspace size from the LLM's native context window, KVMem provides a practical path toward long-running agents whose workspaces can grow beyond that window.

长上下文智能体虚拟化本地部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。