arXiv:2605.30842cs.LG2026-05

让记忆管理与推理并行,显著降低长上下文任务延迟

CoMem: Context Management with A Decoupled Long-Context Model

论文配图:CoMem: Context Management with A Decoupled Long-Context Model
图 1 · 摘自论文原文
  • 将记忆压缩与主推理流程解耦,异步并行处理
  • 在SWE-Bench-Verified上实现1.4倍延迟降低,性能几乎不变
  • 适合追求低延迟的长程智能体部署场景

上下文管理使代理模型通过迭代总结历史交互来解决长周期任务。然而,这一过程通常会因额外的摘要生成带来大量解码开销,显著影响部署时的端到端响应延迟。本文提出CoMem框架,将记忆管理与主代理工作流解耦,使两者可并行执行。我们设计了k步异步流水线,将记忆模型的摘要生成与代理推理重叠,有效掩盖上下文处理延迟。为确保异步设置下的鲁棒性,引入基于奖励的训练策略,使记忆模型捕获足够支持代理决策的统计信息。理论分析表明,CoMem在效率与效果之间优于耦合架构。在SWE-Bench-Verified上的实验显示,相比原始长上下文方案,CoMem实现1.4倍延迟提升,同时保持大部分性能。此外,延迟收益随系统吞吐量增加而更显著,为独立优化推理与记忆压缩提供了模块化路径。

原文摘要 · Abstract (English)

Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs substantial decoding overhead for the extra summarization tokens, which significantly affect the end-to-end response latency at deployment. In this paper, we introduce CoMem, a novel framework that decouples memory management from the primary agent workflow, enabling these processes to execute in parallel. We propose a $k$-step-off asynchronous pipeline that overlaps the memory model's summarization with the agent's inference, effectively masking the latency of context processing. To ensure robustness under this asynchronous setting, we introduce a reward-driven training strategy that aligns the memory model to capture sufficient statistics for the agent's decision-making. Theoretical analysis confirms that CoMem offers a superior efficiency-effectiveness trade-off compared to coupled architectures. Our extensive experimental results on SWE-Bench-Verified show that CoMem provides 1.4x latency improvements upon vanilla long-context solutions while preserving most of the performance. Furthermore, we demonstrate that these latency gains scale favorably with increased system throughput, offering a modular path forward for the independent optimization of agent reasoning and memory compression.

长上下文推理加速异步处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。