arXiv:2605.23296cs.AI2026-05被引 2

提出并行压缩技术,让长时对话代理推理更快更可控。

Parallel Context Compaction for Long-Horizon LLM Agent Serving

论文配图:Parallel Context Compaction for Long-Horizon LLM Agent Serving
图 1 · 摘自论文原文
  • 采用并行压缩替代串行摘要,提升处理效率
  • 相同压缩量下,端到端耗时降低,吞吐量提升
  • 支持精确控制摘要长度,适合需要稳定性的场景

长时序大模型代理会累积大量对话历史,超出模型上下文窗口。传统基于大模型的摘要压缩虽能限制上下文长度,但存在本质信息丢失,且摘要过程阻塞推理达数十秒。此外,提示指令常被忽略,导致每次运行的输出词数和保留信息波动大,知识留存不可预测。本文提出并行压缩机制,针对涵盖8B至120B参数、混合密集与MoE架构、含推理与非推理模型的四种骨干网络,在HotpotQA多跳问答与LoCoMo长上下文对话基准上进行评估。并行压缩使操作者可精细控制摘要体积,支持每块针对性提示工程;在匹配压缩解码量条件下,相比串行同步基线,显著降低端到端墙钟时间并提升压缩吞吐量。

原文摘要 · Abstract (English)

Long-horizon LLM agents accumulate growing conversation histories that eventually exceed the model's context window. Context compaction via LLM-based summarization keeps the conversation bounded, but summarization is inherently lossy and the blocking call stalls agent inference for tens of seconds. Moreover, the operator has no fine-grained control over summary volume since prompt instructions are largely ignored, and as context grows, both the amount of output tokens the model produces and the information it retains fluctuate substantially from run to run, making the agent's retained knowledge unpredictable across runs. We introduce \textbf{parallel compaction} for long-horizon agentic flows and characterize it against the sequential synchronous baseline across four backbones spanning 8B to 120B parameters, mixing dense and MoE architectures with reasoning and non-reasoning models, on the HotpotQA multi-hop QA and LoCoMo long-context dialogue benchmarks. Parallel compaction gives the operator fine-grained, predictable control over summary volume and enables more targeted prompt engineering per block. At matched compaction decode volume, it reduces end-to-end wall time and improves compaction throughput over the sequential baseline.

长上下文推理优化并行计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。