让大模型直接读取并行分支的缓存,加速推理且保持精度。
Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows

- 设计新框架,让合成器直接使用并行分支的键值缓存,跳过文本拼接。
- 在9个数据集上7个表现超越传统拼接法,首字延迟降低2.5至11倍。
- 适合需要高效多分支协同的智能体系统,如数学推理与代码生成。
大型语言模型越来越多地作为智能体系统的执行引擎,但仍通过串行文本接口消耗上下文,与现代结构化智能体工作流不匹配。在这些工作流中,独立分支可并行探索子任务、检索证据或生成候选解,再进行最终合成。现有系统通常通过拼接文本输出合并分支,丢弃了并行结构并带来冗余预填充计算。本文提出Parallel-Synthesis,一种即插即用框架,使合成器能直接消费并行工作代理生成的键值缓存。该框架结合缓存映射器校准各分支独立生成的缓存,并使用微调后的合成适配器实现从非串行缓存接口生成内容。训练数据包含并行缓存上下文、跨缓存聚合以及标准文本拼接合成中的推理行为蒸馏。在涵盖数学、科学问答、代码生成、GAIA和多智能体数据库诊断的九个下游数据集上,Parallel-Synthesis在七项任务中达到或超过文本合成性能,其余两项也接近。同时,首字生成时间缩短2.5至11倍,表明直接基于缓存的合成是更原生高效的并行分支合成接口。
原文摘要 · Abstract (English)
Large language models increasingly serve as execution engines for agentic systems, yet they still consume context through a sequential text interface. This creates a mismatch with modern structured agent workflows, in which independent branches explore subtasks, retrieve evidence, or generate candidate solutions before a final synthesis step. Existing systems typically merge these branches by concatenating their textual outputs, which discards the parallel structure and incurs redundant prefill computation. In this work, we introduce Parallel-Synthesis, a plug-and-play framework that enables a synthesizer to directly consume the KV caches produced by parallel worker agents. Parallel-Synthesis combines a cache mapper that calibrates independently generated branch caches with a fine-tuned synthesizer adapter that enables generation from this non-sequential cache interface. We train Parallel-Synthesis using data that exposes the synthesizer to parallel cache contexts, teaches aggregation across cached branches, and distills reasoning behavior from standard text-concatenation-based synthesis. Across nine downstream datasets spanning math, science QA, code generation, GAIA, and multi-agent database diagnosis, Parallel-Synthesis matches or outperforms text-based synthesis on seven datasets and remains close on the other two. It also reduces time-to-first-token by 2.5x-11x, suggesting that direct cache-based synthesis is a promising interface for more native and efficient synthesis over parallel agent branches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。