压缩代理间隐状态通信，用少量保留信息实现更高效协作。
When Less Latent Leads to Better Relay: Information-Preserving Compression for Latent Multi-Agent LLM Collaboration
- 通过正交回填机制，将丢弃的隐状态残差注入保留部分。
- 仅保留9.9%~20.2%的提示键值对，带宽降低4.7倍，显存降8%。
- 适合追求低延迟、高效率多智能体大模型协作的场景。
多智能体大模型系统正从离散标记消息转向更丰富的状态传递。近期工作如LatentMAS传输完整键值（KV）缓存，但代价高昂。本文将KV缓存淘汰策略引入该场景，提出正交回填（OBF），将被丢弃的KV状态低秩残差正交注入保留部分。仅需保留9.9%–20.2%的提示KV，压缩中继可使带宽减少4.7倍，GPU内存降低8%，且在九个基准测试上误差低于5%的时延开销，多数任务表现优于全量传输。OBF在所有九项任务上均达到或超过按头淘汰法效果，增益与淘汰法造成的精度损失呈正相关（三模型尺度下r=0.78），有效弥补淘汰损失。代码已开源。
原文摘要 · Abstract (English)
Multi-agent LLM systems are moving beyond discrete-token messages toward richer relays that preserve internal state. Recent work such as LatentMAS transmits full key-value (KV) caches between agents but pays a high memory and communication cost. We adapt KV-cache eviction to this setting and introduce \textbf{Orthogonal BackFill (OBF)}, which injects a low-rank residual from the discarded KV states back into the retained ones, orthogonal to what is already kept. With only $9.9\%$-$20.2\%$ of the prompt KV retained, compressed relay cuts bandwidth by $4.7\times$ and GPU memory by $8\%$ at under $5\%$ wall-clock overhead, and stays close to full relay in accuracy across nine benchmarks, ahead of it on several. OBF matches or improves over headwise eviction on all nine, and its gain is proportional to the accuracy gap eviction opens against full relay ($r{=}0.78$ across three model scales), so it gives back part of what eviction takes. Code is available at https://github.com/markli404/When-Less-Latent-Leads-to-Better-Relay.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。