用权重扰动代替语言交流,让大模型协作更高效。
Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights

- 发送方将内部状态转为低秩权重扰动,直接作用于接收方。
- 在五项基准上提升8.5分准确率,减少超80%的处理文本量。
- 适合追求推理速度与资源效率的多智能体系统开发者。
多智能体大模型系统通常通过自然语言消息协作,但这种接口会将中间计算序列化为文本令牌,导致生成令牌成本高、预填充开销大、键值缓存内存占用高。本文提出一种替代通信方式:不将发送方消息追加到接收方上下文,而是将发送方隐藏状态编译为针对接收方架构的瞬时权重扰动。引入TFlow(Thought Flow)框架,用于已知且固定的接收方结构。每个查询中,冻结角色提示的发送方代理处理输入,一个学习的参数生成器将其内部激活映射为低秩LoRA扰动,作用于接收方模块。这些扰动仅在接收方生成阶段融合并应用,实现实例级适应,不永久修改模型或扩大接收方文本上下文。使用三个Qwen3-4B代理,TFlow在五项基准上相比独立接收方最高提升8.5准确率点,同时处理文本量减少达32.69%。相较基于文本的三代理基线,总处理文本量减少高达83.27%,推理耗时降低最多4.6倍,且在四项基准上保持竞争力。结果表明,瞬时低秩权重扰动可作为高效多智能体大模型协作的可执行通信介质。
原文摘要 · Abstract (English)
Multi-agent LLM systems usually collaborate by exchanging natural-language messages. This interface is simple and interpretable, but it forces each sender's intermediate computation to be serialized into tokens and then reprocessed by the receiver, thereby increasing the generated-token cost, prefill overhead, and KV-cache memory. We study an alternative communication interface: instead of appending a sender's message to the receiver's context, compile the sender's hidden states into a transient, receiver-specific weight perturbation. We introduce TFlow (Thought Flow), a weight-space communication framework for a known and fixed receiver architecture. For each query, frozen role-prompted sender agents process the input, and a learned parameter generator maps their internal activations into low-rank LoRA perturbations targeting the receiver's modules. These perturbations are fused and applied only during the receiver's generation, enabling instance-level adaptation without permanently changing the model or enlarging the receiver's text context. With three Qwen3-4B agents, TFlow improves over a standalone receiver by up to 8.5 accuracy points across five benchmarks while reducing processed tokens by up to 32.69%. Compared with a text-based three-agent baseline, it reduces total processed tokens by up to 83.27% and the wall-clock inference time by up to 4.6$\times$, while maintaining competitive accuracy on four of five benchmarks. These results suggest that transient low-rank weight perturbations can serve as an executable communication medium for efficient multi-agent LLM collaboration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。