多智能体系统何时更优?关键在通信受限下的信息压缩平衡。
When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

- 通过信息瓶颈视角,分析多智能体间通信受限时的上下文压缩机制。
- 实验显示:通信接近充足时,弱模型用多智能体提升显著;强模型反而受损。
- 适用于研究智能体协作设计或优化通信效率的研究者。
基于大语言模型的多智能体系统(MAS)在复杂任务中展现出潜力,但其相对于单智能体系统(SAS)的优势尚不明确,且表现不一致。本文从信息瓶颈角度揭示两者的本质差异:SAS将完整推理过程存于共享上下文,而MAS使用隔离的局部上下文并通过有限传递消息连接。当传递带宽无限时,任何SAS均可被一个传输完整上游上下文的MAS模拟;因此,MAS的非平凡优势源于有界通信。此时压缩会引入根本性权衡:减少冗余上下文可提升效率,但可能损失任务相关信息。我们形式化该权衡为受有效参数β控制的信息瓶颈,反映模型能力如何影响平衡。实验在五个基准、三种模型规模上进行18次对照测试,发现当通信接近充足时,弱模型始终受益;而通信导致信息损失时,强模型因已能从冗余上下文中提取有用信息,压缩反而带来损失。研究表明,多智能体设计本质上是信息瓶颈优化问题,解释了通信受限下合作的增益与风险。
原文摘要 · Abstract (English)
LLM powered multi-agent systems (MAS) have emerged as a promising paradigm for complex tasks. However, their advantages over single-agent systems (SAS) remain unclear, with performance varying inconsistently across settings. Here, we provide an information bottleneck perspective on elucidating the differences between MAS and SAS. Specifically, our key observation is that a SAS accumulates its full reasoning trace in one shared context, while a MAS uses isolated local contexts connected by bounded relay messages. We show that, under infinite relay bandwidth, any SAS can be simulated by a MAS that transmits the full upstream context. Thus, the nontrivial advantage of MAS arises under bounded relays, where compression introduces a fundamental trade-off: reducing redundant context can improve efficiency, but may also incur loss of task-relevant information. We formalize this trade-off as an information bottleneck controlled by an effective parameter $β$, which captures how the balance shifts with model capability, and shows that MAS gains arise when context reduction outweighs relay information loss. We conduct 18 controlled experiments across five benchmarks and three model scales to validate our theoretical studies. We observe that MAS consistently helps when relays are near-sufficient, especially for weaker models. In contrast, MAS gains shrink or reverse when relays incur information loss, especially for stronger models that can already extract useful information from redundant context and thus gain little from compression. Our study shows that multi-agent design is fundamentally an information-bottleneck optimization problem. This perspective explains when bounded inter-agent communication helps or hurts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。