通过图注意力整合队友信息,让多智能体协作更高效。
SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learning

- 用图变换器在智能体间传递内容相关的协作信号。
- 五项任务中均优于12种基线方法,性能显著提升。
- 适合需要强协作的复杂环境,如游戏或机器人团队。
在部分局部观测下,协作式多智能体强化学习面临根本性信息瓶颈:做出联合最优动作所需的知识分散在团队中,而每个智能体必须在未获知队友观测、意图或动作的情况下做出决策。现有方法要么忽略该瓶颈,将信息压缩为标量混合信号,要么通过学习通信通道绕开。本文将行动协调视为智能体间结构化信息整合问题,提出「基于整体信息整合的结构化智能体协作」(SACHI),通过在智能体协调图上使用图变换器卷积,在决策前将接收端敏感、内容依赖的信号注入每个智能体表示。我们在涵盖空间、沟通与对抗性协调挑战的五个合作任务上评估SACHI,对比十二种基线。结果表明,SACHI在所有任务中表现持平或超越最佳基线;通过归一化指标、自助法置信区间、弗里德曼排名和性能分析等严格统计检验,证实其优势具有统计显著性、跨环境鲁棒性,且非由模型容量增加导致。参数匹配的消融实验进一步将性能提升归因于消息传递算子的内容依赖程度这一单一架构特性。
原文摘要 · Abstract (English)
Cooperative multi-agent reinforcement learning agents that act on partial local observations face a fundamental information bottleneck: the knowledge needed to select jointly optimal actions is scattered across the team, yet each agent must commit to a decision without access to its teammates' observations, intentions, or chosen actions. Existing methods either ignore this bottleneck, compress it into a scalar mixing signal, or route around it with learned communication channels. Framing action coordination as a problem of structured information integration among agents, we propose \textit{structured agent coordination via holistic information integration}, or SACHI, in which graph transformer convolutions over an inter-agent coordination graph enrich each agent's representation with receiver-sensitive, content-dependent signals from teammates prior to action selection. We evaluate SACHI across five cooperative tasks spanning spatial, communicative, and adversarial coordination challenges against twelve baselines. SACHI consistently matches or outperforms the best baseline on every task, and rigorous aggregate statistical analyses, including normalized metrics with bootstrap confidence intervals, Friedman ranking, and performance profiling, confirm that this advantage is statistically significant, robust across environments, and not attributable to increased model capacity. Parameter-matched ablations further trace the source of the gains to a single architectural property: the degree of content-dependence in the message-passing operator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。