arXiv:2603.04833cs.MAcs.AI2026-03

让多智能体通信更高效:自动判断何时何人通信,提升协作能力。

SCoUT: Scalable Communication via Utility-Guided Temporal Grouping in Multi-Agent Reinforcement Learning

  • 通过软分组抽象,动态确定通信对象和时机。
  • 利用反事实优势计算,精准分配通信决策的奖励信号。
  • 训练时集中优化,执行时完全去中心化,适合大规模系统。

通信可提升部分可观测多智能体强化学习(MARL)中的协作效率,但需决定何时及与谁通信,且单条消息对后续奖励的影响难以分离。我们提出SCoUT(基于效用引导的时间分组的可扩展通信),通过传统MARL中的时间与智能体抽象解决上述挑战。训练期间,每K个环境步(宏步)使用Gumbel-Softmax重采样软智能体组,形成潜在聚类,作为接收者选择的可微先验。相同分组下,群组感知的评价网络为每组预测价值,并通过相同软分配映射至个体基线,降低评价网络复杂度与方差。每个智能体采用三头策略:环境动作、发送决策与接收者选择。为获得精确通信学习信号,我们通过解析移除发送者贡献的方式,推导出反事实通信优势,实现对发送与接收选择决策的精确信用分配。执行时,所有集中式训练组件被丢弃,仅运行个体策略,保持去中心化执行。项目网站、视频与代码:https://scout-comm.github.io/

原文摘要 · Abstract (English)

Communication can improve coordination in partially observed multi-agent reinforcement learning (MARL), but learning \emph{when} and \emph{who} to communicate with requires choosing among many possible sender-recipient pairs, and the effect of any single message on future reward is hard to isolate. We introduce \textbf{SCoUT} (\textbf{S}calable \textbf{Co}mmunication via \textbf{U}tility-guided \textbf{T}emporal grouping), which addresses both these challenges via temporal and agent abstraction within traditional MARL. During training, SCoUT resamples \textit{soft} agent groups every \(K\) environment steps (macro-steps) via Gumbel-Softmax; these groups are latent clusters that induce an affinity used as a differentiable prior over recipients. Using the same assignments, a group-aware critic predicts values for each agent group and maps them to per-agent baselines through the same soft assignments, reducing critic complexity and variance. Each agent is trained with a three-headed policy: environment action, send decision, and recipient selection. To obtain precise communication learning signals, we derive counterfactual communication advantages by analytically removing each sender's contribution from the recipient's aggregated messages. This counterfactual computation enables precise credit assignment for both send and recipient-selection decisions. At execution time, all centralized training components are discarded and only the per-agent policy is run, preserving decentralized execution. Project website, videos and code: \hyperlink{https://scout-comm.github.io/}{https://scout-comm.github.io/}

多智能体通信优化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。