arXiv:2511.01554cs.MAcs.IT2025-11

让智能体学会按需发送信息,精确控制通信量。

Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning

  • 用可微离散通信框架,让智能体自主调节信息发送精度。
  • 通信带宽降低超十倍,性能不降反升。
  • 简单Transformer+该方法胜过复杂专用设计,适合高效多智能体系统。

多智能体强化学习中的有效通信对成功至关重要,但受限于带宽。以往方法仅能决定是否通信,无法优化消息精度。我们通过扩展可微离散通信学习(DDCL)框架,支持无界信号,使其成为适用于任意MARL架构的通用、即插即用模块。实验验证三个关键结果:第一,在受控环境中定性分析显示,智能体能根据任务需求动态调节消息精度;第二,将该变体集成至四种前沿MARL算法,通信带宽降低超过一个数量级,同时保持或超越原有性能;第三,直接验证了MARL通信中的“苦涩教训”:基于Transformer的简单策略结合DDCL,性能媲美甚至超过复杂的专用架构,质疑了定制化通信设计的必要性。

原文摘要 · Abstract (English)

Effective communication in multi-agent reinforcement learning (MARL) is critical for success but constrained by bandwidth, yet past approaches have been limited to complex gating mechanisms that only decide \textit{whether} to communicate, not \textit{how precisely}. Learning to optimize message precision at the bit-level is fundamentally harder, as the required discretization step breaks gradient flow. We address this by generalizing Differentiable Discrete Communication Learning (DDCL), a framework for end-to-end optimization of discrete messages. Our primary contribution is an extension of DDCL to support unbounded signals, transforming it into a universal, plug-and-play layer for any MARL architecture. We verify our approach with three key results. First, through a qualitative analysis in a controlled environment, we demonstrate \textit{how} agents learn to dynamically modulate message precision according to the informational needs of the task. Second, we integrate our variant of DDCL into four state-of-the-art MARL algorithms, showing it reduces bandwidth by over an order of magnitude while matching or exceeding task performance. Finally, we provide direct evidence for the \enquote{Bitter Lesson} in MARL communication: a simple Transformer-based policy leveraging DDCL matches the performance of complex, specialized architectures, questioning the necessity of bespoke communication designs.

多智能体通信优化可微离散Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。