分离通信与策略,让多智能体在带宽受限时仍保持高效协作。
Decoupling Communication from Policy: Robust MARL under Bandwidth Constraints

- 设计独立通信路径,解耦策略与通信的隐状态空间。
- 在有限带宽下性能下降极小,优于现有方法。
- 适合无人机群等资源受限的实际协同场景。
通信在多智能体强化学习中促进协作,但现实应用如无人机搜救常面临严重带宽限制。现有通信架构常将共享隐状态同时用于策略执行与智能体间通信,导致压缩消息尺寸会直接削弱策略的隐空间,引发性能显著下降。本文提出两个贡献:一是引入β,一个归一化的单智能体带宽预算,统一稀疏性、通信轮次和消息维度为单一可比约束;二是提出SLIM,一种最小化架构,将通信路径与策略隐状态解耦,实现通信带宽与策略容量的独立调控,并支持实时通信。在多个部分可观测的MARL基准测试上验证,该方法在通信受限条件下达到最优性能,具备良好的可扩展性与鲁棒性,带宽减少时仅产生微小性能损失。
原文摘要 · Abstract (English)
Communication enables coordination in multi-agent reinforcement learning (MARL), but many real-world applications, e.g., search-and-rescue with drone swarms, operate under severe bandwidth constraints. Many communication architectures still expose a coupled bottleneck in which a shared latent representation is used for both policy execution and inter-agent communication. Consequently, reducing message size directly limits the policy's latent space, often leading to significant performance degradation. We address this with two contributions. First, we introduce $β$, a normalised per-agent bandwidth budget that unifies sparsity, rounds, and message dimension into a single comparable constraint. Second, we provide SLIM, a minimal architecture that decouples the communication pathway from the policy's latent representation, allowing us to isolate the effect of bandwidth from the effect of policy capacity while benefiting from in-step communication. We evaluate our method on several partially-observable MARL benchmarks, where communication is essential. Our approach achieves state-of-the-art performance and exhibits scalability and robustness under limited communication, with only marginal degradation as bandwidth is reduced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。