通过隐空间通信提升策略学习，实现高效去中心化多智能体强化学习。
Communication-Enhanced Tutoring for Efficient Decentralized Multi-Agent Reinforcement Learning
- 训练时共享隐状态信息，提升策略决策能力
- 测试时无通信条件下性能超越现有方法
- 适用于复杂场景下需独立执行的多智能体系统
集中训练、去中心化执行（CTDE）是多智能体强化学习的主流范式，使智能体在测试时独立行动，同时在训练中利用额外信息。然而，基于价值分解的主流方法受限于训练与执行阶段的部分可观测性，导致学习效率和最终性能不足。为此，本文提出‘辅导’框架：训练阶段，智能体在隐空间共享信息以学习高性能策略；随后，为恢复去中心化执行，这些策略主动适应通信缺失，并被提炼为仅依赖局部观测的版本。我们在Hallway（据我们所知，此前未在无测试通信条件下解决）、更困难的SMAC设置以及SMACv2上验证了该方法的有效性。
原文摘要 · Abstract (English)
Centralized Training with Decentralized Execution (CTDE) is the dominant paradigm in multi-agent reinforcement learning (MARL), enabling agents to act independently at test time while leveraging additional information during training. However, the most prominent methods within CTDE, based on value decomposition, are limited in learning efficiency and final performance by partial observability in both training and execution. To overcome this limitation, in this work, we propose the framework of tutoring: In training, the agents share information in their latent space to develop well-informed policies that achieve strong performance. Then, to recover decentralized execution, these policies concurrently adjust to anticipate lack of communication, and they are distilled into counterparts that rely solely on local observations. We demonstrate the effectiveness of our approach on Hallway, which, to the best of our knowledge, has not been solved before without test-time communication, SMAC under settings more difficult than the standard ones, and SMACv2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。