提出MAGPO框架，让多智能体在受限通信下更高效协作学习。
Multi-Agent Guided Policy Optimization
- 用自回归联合策略实现可扩展的协同探索
- 在43个任务中优于主流CTDE基线，媲美集中式方法
- 兼具理论保证与实用价值，适合需要稳定协作的场景
由于部分可观测性和通信受限等实际约束，集中训练、分散执行（CTDE）已成为合作多智能体强化学习（MARL）的主流范式。然而，现有CTDE方法往往未能充分利用集中训练，或缺乏理论保障。本文提出多智能体引导策略优化（MAGPO），通过将集中式引导与分散执行结合，更好利用集中训练。MAGPO采用自回归联合策略实现可扩展的协同探索，并显式对齐其与分散策略，确保在部分可观测性下的可部署性。我们提供了单调策略改进的理论保证，并在6个不同环境的43个任务上进行实验验证。结果表明，MAGPO持续优于强基准，性能匹配甚至超越完全集中式方法，为分散式多智能体学习提供了一种有原则且实用的解决方案。代码与数据见：https://github.com/liyheng/MAGPO。
原文摘要 · Abstract (English)
Due to practical constraints such as partial observability and limited communication, Centralized Training with Decentralized Execution (CTDE) has become the dominant paradigm in cooperative Multi-Agent Reinforcement Learning (MARL). However, existing CTDE methods often underutilize centralized training or lack theoretical guarantees. We propose Multi-Agent Guided Policy Optimization (MAGPO), a novel framework that better leverages centralized training by integrating centralized guidance with decentralized execution. MAGPO uses an autoregressive joint policy for scalable, coordinated exploration and explicitly aligns it with decentralized policies to ensure deployability under partial observability. We provide theoretical guarantees of monotonic policy improvement and empirically evaluate MAGPO on 43 tasks across 6 diverse environments. Results show that MAGPO consistently outperforms strong CTDE baselines and matches or surpasses fully centralized approaches, offering a principled and practical solution for decentralized multi-agent learning. Our code and experimental data can be found in https://github.com/liyheng/MAGPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。