让智能体学会协作动作依赖，提升多智能体决策效果
Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning
- 构建动作依赖图，让智能体根据全局协作上下文做决策
- 在部分可观测任务中成功率达80%-95%，远超其他方法的10%-25%
- 适合需要强协作与防冲突的多智能体系统研究者
在多智能体强化学习中，协调行动是合作的核心。成功的去中心化决策不仅依赖于个体动作质量，更取决于各智能体间动作的兼容性，以实现行为同步、避免冲突并满足全局约束。本文提出动作图策略（Action Graph Policies, AGP），建模智能体可用动作选择间的依赖关系，构建称为“协调上下文”的机制，使智能体能基于全局动作依赖进行决策。理论上，AGP生成的联合策略表达能力严格强于完全独立策略，且可实现比集中式价值分解方法贪婪执行更优的协同动作。实验表明，在具有部分可观测性和反协作惩罚的经典协调任务中，AGP成功率可达80%-95%，而其他MARL方法仅达10%-25%。此外，AGP在多种多智能体环境中均显著优于基线方法。
原文摘要 · Abstract (English)
Coordinating actions is the most fundamental form of cooperation in multi-agent reinforcement learning (MARL). Successful decentralized decision-making often depends not only on good individual actions, but on selecting compatible actions across agents to synchronize behavior, avoid conflicts, and satisfy global constraints. In this paper, we propose Action Graph Policies (AGP), that model dependencies among agents' available action choices. It constructs, what we call, \textit{coordination contexts}, that enable agents to condition their decisions on global action dependencies. Theoretically, we show that AGPs induce a strictly more expressive joint policy compared to fully independent policies and can realize coordinated joint actions that are provably more optimal than greedy execution even from centralized value-decomposition methods. Empirically, we show that AGP achieves 80-95\% success on canonical coordination tasks with partial observability and anti-coordination penalties, where other MARL methods reach only 10-25\%. We further demonstrate that AGP consistently outperforms these baselines in diverse multi-agent environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。