构建对抗性环境SC2BA,让多智能体算法互相比拼,更真实评估性能。
StarCraft+: Benchmarking Multi-agent Algorithms in Adversary Paradigm
- 创建可互相对抗的多智能体对战环境SC2BA,支持公平、易用、可定制的算法比拼。
- 在双算法对战和多算法混合对战中测试经典算法,发现其有效性与扩展性问题。
- 适合研究多智能体强化学习、博弈对抗与基准测试的学者与开发者。
深度多智能体强化学习(MARL)在协同智能领域快速发展,星战多智能体挑战(SMAC)是该领域广泛使用的基准。然而,当前MARL算法的对手通常为预设固定模式的内置AI,缺乏多样性和灵活性,影响评估效果。为此,本文构建了名为星际争霸II对战场(SC2BA)的多智能体算法对算法对抗环境,以在对抗范式下刷新MARL算法的评测标准。基于星际争霸2平台,SC2BA专为算法间对抗设计,兼顾公平性、可用性与可定制性,并开发了易用的对抗型PyMARL(APyMARL)库。在SC2BA基础上,我们以两种对抗模式评估经典MARL算法:双算法配对对抗与多算法混合对抗,前者测试算法间的成对对抗,后者关注多个算法行为的综合对抗。大量基准实验揭示了现有算法在有效性、敏感性与可扩展性方面的关键问题。相关环境及复现实验已开源至GitHub,我们相信本工作将推动MARL领域未来发展。
原文摘要 · Abstract (English)
Deep multi-agent reinforcement learning (MARL) algorithms are booming in the field of collaborative intelligence, and StarCraft multi-agent challenge (SMAC) is widely-used as the benchmark therein. However, imaginary opponents of MARL algorithms are practically configured and controlled in a fixed built-in AI mode, which causes less diversity and versatility in algorithm evaluation. To address this issue, in this work, we establish a multi-agent algorithm-vs-algorithm environment, named StarCraft II battle arena (SC2BA), to refresh the benchmarking of MARL algorithms in an adversary paradigm. Taking StarCraft as infrastructure, the SC2BA environment is specifically created for inter-algorithm adversary with the consideration of fairness, usability and customizability, and meantime an adversarial PyMARL (APyMARL) library is developed with easy-to-use interfaces/modules. Grounding in SC2BA, we benchmark those classic MARL algorithms in two types of adversarial modes: dual-algorithm paired adversary and multi-algorithm mixed adversary, where the former conducts the adversary of pairwise algorithms while the latter focuses on the adversary to multiple behaviors from a group of algorithms. The extensive benchmark experiments exhibit some thought-provoking observations/problems in the effectivity, sensibility and scalability of these completed algorithms. The SC2BA environment as well as reproduced experiments are released in \href{https://github.com/dooliu/SC2BA}{Github}, and we believe that this work could mark a new step for the MARL field in the coming years.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。