综述多智能体强化学习在各类游戏中的应用与挑战
A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
- 系统梳理从回合制到实时游戏的MARL应用方法
- 指出非平稳性、部分可观测等核心挑战及其解决案例
- 适合对游戏AI和MARL研究感兴趣的开发者与学者
近年来，多智能体强化学习（MARL）在现代游戏中的应用潜力不断显现。从基础研究到《星海争霸II》的AlphaStar和《Dota 2》的OpenAI Five等里程碑成果，MARL已通过自对弈、监督学习和深度强化学习等技术，在多种游戏环境中实现超人类表现。本文全面考察MARL在从双人回合制游戏到实时多智能体游戏（包括体育类、第一人称射击类、即时战略类和多人在线战术竞技类游戏）的应用。分析了非平稳性、部分可观测性、稀疏奖励、团队协作与可扩展性等关键挑战，并总结了在《火箭联盟》《我的世界》《雷神之锤3》《星海争霸II》《Dota 2》《王者荣耀》等游戏中的成功实践。论文还提出一种评估游戏复杂度的新方法，展望未来研究方向，推动MARL在游戏开发中的应用，激发该快速演进领域持续创新。
原文摘要 · Abstract (English)
Recent advancements in multi-agent reinforcement learning (MARL) have demonstrated its application potential in modern games. Beginning with foundational work and progressing to landmark achievements such as AlphaStar in StarCraft II and OpenAI Five in Dota 2, MARL has proven capable of achieving superhuman performance across diverse game environments through techniques like self-play, supervised learning, and deep reinforcement learning. With its growing impact, a comprehensive review has become increasingly important in this field. This paper aims to provide a thorough examination of MARL's application from turn-based two-agent games to real-time multi-agent video games including popular genres such as Sports games, First-Person Shooter (FPS) games, Real-Time Strategy (RTS) games and Multiplayer Online Battle Arena (MOBA) games. We further analyze critical challenges posed by MARL in video games, including nonstationary, partial observability, sparse rewards, team coordination, and scalability, and highlight successful implementations in games like Rocket League, Minecraft, Quake III Arena, StarCraft II, Dota 2, Honor of Kings, etc. This paper offers insights into MARL in video game AI systems, proposes a novel method to estimate game complexity, and suggests future research directions to advance MARL and its applications in game development, inspiring further innovation in this rapidly evolving field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。