用规则增强强化学习，让游戏AI在大型地图中像真人一样作战。
Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning
- 结合导航网格与射击规则，提升AI导航和战斗能力。
- 通过奖励机制让AI行为更接近人类玩家。
- 适合研究复杂场景下可交互AI的开发者参考。
在3D第一人称射击（FPS）游戏中，大型复杂地图中的游戏AI部署仍面临挑战。本文针对腾讯游戏开发的竞技类多人在线FPS游戏Arena Breakout，提出名为私人军事公司代理（PMCA）的新一代游戏AI系统。该系统可在大地图中与玩家互动并利用地形优势进行战斗。为应对现代FPS游戏中的导航与战斗难题，我们提出一种融合导航网格（Navmesh）与射击规则的深度强化学习方法（NSRL）。Navmesh提升全局导航能力，射击行为则由规则控制以确保可控性。NSRL使用深度强化学习模型预测何时启用导航网格，从而生成多样化行为。此外，采用定制化奖励函数引导AI学习类人行为，使其表现更贴近真实玩家。
原文摘要 · Abstract (English)
In the realm of competitive gaming, 3D first-person shooter (FPS) games have gained immense popularity, prompting the development of game AI systems to enhance gameplay. However, deploying game AI in practical scenarios still poses challenges, particularly in large-scale and complex FPS games. In this paper, we focus on the practical deployment of game AI in the online multiplayer competitive 3D FPS game called Arena Breakout, developed by Tencent Games. We propose a novel gaming AI system named Private Military Company Agent (PMCA), which is interactable within a large game map and engages in combat with players while utilizing tactical advantages provided by the surrounding terrain. To address the challenges of navigation and combat in modern 3D FPS games, we introduce a method that combines navigation mesh (Navmesh) and shooting-rule with deep reinforcement learning (NSRL). The integration of Navmesh enhances the agent's global navigation capabilities while shooting behavior is controlled using rule-based methods to ensure controllability. NSRL employs a DRL model to predict when to enable the navigation mesh, resulting in a diverse range of behaviors for the game AI. Customized rewards for human-like behaviors are also employed to align PMCA's behavior with that of human players.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。