用深度强化学习让游戏关卡自动生成并适配智能体求解。
Procedural Game Level Design with Deep Reinforcement Learning
- 双智能体协作：一只鸟解谜，一座岛生成道具位置。
- 鸟能高效寻花收集，岛根据障碍物和表现反馈调整布局。
- 适合对自动生成关卡感兴趣的开发者与研究者。
程序化内容生成（PCG）在游戏开发中日益流行，可减少人工投入，生成动态、可重玩且可扩展的环境。本文提出一种基于Unity 3D环境的深度强化学习（DRL）方法，用于程序化关卡设计。系统包含两个智能体：作为求解者的蜂鸟智能体，负责导航、寻花与收集；作为生成者的浮岛智能体，负责根据障碍物位置、蜂鸟初始状态及前序实验的性能反馈，生成合理的花朵布局。两者均使用Unity ML-Agents工具包中的近端策略优化（PPO）算法进行训练。通过交互，系统涌现出适应多种环境配置的鲁棒行为。结果表明，该方法不仅能生成高效可行的智能体行为，还为由机器学习驱动的自主关卡设计开辟新路径。本工作展示了DRL在虚拟环境中同时生成与求解内容的潜力，推动AI在创意游戏开发中的边界。
原文摘要 · Abstract (English)
Procedural content generation (PCG) has become an increasingly popular technique in game development, allowing developers to generate dynamic, replayable, and scalable environments with reduced manual effort. In this study, a novel method for procedural level design using Deep Reinforcement Learning (DRL) within a Unity-based 3D environment is proposed. The system comprises two agents: a hummingbird agent, acting as a solver, and a floating island agent, responsible for generating and placing collectible objects (flowers) on the terrain in a realistic and context-aware manner. The hummingbird is trained using the Proximal Policy Optimization (PPO) algorithm from the Unity ML-Agents toolkit. It learns to navigate through the terrain efficiently, locate flowers, and collect them while adapting to the ever-changing procedural layout of the island. The island agent is also trained using the Proximal Policy Optimization (PPO) algorithm. It learns to generate flower layouts based on observed obstacle positions, the hummingbird's initial state, and performance feedback from previous episodes. The interaction between these agents leads to emergent behavior and robust generalization across various environmental configurations. The results demonstrate that the approach not only produces effective and efficient agent behavior but also opens up new opportunities for autonomous game level design driven by machine learning. This work highlights the potential of DRL in enabling intelligent agents to both generate and solve content in virtual environments, pushing the boundaries of what AI can contribute to creative game development processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。