构建了兼容现代强化学习框架的机器人足球仿真环境
R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning
- 通过共享内存与周期同步，打通传统足球仿真与Python强化学习流程
- 支持11人对11人完整比赛与特定场景训练，内置预期控球价值奖励机制
- 适合多智能体强化学习研究者快速搭建机器人足球实验平台
机器人足球是多智能体强化学习的挑战性测试平台，融合部分可观测性、合作与对抗交互、稀疏奖励和长时程战术行为。RoboCup 2D 足球仿真（RCSS2D）提供成熟平台，但其面向竞赛的客户端-服务器架构难以直接适配现代Python多智能体强化学习（MARL）工作流。我们提出R2D-RL，通过共享内存通信与周期级同步，将RCSS2D与基于HELIOS的球员客户端连接至Python MARL接口。R2D-RL支持全场地与场景化训练，可配置对手，具备离散与混合参数化动作空间、动作掩码、基于预期控球价值（EPV）的奖励塑造及并行执行能力。我们提供了前门场景与11对11全场地基准任务，并给出基线结果。
原文摘要 · Abstract (English)
Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform, but its competition-oriented server-client architecture is difficult to use directly with modern Python-based MARL workflows. We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. R2D-RL supports full-field and scenario-based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value (EPV)-based reward shaping, and parallel execution. We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。