四足机器人团队自主踢球,实现协作与对抗。
Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams
- 分层强化学习:底层练动作,高层定策略
- 实战中实现传球、拦截等复杂配合行为
- 纯本地感知,适合真实场景足球比赛
实现腿部机器人之间的协调协作需要精细的运动控制和长时程的战略决策。机器人足球是这一挑战的理想测试平台,融合了动态性、竞争性和多智能体交互。本文提出一种分层多智能体强化学习(MARL)框架,使四足机器人足球实现完全自主与去中心化。首先训练一系列高动态底层技能,如行走、带球和踢球;在此基础上,使用基于虚构自我博弈(FSP)的多智能体近端策略优化(MAPPO)训练高层战略规划策略。该方法使智能体能适应多样对手策略,并涌现出协同传球、拦截和动态角色分配等复杂团队行为。通过大量消融实验验证,该方法在合作与竞争性多智能体足球任务中表现显著优于基线。我们将学习到的策略部署于真实四足机器人,仅依赖机载本体感知与去中心化定位,在室内外足球场成功实现机器人间及机器人与人类的自主足球对战。
原文摘要 · Abstract (English)
Achieving coordinated teamwork among legged robots requires both fine-grained locomotion control and long-horizon strategic decision-making. Robot soccer offers a compelling testbed for this challenge, combining dynamic, competitive, and multi-agent interactions. In this work, we present a hierarchical multi-agent reinforcement learning (MARL) framework that enables fully autonomous and decentralized quadruped robot soccer. First, a set of highly dynamic low-level skills is trained for legged locomotion and ball manipulation, such as walking, dribbling, and kicking. On top of these, a high-level strategic planning policy is trained with Multi-Agent Proximal Policy Optimization (MAPPO) via Fictitious Self-Play (FSP). This learning framework allows agents to adapt to diverse opponent strategies and gives rise to sophisticated team behaviors, including coordinated passing, interception, and dynamic role allocation. With an extensive ablation study, the proposed learning method shows significant advantages in the cooperative and competitive multi-agent soccer game. We deploy the learned policies to real quadruped robots relying solely on onboard proprioception and decentralized localization, with the resulting system supporting autonomous robot-robot and robot-human soccer matches on indoor and outdoor soccer courts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。