分层强化学习让机器人踢球更准，真实场景成功率80%。
HierKick: Hierarchical Reinforcement Learning for Vision-Guided Soccer Robot Control
- 高/低频双层策略：5Hz高层决策+50Hz底层控制，兼顾规划与执行
- 四步连贯动作：接近、对齐、带球、射门，真实世界成功率达80%
- 模块化设计可复用技能，适合复杂动态环境的机器人控制
足球机器人控制涉及多时标决策，需平衡长期战术规划与短期运动执行。传统端到端强化学习在复杂动态环境中面临挑战。本文提出基于双频率分层强化学习的视觉引导控制框架HierKick。该框架采用分层控制结构：5 Hz高层策略结合YOLOv8实时检测，通过教练模型选择任务；50 Hz预训练底层控制器实现精确关节控制。通过此架构，系统可完成接近、对齐、带球和踢球四个步骤。实验结果表明，该框架在IsaacGym中成功率达95.2%，在Mujoco中为89.8%，真实世界环境下达80%。HierKick为复杂环境中的机器人控制提供了有效分层范式，其模块化设计与技能复用特性，为智能机器人控制开辟新路径。
原文摘要 · Abstract (English)
Controlling soccer robots involves multi-time-scale decision-making, which requires balancing long-term tactical planning and short-term motion execution. Traditional end-to-end reinforcement learning (RL) methods face challenges in complex dynamic environments. This paper proposes HierKick, a vision-guided soccer robot control framework based on dual-frequency hierarchical RL. The framework adopts a hierarchical control architecture featuring a 5 Hz high-level policy that integrates YOLOv8 for real-time detection and selects tasks via a coach model, and a pre-trained 50 Hz low-level controller for precise joint control. Through this architecture, the framework achieves the four steps of approaching, aligning, dribbling, and kicking. Experimental results show that the success rates of this framework are 95.2\% in IsaacGym, 89.8\% in Mujoco, and 80\% in the real world. HierKick provides an effective hierarchical paradigm for robot control in complex environments, extendable to multi-time-scale tasks, with its modular design and skill reuse offering a new path for intelligent robot control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。