四足机器人在崎岖地形上动态控球，靠分层强化学习实现稳定表现。
Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning
- 分层强化学习：高层策略动态切换低层技能
- 真实场景测试中控球成功率显著优于基线方法
- 适合研究机器人动态操作与复杂环境适应的团队
提升四足机器人在复杂地形中的动态运动-操作能力至关重要。尤其在崎岖环境中进行动态控球面临两大挑战：一是协调不同运动模式以无缝融合地形通过与球体控制；二是端到端深度强化学习中稀疏奖励导致策略难以高效收敛。为此，我们提出一种分层强化学习框架。高层策略基于本体感知数据和球的位置，自适应地在预训练的低层技能（如控球、崎岖地形导航）间切换。我们进一步提出动态技能聚焦策略优化方法，抑制非活跃技能的梯度，增强关键技能的学习。仿真与真实世界实验均表明，所提方法在崎岖地形上的动态控球表现优于基线方法，验证了其在复杂环境中的有效性。视频见 dribble-hrl.github.io。
原文摘要 · Abstract (English)
Advancing the dynamic loco-manipulation capabilities of quadruped robots in complex terrains is crucial for performing diverse tasks. Specifically, dynamic ball manipulation in rugged environments presents two key challenges. The first is coordinating distinct motion modalities to integrate terrain traversal and ball control seamlessly. The second is overcoming sparse rewards in end-to-end deep reinforcement learning, which impedes efficient policy convergence. To address these challenges, we propose a hierarchical reinforcement learning framework. A high-level policy, informed by proprioceptive data and ball position, adaptively switches between pre-trained low-level skills such as ball dribbling and rough terrain navigation. We further propose Dynamic Skill-Focused Policy Optimization to suppress gradients from inactive skills and enhance critical skill learning. Both simulation and real-world experiments validate that our methods outperform baseline approaches in dynamic ball manipulation across rugged terrains, highlighting its effectiveness in challenging environments. Videos are on our website: dribble-hrl.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。