用强化学习让滚球机器人在不平地形自主导航，无需复杂假设。
Reinforcement Learning for Ballbot Navigation in Uneven Terrain
- 通过视觉观测和奖励函数设计，让强化学习策略学会导航
- 仅需4-5小时训练数据，即可在随机不平地形中稳定移动
- 开源仿真环境，适合研究机器人自适应控制的学者
滚球机器人导航通常依赖控制理论方法，而将强化学习（RL）应用于该问题的研究仍较少，且多局限于特定子任务（如平衡恢复）。与基于控制理论的方法不同，强化学习无需对环境动力学做简化假设（如忽略球与地面间的打滑）。此外，RL代理可轻松利用深度图等外部观测信息，无需从原理上显式建模，从而提升适应性。尽管具备这些优势，关于基于强化学习的滚球机器人控制与导航能力、数据效率及局限性的研究仍十分有限，且缺乏开源、适配强化学习的仿真平台。本文提出一个基于MuJoCo的开源滚球机器人仿真环境，并表明：通过适当的外感受观测条件和奖励函数设计，经典无模型强化学习方法所学习的策略能够有效穿越随机生成的不平地形，仅需约四到五小时的训练数据（系统运行频率为500Hz）。相关代码已公开。
原文摘要 · Abstract (English)
Ballbot (i.e. Ball balancing robot) navigation usually relies on methods rooted in control theory (CT), and works that apply Reinforcement learning (RL) to the problem remain rare while generally being limited to specific subtasks (e.g. balance recovery). Unlike CT based methods, RL does not require (simplifying) assumptions about environment dynamics (e.g. the absence of slippage between the ball and the floor). In addition to this increased accuracy in modeling, RL agents can easily be conditioned on additional observations such as depth-maps without the need for explicit formulations from first principles, leading to increased adaptivity. Despite those advantages, there has been little to no investigation into the capabilities, data-efficiency and limitations of RL based methods for ballbot control and navigation. Furthermore, there is a notable absence of an open-source, RL-friendly simulator for this task. In this paper, we present an open-source ballbot simulation based on MuJoCo, and show that with appropriate conditioning on exteroceptive observations as well as reward shaping, policies learned by classical model-free RL methods are capable of effectively navigating through randomly generated uneven terrain, using a reasonable amount of data (four to five hours on a system operating at 500hz). Our code is made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。