用强化学习让扫地机器人在狭窄空间自动脱困,成功率更高且更少碰撞。
Embodied Escaping: End-to-End Reinforcement Learning for Robot Navigation in Narrow Environment
- 基于强化学习与动作掩码,实现端到端的狭窄区域脱困策略。
- 在真实场景中测试,脱困成功率显著优于传统路径规划和强化学习方法。
- 适合需要自主避障和复杂环境导航的移动机器人研究者使用。
自主导航是室内环境中机器人吸尘器的基础任务。由于其核心功能是清洁整个区域,机器人不可避免会遇到杂物堆积、通道狭窄导致的死区。现有规划方法常因环境约束复杂、搜索空间高维及操作难度大而无法成功脱困。为此,本文提出一种基于强化学习的具身脱困模型,采用高效的动作掩码机制处理死区逃逸。为缓解训练中的稀疏奖励问题,引入混合训练策略以提升学习效率;针对冗余无效动作,设计新型动作表示,将离散动作空间重构为具有统一转向半径的形式;同时提出动作掩码策略,快速筛选有效动作,兼顾精度与效率。实验中机器人配备激光雷达(Lidar)、惯性测量单元(IMU)及双轮编码器。在多种难度级别的真实场景中,定量与定性实验均表明,该方法能稳定完成复杂死区脱困,且在成功率与防碰撞能力上显著优于对比的路径规划与强化学习方法。
原文摘要 · Abstract (English)
Autonomous navigation is a fundamental task for robot vacuum cleaners in indoor environments. Since their core function is to clean entire areas, robots inevitably encounter dead zones in cluttered and narrow scenarios. Existing planning methods often fail to escape due to complex environmental constraints, high-dimensional search spaces, and high difficulty maneuvers. To address these challenges, this paper proposes an embodied escaping model that leverages reinforcement learning-based policy with an efficient action mask for dead zone escaping. To alleviate the issue of the sparse reward in training, we introduce a hybrid training policy that improves learning efficiency. In handling redundant and ineffective action options, we design a novel action representation to reshape the discrete action space with a uniform turning radius. Furthermore, we develop an action mask strategy to select valid action quickly, balancing precision and efficiency. In real-world experiments, our robot is equipped with a Lidar, IMU, and two-wheel encoders. Extensive quantitative and qualitative experiments across varying difficulty levels demonstrate that our robot can consistently escape from challenging dead zones. Moreover, our approach significantly outperforms compared path planning and reinforcement learning methods in terms of success rate and collision avoidance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。