用分层强化学习让机器人在复杂环境中安全导航，避免困在局部区域。
Hierarchical Reinforcement Learning for Safe Mapless Navigation with Congestion Estimation
- 分层策略：高层生成子目标，低层实时规划动作。
- 引入拥堵评估机制，有效避开局部死胡同。
- 在办公、家庭等场景实测，物理机器人验证泛化能力。
基于强化学习的无地图导航具有巨大潜力，但在存在局部极小值的室内环境中面临挑战。本文提出一种基于分层强化学习（HRL）的安全无地图导航框架，以提升对这类区域的导航能力。高层策略生成引导导航过程的子目标；特别地，设计了一种考虑环境拥堵情况的子目标更新机制，能有效避免机器人陷入局部极小区域。底层运动规划策略通过安全强化学习训练，根据获取的子目标输出实时控制指令。为增强机器人环境感知能力，引入一种新的障碍物编码方法，用于评估障碍物对运动规划的影响。在办公室、家庭和餐厅环境中进行仿真验证，结果表明该框架在静态与动态场景中均表现优异。最后，在TurtleBot3机器人上开展物理实验，验证了其强大的泛化能力。
原文摘要 · Abstract (English)
Reinforcement learning-based mapless navigation holds significant potential. However, it faces challenges in indoor environments with local minima area. This paper introduces a safe mapless navigation framework utilizing hierarchical reinforcement learning (HRL) to enhance navigation through such areas. The high-level policy creates a sub-goal to direct the navigation process. Notably, we have developed a sub-goal update mechanism that considers environment congestion, efficiently avoiding the entrapment of the robot in local minimum areas. The low-level motion planning policy, trained through safe reinforcement learning, outputs real-time control instructions based on acquired sub-goal. Specifically, to enhance the robot's environmental perception, we introduce a new obstacle encoding method that evaluates the impact of obstacles on the robot's motion planning. To validate the performance of our HRL-based navigation framework, we conduct simulations in office, home, and restaurant environments. The findings demonstrate that our HRL-based navigation framework excels in both static and dynamic scenarios. Finally, we implement the HRL-based navigation framework on a TurtleBot3 robot for physical validation experiments, which exhibits its strong generalization capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。