arXiv:2605.02528cs.ROcs.LG2026-05

用程序化地图提升强化学习导航泛化能力,实测效果显著优于传统方法。

Beyond Specialization: Robust Reinforcement Learning Navigation via Procedural Map Generators

论文配图:Beyond Specialization: Robust Reinforcement Learning Navigation via Procedural Map Generators
图 1 · 摘自论文原文
  • 引入四种程序化地图生成器,在2D仿真中测试策略泛化性。
  • 联合训练策略在跨生成器任务上达到91.5%成功率,远超单一布局训练。
  • 基于A*的子目标输入是鲁棒性的关键,适合需要高适应性的导航场景。

深度强化学习(DRL)导航策略常因训练环境结构单一而过拟合。尽管程序化地图生成可提供大规模多样性,但此前缺乏对不同生成器类型影响的系统评估。本文将四种生成器(稀疏、迷宫、图结构、波函数坍缩)集成到专注于激光雷达导航训练效率的MuRoSim 2D仿真器中,对五种导航策略在每类生成器下1000张种子地图上进行交叉评估,覆盖三个训练随机种子。结果表明跨生成器迁移存在强烈非对称性:仅在稀疏布局训练的专家策略在迷宫地图上成功率降至3.3%,而联合生成器训练的策略平均成功率高达91.5 ± 1.1%。进一步发现,以A*路径规划器输出作为子目标输入,使成功率从90.2 ± 1.4%的前馈基线提升至98.9 ± 0.4%,优于仅使用GRU递归的方案。所提DRL策略在高速(2.0 m/s)下表现超越经典Carrot+A*控制器(后者在1.0 m/s时成功,2.0 m/s时骤降至24.9%),体现学习速度适应的优势。真实机器人实验在RoboMaster平台验证了模拟到现实的迁移可行性,但迷宫布局暴露仍存失效模式,递归结构有助于缓解。

原文摘要 · Abstract (English)

Deep reinforcement learning (DRL) navigation policies often overfit to the structure of their training environments, as environmental diversity is typically constrained by the manual effort required to design diverse scenarios. While procedural map generation offers scalable diversity, no prior work systematically compares how different generator types affect policy generalization. We integrate four generators (sparse, maze, graph, and Wave Function Collapse) with guaranteed navigability into MuRoSim, a 2D simulator focusing on training efficiency for LiDAR-based navigation. We cross-evaluate five navigation policies on 1000 seeded maps per generator across three training seeds. Results show a strongly asymmetric cross-generator transfer: a specialist trained on sparse layouts falls to 3.3% success on mazes, whereas a policy trained on the combined generator set achieves 91.5 +/- 1.1% mean success. We further demonstrate that A* path-planner subgoal inputs are the dominant factor for robustness, raising success from the 90.2 +/- 1.4% feedforward baseline to 98.9 +/- 0.4% and outperforming GRU recurrence, which only improves the reactive baseline. The DRL policies outperform a classical Carrot+A* controller, which matches their success only at low speeds (1.0 m/s) but collapses to 24.9% at 2.0 m/s. This highlights learned speed adaptation as the decisive advantage of the learned approach. Real-world experiments on a RoboMaster confirm sim-to-real transfer in a cluttered arena, while a maze-like layout exposes remaining failure modes that recurrence helps mitigate.

强化学习导航程序化生成仿真

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。