提出高效可定制的多机器人导航环境，支持快速训练与真实场景部署。
Efficient Environment Design for Multi-Robot Navigation via Continuous Control
- 基于马尔可夫决策过程构建连续控制导航环境，任务为最短路径访问兴趣区域。
- 在3D农田仿真中验证，多种强化学习方法（如PPO、TQC）均实现稳定收敛。
- 适合关注真实世界多机器人系统设计的研究者，尤其重视训练效率与鲁棒性。
在具有不确定性的连续状态与动作空间中，多机器人导航与路径规划仍是开放挑战。深度强化学习（RL）虽是主流方法，但因样本效率低、训练周期长，实际应用受限。现有基于RL的多机器人导航研究缺乏环境设计的形式化保障。本文提出一种高效且高度可定制的连续控制多机器人导航环境，要求机器人沿最短路径访问一组兴趣区域（ROIs）。任务被形式化为马尔可夫决策过程（MDP），并建模为优化问题，关联到寻找最优策略。我们设计了多种环境变体，采用梯度与非梯度方法（包括A2C、PPO、TRPO、TQC、CrossQ和ARS）评估性能。为验证现实适用性，我们在使用CoppeliaSim机器人模拟器的3D农业环境中部署该环境，通过推理测试学习模型的鲁棒性。本工作旨在指导研究人员如何构建适用于真实系统的基于MDP的环境，并以有限资源和合理时间，利用现有先进强化学习方法求解。
原文摘要 · Abstract (English)
Multi-robot navigation and path planning in continuous state and action spaces with uncertain environments remains an open challenge. Deep Reinforcement Learning (RL) is one of the most popular paradigms for solving this task, but its real-world application has been limited due to sample inefficiency and long training periods. Moreover, the existing works using RL for multi-robot navigation lack formal guarantees while designing the environment. In this paper, we introduce an efficient and highly customizable environment for continuous-control multi-robot navigation, where the robots must visit a set of regions of interest (ROIs) by following the shortest paths. The task is formally modeled as a Markov Decision Process (MDP). We describe the multi-robot navigation task as an optimization problem and relate it to finding an optimal policy for the MDP. We crafted several variations of the environment and measured the performance using both gradient and non-gradient based RL methods: A2C, PPO, TRPO, TQC, CrossQ and ARS. To show real-world applicability, we deployed our environment to a 3-D agricultural field with uncertainties using the CoppeliaSim robot simulator and measured the robustness by running inference on the learned models. We believe our work will guide the researchers on how to develop MDP-based environments that are applicable to real-world systems and solve them using the existing state-of-the-art RL methods with limited resources and within reasonable time periods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。