arXiv:2409.14491cs.MAcs.RO2024-09ICRA被引 12

用简单模仿学习+碰撞修复,快速超越复杂强化学习的多智能体路径规划方法

Work Smarter Not Harder: Simple Imitation Learning with CS-PIBT Outperforms Large Scale Imitation Learning for MAPF

  • 用70万条高质量路径数据模仿学习,结合1步碰撞修复机制CS-PIBT
  • 训练仅需几分钟,解决数百个智能体的路径规划,性能远超现有方法
  • 强调未来模型应优先处理长期规划,1步碰撞可高效修复

多智能体路径规划(MAPF)旨在共享工作空间中为一组智能体寻找高效且无冲突的路径。该领域长期聚焦于高性能启发式搜索算法。近期一些研究尝试使用机器学习技术求解MAPF,通常依赖复杂架构、强化学习及特定设置,但未使用大量高质量监督数据。本文最初目标是验证:通过大规模模仿学习高质量启发式搜索方法,可达到顶尖的机器学习MAPF性能。然而发现,仅用简单的大规模模仿学习(70万样本,每例含数百个智能体)无法取得理想效果。关键突破在于引入先前的后处理方法CS-PIBT,用于修复1步碰撞。结合此机制,仅用简单模型可在数分钟内训练完成,显著优于现有所有机器学习策略。这对未来具有局部通信能力的机器学习MAPF策略具有重要启示:(1)应始终采用智能的1步碰撞防护机制(如CS-PIBT);(2)必须将带贪婪动作的碰撞防护(如PIBT)作为基线;(3)激励新模型专注于更长视野或复杂规划,因1步碰撞可被高效处理。

原文摘要 · Abstract (English)

Multi-Agent Path Finding (MAPF) is the problem of effectively finding efficient collision-free paths for a group of agents in a shared workspace. The MAPF community has largely focused on developing high-performance heuristic search methods. Recently, several works have applied various machine learning (ML) techniques to solve MAPF, usually involving sophisticated architectures, reinforcement learning techniques, and set-ups, but none using large amounts of high-quality supervised data. Our initial objective in this work was to show how simple large scale imitation learning of high-quality heuristic search methods can lead to state-of-the-art ML MAPF performance. However, we find that, at least with our model architecture, simple large scale (700k examples with hundreds of agents per example) imitation learning does \textit{not} produce impressive results. Instead, we find that by using prior work that post-processes MAPF model predictions to resolve 1-step collisions (CS-PIBT), we can train a simple ML MAPF model in minutes that dramatically outperforms existing ML MAPF policies. This has serious implications for all future ML MAPF policies (with local communication) which currently struggle to scale. In particular, this finding implies that future learnt policies should (1) always use smart 1-step collision shields (e.g. CS-PIBT), (2) always include the collision shield with greedy actions as a baseline (e.g. PIBT) and (3) motivates future models to focus on longer horizon / more complex planning as 1-step collisions can be efficiently resolved.

多智能体路径规划模仿学习碰撞修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。