用强化学习动态分配机器人路径优先级,提升仓库长期协同导航效率。
Learning-guided Prioritized Planning for Lifelong Multi-Agent Path Finding in Warehouse Automation
- 用强化学习优化优先级策略,结合滚动规划实现高效多机路径调度。
- 在真实仓库仿真中达成最高整体吞吐量,优于传统方法且适应多种场景。
- 可解释性强,能主动疏导拥堵区域,适合大规模智能仓储系统应用。
长期多智能体路径规划(Lifelong MAPF)对现代仓库自动化至关重要,需多个机器人持续规划无冲突路径以优化系统整体吞吐量。然而,仓库环境复杂性与长期动态性常导致传统基于搜索的求解器需频繁调整,代价高昂。尽管已有机器学习方法尝试,其优势仍不明确。本文提出首个融合强化学习与基于搜索规划的框架——RL-RH-PP,以经典优先级规划(PP)为骨干,利用强化学习动态分配优先级。通过将动态优先级分配建模为部分可观测马尔可夫决策过程(POMDP),RL-RH-PP利用长期规划的序列决策特性,并将智能体间的时空交互复杂性交由强化学习处理。采用注意力机制神经网络在线自回归解码优先级顺序,支持高效的单智能体序列规划。在真实仓库仿真中,RL-RH-PP在所有基线中达到最高总吞吐量,且在不同机器人密度、规划窗口和仓库布局下均表现良好。可解释性分析表明,该方法能主动优先处理拥堵区域的智能体,并战略性引导其避让,缓解交通压力,提升整体效率。结果凸显学习引导方法在增强传统启发式策略方面的潜力。
原文摘要 · Abstract (English)
Lifelong Multi-Agent Path Finding (MAPF) is critical for modern warehouse automation, which requires multiple robots to continuously navigate conflict-free paths to optimize the overall system throughput. However, the complexity of warehouse environments and the long-term dynamics of lifelong MAPF often demand costly adaptations to classical search-based solvers. While machine learning methods have been explored, their superiority over search-based methods remains inconclusive. In this paper, we introduce Reinforcement Learning (RL) guided Rolling Horizon Prioritized Planning (RL-RH-PP), the first framework integrating RL with search-based planning for lifelong MAPF. Specifically, we leverage classical Prioritized Planning (PP) as a backbone for its simplicity and flexibility in integrating with a learning-based priority assignment policy. By formulating dynamic priority assignment as a Partially Observable Markov Decision Process (POMDP), RL-RH-PP exploits the sequential decision-making nature of lifelong planning while delegating complex spatial-temporal interactions among agents to reinforcement learning. An attention-based neural network autoregressively decodes priority orders on-the-fly, enabling efficient sequential single-agent planning by the PP planner. Evaluations in realistic warehouse simulations show that RL-RH-PP achieves the highest total throughput among baselines and generalizes effectively across agent densities, planning horizons, and warehouse layouts. Our interpretive analyses reveal that RL-RH-PP proactively prioritizes congested agents and strategically redirects agents from congestion, easing traffic flow and boosting throughput. These findings highlight the potential of learning-guided approaches to augment traditional heuristics in modern warehouse automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。