用深度强化学习实时协同优化机器人调度与订单分配,提升仓储效率。
SOAR: Real-Time Joint Optimization of Order Allocation and Robot Scheduling in Robotic Mobile Fulfillment Systems

- 将订单分配与机器人调度统一为事件驱动的强化学习决策过程。
- 实测在真实场景下使任务完成时间平均减少15.4%,延迟低于100ms。
- 适合需要高实时性与全局优化的智能仓储系统研发与部署。
机器人移动分拣系统(RMFS)依赖移动机器人实现库存自动转运,通过协调订单分配与机器人调度以提升仓储效率。然而,由于严格的实时性要求以及多阶段决策间的强耦合,优化该系统极具挑战。现有方法或分解问题为独立子任务以保证响应速度,牺牲全局最优;或依赖计算成本高的全局优化模型,难以适应动态工业环境。为此,我们提出SOAR——一种面向实时联合优化的统一深度强化学习框架。SOAR通过软化订单分配作为观测,将订单分配与机器人调度融合为统一过程,并建模为事件驱动的马尔可夫决策过程,使智能体能对异步系统事件同步响应。技术上,采用异构图变压器编码仓库状态并融入阶段领域知识;同时引入奖励塑形策略应对长时序任务中的稀疏反馈。在合成数据及与Geekplus合作的真实工业数据集上的大量实验表明,SOAR在全局完工时间上减少7.5%,平均订单完成时间降低15.4%,延迟低于100ms。此外,仿真到真实环境的部署验证了其实际可行性与生产环境中显著的性能提升。代码已公开于https://github.com/200815147/SOAR。
原文摘要 · Abstract (English)
Robotic Mobile Fulfillment Systems (RMFS) rely on mobile robots for automated inventory transportation, coordinating order allocation and robot scheduling to enhance warehousing efficiency. However, optimizing RMFS is challenging due to strict real-time constraints and the strong coupling of multi-phase decisions. Existing methods either decompose the problem into isolated sub-tasks to guarantee responsiveness at the cost of global optimality, or rely on computationally expensive global optimization models that are unsuitable for dynamic industrial environments. To bridge this gap, we propose SOAR, a unified Deep Reinforcement Learning framework for real-time joint optimization. SOAR transforms order allocation and robot scheduling into a unified process by utilizing soft order allocations as observations. We formulate this as an Event-Driven Markov Decision Process, enabling the agent to perform simultaneous scheduling in response to asynchronous system events. Technically, we employ a Heterogeneous Graph Transformer to encode the warehouse state and integrate phased domain knowledge. Additionally, we incorporate a reward shaping strategy to address sparse feedback in long-horizon tasks. Extensive experiments on synthetic and real-world industrial datasets, in collaboration with Geekplus, demonstrate that SOAR reduces global makespan by 7.5\% and average order completion time by 15.4\% with sub-100ms latency. Furthermore, sim-to-real deployment confirms its practical viability and significant performance gains in production environments. The code is available at https://github.com/200815147/SOAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。