解决电动自动驾驶出租车调度与充电分配难题,提升系统长期收益。
Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation
- 将车辆调度分解为原子动作,大幅降低决策复杂度。
- 在纽约真实数据上,平均收益接近理论最优解的95%以上。
- 适合研究智能交通、大规模强化学习调度的开发者参考。
Waymo等公司已在多个美国城市部署自动驾驶出租车服务。这些出租车为电动车,其运营需在随机环境中联合优化乘客匹配、车辆调度与充电计划。本文将网约车系统建模为无限时域的离散时间平均奖励马尔可夫决策过程。随着车队规模扩大,调度难度急剧上升,因状态空间与动作空间随车辆数呈指数增长。为此,提出可扩展的深度强化学习算法——原子近端策略优化(Atomic-PPO),通过原子动作分解减少动作空间。基于纽约市真实网约车行程数据进行评估,以调度策略的长期平均奖励相对于流体模型上界的表现作为衡量标准。实验表明,Atomic-PPO显著优于基准方法。此外,通过大量数值实验分析了充电桩配置效率,并评估了车辆续航与充电速度对系统性能的影响。
原文摘要 · Abstract (English)
Pioneering companies such as Waymo have deployed robo-taxi services in several U.S. cities. These robo-taxis are electric vehicles, and their operations require the joint optimization of ride matching, vehicle repositioning, and charging scheduling in a stochastic environment. We model the operations of the ride-hailing system with robo-taxis as a discrete-time, average-reward Markov Decision Process with an infinite horizon. As the fleet size grows, dispatching becomes challenging, as both the system state space and the fleet dispatching action space grow exponentially with the number of vehicles. To address this, we introduce a scalable deep reinforcement learning algorithm, called Atomic Proximal Policy Optimization (Atomic-PPO), that reduces the action space using atomic action decomposition. We evaluate our algorithm using real-world NYC for-hire vehicle trip records and measure its performance by the long-run average reward achieved by the dispatching policy, relative to a fluid-based upper bound. Our experiments demonstrate the superior performance of Atomic-PPO compared to benchmark methods. Furthermore, we conduct extensive numerical experiments to analyze the efficient allocation of charging facilities and assess the impact of vehicle range and charger speed on system performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。