面向城市电动网约车调度的鲁棒强化学习框架,兼顾充电约束与需求不确定性。
A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch

- 用图卷积网络建模空间相关性,结合混合整数规划保障充电与电网约束
- 在纽约出租车数据上实现122万美元净收益,显著高于其他基线方法
- 适合关注交通能源协同优化与工业部署可靠性的研究者和工程师
本文研究城市尺度电动网约车调度问题,需在不确定且具有空间相关性的需求与行驶时间下,满足充电桩与馈线容量限制。将问题建模为六边形网格半马尔可夫决策过程(semi-MDP),包含离散服务、再定位、充电动作及连续充电功率,动作持续时间可变。为保证训练与部署时的物理可行性,策略通过掩码温度退火的高层意图生成器学习,并在每步决策中通过限时滚动混合整数线性规划(MILP)严格执行电量、端口与馈线约束。为缓解分布偏移,采用基于Wasserstein-1模糊集与图对齐马哈拉诺比斯度量的软演员-评论家(SAC)优化。鲁棒更新使用Kantorovich-Rubinstein对偶、投影次梯度内循环与原对偶风险预算更新。架构包含双层图卷积网络编码器、双评论家与驱动对抗者的值网络。基于纽约市出租车数据构建的大规模仿真器实验表明,PD-RSAC在净收益达122万美元,优于0.58万至0.70万美元的强启发式、单智能体与多智能体强化学习基线(包括贪心法、SAC、MAPPO、MADDPG),且零馈线越限。
原文摘要 · Abstract (English)
We study city-scale control of electric-vehicle (EV) ride-hailing fleets where dispatch, repositioning, and charging decisions must respect charger and feeder limits under uncertain, spatially correlated demand and travel times. We formulate the problem as a hex-grid semi-Markov decision process (semi-MDP) with mixed actions -- discrete actions for serving, repositioning, and charging, together with continuous charging power -- and variable action durations. To guarantee physical feasibility during both training and deployment, the policy learns over high-level intentions produced by a masked, temperature-annealed actor. These intentions are projected at every decision step through a time-limited rolling mixed-integer linear program (MILP) that strictly enforces state-of-charge, port, and feeder constraints. To mitigate distributional shifts, we optimize a Soft Actor-Critic (SAC) agent against a Wasserstein-1 ambiguity set with a graph-aligned Mahalanobis ground metric that captures spatial correlations. The robust backup uses the Kantorovich-Rubinstein dual, a projected subgradient inner loop, and a primal-dual risk-budget update. Our architecture combines a two-layer Graph Convolutional Network (GCN) encoder, twin critics, and a value network that drives the adversary. Experiments on a large-scale EV fleet simulator built from NYC taxi data show that PD-RSAC achieves the highest net profit, reaching \$1.22M, compared with \$0.58M-\$0.70M for strong heuristic, single-agent RL, and multi-agent RL baselines, including Greedy, SAC, MAPPO, and MADDPG, while maintaining zero feeder-limit violations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。