用多智能体强化学习优化城市交通路由，减少拥堵和出行时间。
Network-Constrained Policy Optimization for Adaptive Multi-agent Vehicle Routing
- 每个路口由独立智能体根据局部与邻近路况决策，结合图注意力网络建模。
- 在曼哈顿和多伦多实测中，平均出行时间最多降低15.9%，成功率100%。
- 通过枢纽分层结构提升大规模网络扩展性，适合智能交通系统应用。
城市道路网络中的交通拥堵导致出行时间延长和排放增加，尤其在高峰时段。虽然最短路径优先（SPF）算法在静态单车场景下最优，但在动态多车环境中表现差，常因车辆路径重合加剧拥堵。本文提出自适应导航（AN），一种去中心化的多智能体强化学习框架，每个路口智能体基于本地交通状态和邻域信息（通过图注意力网络GAT建模）提供路由指引。为提升大网络可扩展性，进一步提出分层枢纽式自适应导航（HHAN），仅在关键枢纽部署智能体，车辆在枢纽间受控路由，枢纽内仍由SPF处理微路由。HHAN采用集中训练、分散执行（CTDE）与注意式Q-Mixing（A-QMIX）框架，通过注意力聚合异步车辆决策；枢纽智能体使用融合局部拥堵与预测动态的流量感知状态特征，实现主动路由。在合成网格及真实城市地图（多伦多、曼哈顿）上的实验表明，AN相比SPF和学习基线显著降低平均出行时间，保持100%路由成功率；HHAN可扩展至含数百个路口的网络，在重载交通下最高实现15.9%的性能提升。结果表明，网络约束型多智能体强化学习在可扩展、协同、抗拥堵的智能交通路由中具有潜力。
原文摘要 · Abstract (English)
Traffic congestion in urban road networks leads to longer trip times and higher emissions, especially during peak periods. While the Shortest Path First (SPF) algorithm is optimal for a single vehicle in a static network, it performs poorly in dynamic, multi-vehicle settings, often worsening congestion by routing all vehicles along identical paths. We address dynamic vehicle routing through a multi-agent reinforcement learning (MARL) framework for coordinated, network-aware fleet navigation. We first propose Adaptive Navigation (AN), a decentralized MARL model where each intersection agent provides routing guidance based on (i) local traffic and (ii) neighborhood state modeled using Graph Attention Networks (GAT). To improve scalability in large networks, we further propose Hierarchical Hub-based Adaptive Navigation (HHAN), an extension of AN that assigns agents only to key intersections (hubs). Vehicles are routed hub-to-hub under agent control, while SPF handles micro-routing within each hub region. For hub coordination, HHAN adopts centralized training with decentralized execution (CTDE) under the Attentive Q-Mixing (A-QMIX) framework, which aggregates asynchronous vehicle decisions via attention. Hub agents use flow-aware state features that combine local congestion and predictive dynamics for proactive routing. Experiments on synthetic grids and real urban maps (Toronto, Manhattan) show that AN reduces average travel time versus SPF and learning baselines, maintaining 100% routing success. HHAN scales to networks with hundreds of intersections, achieving up to 15.9% improvement under heavy traffic. These findings highlight the potential of network-constrained MARL for scalable, coordinated, and congestion-aware routing in intelligent transportation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。