arXiv:2503.23626cs.MAcs.LG2025-03被引 7

用约束强化学习优化红绿灯,让城市交通更高效公平。

A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control

  • 将红绿灯控制建模为带约束的多智能体强化学习问题。
  • 在三个真实数据集上,性能比现有算法提升10%以上。
  • 适合城市交通规划者部署可扩展的智能信号系统。

现代城市交通拥堵受传统定时信号系统的局限,难以适应动态车流。自适应信号控制(ATSC)通过实时调整信号时序来应对,但现有方法难以在真实约束下应用,如兼顾效率、减少碰撞和各路口公平性。本文将ATSC问题视为约束多智能体强化学习(MARL)问题,提出新算法MAPPO-LCE,结合拉格朗日乘子与成本估计器,稳定平衡奖励与约束。引入三种现实约束:GreenTime、GreenSkip、PhaseSkip,惩罚不符合实际场景的策略。在三个真实数据集上的实验表明,MAPPO-LCE在所有环境与约束下均优于三种基线算法(相比MAPPO提升12.60%,IPPO提升10.29%,QTRAN提升13.10%)。结果证明,约束型MARL是交通规划者部署可扩展高效信号控制的有效工具。代码已开源。

原文摘要 · Abstract (English)

Traffic congestion in modern cities is exacerbated by the limitations of traditional fixed-time traffic signal systems, which fail to adapt to dynamic traffic patterns. Adaptive Traffic Signal Control (ATSC) algorithms have emerged as a solution by dynamically adjusting signal timing based on real-time traffic conditions. However, the main limitation of such methods is that they are not transferable to environments under real-world constraints, such as balancing efficiency, minimizing collisions, and ensuring fairness across intersections. In this paper, we view the ATSC problem as a constrained multi-agent reinforcement learning (MARL) problem and propose a novel algorithm named Multi-Agent Proximal Policy Optimization with Lagrange Cost Estimator (MAPPO-LCE) to produce effective traffic signal control policies. Our approach integrates the Lagrange multipliers method to balance rewards and constraints, with a cost estimator for stable adjustment. We also introduce three constraints on the traffic network: GreenTime, GreenSkip, and PhaseSkip, which penalize traffic policies that do not conform to real-world scenarios. Our experimental results on three real-world datasets demonstrate that MAPPO-LCE outperforms three baseline MARL algorithms by across all environments and traffic constraints (improving on MAPPO by 12.60%, IPPO by 10.29%, and QTRAN by 13.10%). Our results show that constrained MARL is a valuable tool for traffic planners to deploy scalable and efficient ATSC methods in real-world traffic networks. We provide code at https://github.com/Asatheesh6561/MAPPO-LCE.

交通信号强化学习多智能体约束优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。