MacLight用卷积学习提升信号灯控制,训练更快更稳。
MacLight: Multi-scene Aggregation Convolutional Learning for Traffic Signal Control
- 用变分自编码器压缩全局信息,结合PPO算法实现高效决策
- 在静态与动态交通下均实现最高训练效率和最优收敛性
- 适合需要实时响应的智能交通系统部署
强化学习在大规模路网信号灯控制中展现出潜力。现有最先进方法将路网建模为拓扑图结构,引入图注意力机制并融合局部与全局嵌入以优化策略,但图结构难以并行化,导致显著时间开销。此外,当前研究均未在动态交通系统中部署实验,脱离实际场景。为此,我们提出多场景聚合卷积学习(MacLight),实现更快训练速度与更稳定性能。该方法包含两部分:一是利用变分自编码器紧凑压缩并提取全局表征;二是采用近端策略优化(PPO)作为主干,使价值评估同时考虑局部特征与全局嵌入表示,显著降低时间开销并保证策略更新稳定性。我们在多种交通场景下验证了方法在静态与动态交通系统中的表现。实验表明,相比通用及领域最先进方法,本方案在稳定性、收敛水平和时间效率上均表现最优。
原文摘要 · Abstract (English)
Reinforcement learning methods have proposed promising traffic signal control policy that can be trained on large road networks. Current SOTA methods model road networks as topological graph structures, incorporate graph attention into deep Q-learning, and merge local and global embeddings to improve policy. However, graph-based methods are difficult to parallelize, resulting in huge time overhead. Moreover, none of the current peer studies have deployed dynamic traffic systems for experiments, which is far from the actual situation. In this context, we propose Multi-Scene Aggregation Convolutional Learning for traffic signal control (MacLight), which offers faster training speeds and more stable performance. Our approach consists of two main components. The first is the global representation, where we utilize variational autoencoders to compactly compress and extract the global representation. The second component employs the proximal policy optimization algorithm as the backbone, allowing value evaluation to consider both local features and global embedding representations. This backbone model significantly reduces time overhead and ensures stability in policy updates. We validated our method across multiple traffic scenarios under both static and dynamic traffic systems. Experimental results demonstrate that, compared to general and domian SOTA methods, our approach achieves superior stability, optimized convergence levels and the highest time efficiency. The code is under https://github.com/Aegis1863/MacLight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。