用预训练模型加速交通信号控制,新场景下学习更快更准。
Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reuse
- 先在固定场景预训练策略和环境模型,提升泛化能力。
- 新场景下通过相似度匹配选择最优预训练模型,适应时间大幅缩短。
- 适合需快速部署到不同道路网络的智能交通系统开发者。
多智能体强化学习(MARL)在交通信号控制(TSC)中展现巨大潜力,但现有方法常因训练时使用固定交通模式和道路条件,导致泛化能力不足,难以适应新场景,带来高重训成本与复杂部署。为此,我们提出PLight与PRLight两种算法。PLight采用基于模型的强化学习,利用预设源域交通场景预先训练控制策略与环境模型,环境模型可预测状态转移,便于比较环境特征。PRLight进一步通过计算源域与目标域的相似性,自适应选择最优的PLight代理,加速目标域的学习过程。我们在两种迁移设置下评估:(1)同一道路网络内不同交通场景的适应性;(2)跨不同道路网络的泛化能力。结果表明,与从零开始学习相比,PRLight显著缩短了适应时间,在利用可用场景与目标场景相似性的情况下实现了最优性能。
原文摘要 · Abstract (English)
Multi-agent reinforcement learning (MARL) has shown significant potential in traffic signal control (TSC). However, current MARL-based methods often suffer from insufficient generalization due to the fixed traffic patterns and road network conditions used during training. This limitation results in poor adaptability to new traffic scenarios, leading to high retraining costs and complex deployment. To address this challenge, we propose two algorithms: PLight and PRLight. PLight employs a model-based reinforcement learning approach, pretraining control policies and environment models using predefined source-domain traffic scenarios. The environment model predicts the state transitions, which facilitates the comparison of environmental features. PRLight further enhances adaptability by adaptively selecting pre-trained PLight agents based on the similarity between the source and target domains to accelerate the learning process in the target domain. We evaluated the algorithms through two transfer settings: (1) adaptability to different traffic scenarios within the same road network, and (2) generalization across different road networks. The results show that PRLight significantly reduces the adaptation time compared to learning from scratch in new TSC scenarios, achieving optimal performance using similarities between available and target scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。