将公交优先融入多智能体强化学习,显著降低公交延误。
Integrating Transit Signal Priority into Multi-Agent Reinforcement Learning based Traffic Signal Control
- 用多智能体强化学习设计公交优先策略,分中心训练与分布式执行两种方式。
- 独立式公交优先降22%公交延误,协同式达27%,对普通车辆影响极小。
- 适合城市交通信号优化、智能公交系统研究者参考。
本研究将公交信号优先(TSP)整合到基于多智能体强化学习(MARL)的交通信号控制中。第一部分在微观仿真环境中,基于价值分解网络(VDN)架构,为一对协调交叉口开发自适应信号控制,两个智能体分别控制各交叉口,集中训练后性能略优于传统协调感应控制(在车流比v/c=0.95时)。第二部分,使用已训练的信号控制智能体作为背景,开发事件驱动的公交优先智能体。第一种变体采用去中心化训练与执行(DTDE)框架训练独立的公交优先智能体;第二种变体则通过集中训练与去中心化执行(CTDE)框架和VDN架构,实现两交叉口间的协同公交优先策略。两者最终使公交延误相同,但独立智能体训练过程极不稳定。测试结果显示,独立智能体使公交延误减少22%,协同智能体减少27%,且绝大多数辅路车辆延误仅轻微上升。
原文摘要 · Abstract (English)
This study integrates Transit Signal Priority (TSP) into multi-agent reinforcement learning (MARL) based traffic signal control. The first part of the study develops adaptive signal control based on MARL for a pair of coordinated intersections in a microscopic simulation environment. The two agents, one for each intersection, are centrally trained using a value decomposition network (VDN) architecture. The trained agents show slightly better performance compared to coordinated actuated signal control based on overall intersection delay at v/c of 0.95. In the second part of the study the trained signal control agents are used as background signal controllers while developing event-based TSP agents. In one variation, independent TSP agents are formulated and trained under a decentralized training and decentralized execution (DTDE) framework to implement TSP at each intersection. In the second variation, the two TSP agents are centrally trained under a centralized training and decentralized execution (CTDE) framework and VDN architecture to select and implement coordinated TSP strategies across the two intersections. In both cases the agents converge to the same bus delay value, but independent agents show high instability throughout the training process. For the test runs, the two independent agents reduce bus delay across the two intersections by 22% compared to the no TSP case while the coordinated TSP agents achieve 27% delay reduction. In both cases, there is only a slight increase in delay for a majority of the side street movements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。