用多智能体强化学习实现大规模路网中卡车编队自主协同,省油19%且延迟仅9.6分钟。
Multi-Agent Deep Reinforcement Learning for Distributed and Autonomous Platoon Coordination via Speed-regulation over Large-scale Transportation Networks
- 设计TA-QMIX框架,通过注意力机制增强编队协作信息表达
- 在5000辆卡车测试中,平均节油19.17%,每车延迟仅9.57分钟
- 支持分布式执行,每车决策仅需0.001秒,适合真实路网部署
卡车编队技术可显著降低油耗、提升交通效率与安全性。本文研究大规模交通网络中的编队协同问题,综合调节枢纽处的出发时间与车速,将问题建模为受网络与信息约束的动态随机整数规划。为此,将其转化为部分可观测马尔可夫决策过程,并提出多智能体深度强化学习框架TA-QMIX,利用注意力机制增强对油耗收益与延误时间的表征,训练过程中显式传递协作信息以促进车辆合作。采用集中训练、分布式执行策略,使车辆仅依赖局部信息即可在线决策,具备大规模网络自主执行能力。在长江三角洲区域交通网络上进行对比与消融实验,5000辆卡车重复测试中,平均节油19.17%,单车平均延误9.57分钟,单次决策耗时仅0.001秒。
原文摘要 · Abstract (English)
Truck platooning technology enables a group of trucks to travel closely together, with which the platoon can save fuel, improve traffic flow efficiency, and improve safety. In this paper, we consider the platoon coordination problem in a large-scale transportation network, to promote cooperation among trucks and optimize the overall efficiency. Involving the regulation of both speed and departure times at hubs, we formulate the coordination problem as a complicated dynamic stochastic integer programming under network and information constraints. To get an autonomous, distributed, and robust platoon coordination policy, we formulate the problem into a model of the Decentralized-Partial Observable Markov Decision Process. Then, we propose a Multi-Agent Deep Reinforcement Learning framework named Trcuk Attention-QMIX (TA-QMIX) to train an efficient online decision policy. TA-QMIX utilizes the attention mechanism to enhance the representation of truck fuel gains and delay times, and provides explicit truck cooperation information during the training process, promoting trucks' willingness to cooperate. The training framework adopts centralized training and distributed execution, thus training a policy for trucks to make decisions online using only nearby information. Hence, the policy can be autonomously executed on a large-scale network. Finally, we perform comparison experiments and ablation experiments in the transportation network of the Yangtze River Delta region in China to verify the effectiveness of the proposed framework. In a repeated comparative experiment with 5,000 trucks, our method average saves 19.17\% of fuel with an average delay of only 9.57 minutes per truck and a decision time of 0.001 seconds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。