用强化学习模拟自动驾驶汽车，发现不同行为模式对交通影响差异大。
Impact of Collective Behaviors of Autonomous Vehicles on Urban Traffic Dynamics: A Multi-Agent Reinforcement Learning Approach
- 用深度Q-learning让自动驾驶车在多智能体环境中学习不同行为策略。
- 自动驾驶车可缩短5%行程时间，但对人类司机影响因行为而异。
- 适合研究交通优化与智能体协作的学者，尤其关注行为设计影响。
本研究探讨强化学习（RL）驱动的自动驾驶汽车（AV）在混合交通环境下对城市交通流的影响。聚焦于多智能体设置下的简化日间路径选择问题：城市路网中人类驾驶员选择最短路径出行；将三分之一人口替换为采用深度Q-learning算法的自动驾驶车辆，赋予其自私、合作、竞争、社交、利他及恶意等六种行为目标，通过奖励机制实现。使用自研的多智能体强化学习框架PARCOUR进行仿真。结果显示，自动驾驶车可将自身行程时间优化最多5%，但对人类驾驶员的影响随行为模式显著变化。所有自我导向行为下，自动驾驶车均比人类驾驶员更快到达。研究揭示了各类行为的学习难度差异，并证明多智能体强化学习适用于交通网络集体路径规划，但对共存群体的影响高度依赖所采用的行为类型。
原文摘要 · Abstract (English)
This study examines the potential impact of reinforcement learning (RL)-enabled autonomous vehicles (AV) on urban traffic flow in a mixed traffic environment. We focus on a simplified day-to-day route choice problem in a multi-agent setting. We consider a city network where human drivers travel through their chosen routes to reach their destinations in minimum travel time. Then, we convert one-third of the population into AVs, which are RL agents employing Deep Q-learning algorithm. We define a set of optimization targets, or as we call them behaviors, namely selfish, collaborative, competitive, social, altruistic, and malicious. We impose a selected behavior on AVs through their rewards. We run our simulations using our in-house developed RL framework PARCOUR. Our simulations reveal that AVs optimize their travel times by up to 5\%, with varying impacts on human drivers' travel times depending on the AV behavior. In all cases where AVs adopt a self-serving behavior, they achieve shorter travel times than human drivers. Our findings highlight the complexity differences in learning tasks of each target behavior. We demonstrate that the multi-agent RL setting is applicable for collective routing on traffic networks, though their impact on coexisting parties greatly varies with the behaviors adopted.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。