为多智能体强化学习提供真实交通场景的基准测试,评估车辆协同减排效果。
IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning
- 基于16,334个路口的真实城市数据构建仿真环境,模拟百万级交通场景。
- 多智能体强化学习算法在跨场景泛化任务中表现不佳,难以适应复杂变化。
- 适合研究交通控制、多智能体协同与可泛化强化学习的学者和工程师。
尽管多智能体强化学习(RL)在模拟和双人应用中广受欢迎,但其在复杂现实应用中的成功有限,主要挑战在于对问题变化的泛化能力。情境强化学习(CRL)旨在学习能在不同变体间通用的策略。然而,缺乏标准化的多智能体CRL基准阻碍了该领域进展。为此,我们提出IntersectionZoo,一个基于城市道路网络中协同节能驾驶的真实应用场景的多智能体CRL基准套件。该任务目标是控制车队车辆以降低整体排放。借助来自美国10个主要城市的16,334个信号交叉口数据,结合开源工业级微观交通仿真器,构建了百万级数据驱动的交通场景。通过建模温度、道路状况、出行需求等因素对尾气排放的影响,系统全面捕捉了部分可观测性与多重竞争目标等现实问题特征。我们使用这些场景对主流多智能体RL与类人驾驶算法进行基准测试,结果表明:现有多智能体强化学习算法在CRL设置下泛化能力显著不足。
原文摘要 · Abstract (English)
Despite the popularity of multi-agent reinforcement learning (RL) in simulated and two-player applications, its success in messy real-world applications has been limited. A key challenge lies in its generalizability across problem variations, a common necessity for many real-world problems. Contextual reinforcement learning (CRL) formalizes learning policies that generalize across problem variations. However, the lack of standardized benchmarks for multi-agent CRL has hindered progress in the field. Such benchmarks are desired to be based on real-world applications to naturally capture the many open challenges of real-world problems that affect generalization. To bridge this gap, we propose IntersectionZoo, a comprehensive benchmark suite for multi-agent CRL through the real-world application of cooperative eco-driving in urban road networks. The task of cooperative eco-driving is to control a fleet of vehicles to reduce fleet-level vehicular emissions. By grounding IntersectionZoo in a real-world application, we naturally capture real-world problem characteristics, such as partial observability and multiple competing objectives. IntersectionZoo is built on data-informed simulations of 16,334 signalized intersections derived from 10 major US cities, modeled in an open-source industry-grade microscopic traffic simulator. By modeling factors affecting vehicular exhaust emissions (e.g., temperature, road conditions, travel demand), IntersectionZoo provides one million data-driven traffic scenarios. Using these traffic scenarios, we benchmark popular multi-agent RL and human-like driving algorithms and demonstrate that the popular multi-agent RL algorithms struggle to generalize in CRL settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。