自利的自动驾驶车辆也能提升整体交通效率。
Self-Interest and Systemic Benefits: Emergence of Collective Rationality in Mixed Autonomy Traffic Through Deep Reinforcement Learning
- 用简单奖励设计训练自利车辆,实现集体理性。
- 多种场景下自利车辆仍能提升整体交通性能。
- 适用于未来混合自动驾驶交通系统协同优化。
自动驾驶车辆(AVs)即将商业化,形成由自动驾驶车辆与人类驾驶车辆(HVs)共存的混合自动驾驶交通系统。尽管已有研究证明,将系统级目标融入决策可提升整体交通性能,但当所有驾驶主体(包括人和车)都出于自利动机时,这种益处是否依然存在尚不明确。本文聚焦于集体理性(Collective Rationality, CR)概念,该概念源自博弈论与行为经济学,指个体追求自身利益的同时仍可能实现集体协作。此前研究已通过理论模型和实证数据证明了CR在分析模型与人类驾驶交通中的存在。本文进一步表明,在仅采用深度强化学习(DRL)并设计简单奖励机制的情况下,自利型驾驶代理仍可实现CR。实验结果表明,无论在何种场景下,CR均能稳定涌现,显示其强鲁棒性。我们还提出一种微观动态环境下的机制解释,并基于仿真证据予以验证。研究提示,可通过先进学习方法(如联邦学习)实现自利驱动下驾驶代理间的集体合作。
原文摘要 · Abstract (English)
Autonomous vehicles (AVs) are expected to be commercially available in the near future, leading to mixed autonomy traffic consisting of both AVs and human-driven vehicles (HVs). Although numerous studies have shown that AVs can be deployed to benefit the overall traffic system performance by incorporating system-level goals into their decision making, it is not clear whether the benefits still exist when agents act out of self-interest -- a trait common to all driving agents, both human and autonomous. This study aims to understand whether self-interested AVs can bring benefits to all driving agents in mixed autonomy traffic systems. The research is centered on the concept of collective rationality (CR). This concept, originating from game theory and behavioral economics, means that driving agents may cooperate collectively even when pursuing individual interests. Our recent research has proven the existence of CR in an analytical game-theoretical model and empirically in mixed human-driven traffic. In this paper, we demonstrate that CR can be attained among driving agents trained using deep reinforcement learning (DRL) with a simple reward design. We examine the extent to which self-interested traffic agents can achieve CR without directly incorporating system-level objectives. Results show that CR consistently emerges in various scenarios, which indicates the robustness of this property. We also postulate a mechanism to explain the emergence of CR in the microscopic and dynamic environment and verify it based on simulation evidence. This research suggests the possibility of leveraging advanced learning methods (such as federated learning) to achieve collective cooperation among self-interested driving agents in mixed-autonomy systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。