提出首个面向自动驾驶通用性的行为评估基准,测试大规模强化学习模型的真实泛化能力。
Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving

- 构建BehaviorBench评测框架,支持大模型在nuPlan上标准化评估
- 引入复杂交互场景,仅靠简单规则无法取得高分
- 使用多样化交通行为模型,揭示自对战训练的泛化缺陷
近期自动驾驶研究如GigaFlow和PufferDrive通过大规模强化学习(RL)训练驾驶策略,但这些策略缺乏与标准基准的连接,导致其在真实评测中的表现不明。本文提出BehaviorBench——一个涵盖评估、复杂性和行为多样性的综合评测套件。在评估方面,首次实现PufferDrive与nuPlan的接口对接,使大规模强化学习训练的策略可在成熟规划基准上评估;同时提供可在PufferDrive仿真中快速运行的评估框架。在复杂性方面,发现现有基准过于简单,仅靠车道保持加碰撞检测即可接近满分;因此从Waymo Open Motion Dataset(WOMD)中提取出富含交互的子集,要求必须具备多智能体推理才能获得高分。在行为多样性方面,传统基准仅使用IDM规则交通模型,本文则提供多样化交互式交通代理,以测试策略在异构行为下的鲁棒性。分析表明:尽管通过自对战学习了交互行为,但在标准奖励函数下训练的策略仍会过拟合训练对手,无法泛化至其他交通行为。基于此,提出一种结合PPO策略与规则基规划器的混合规划方案。
原文摘要 · Abstract (English)
Recent Autonomous Driving (AD) works such as GigaFlow and PufferDrive have unlocked Reinforcement Learning (RL) at scale as a training strategy for driving policies. Yet such policies remain disconnected from established benchmarks, leaving the performance of large-scale RL for driving on standardized evaluations unknown. We present BehaviorBench -- a comprehensive test suite that closes this gap along three axes: Evaluation, Complexity, and Behavior Diversity. In terms of Evaluation, we provide an interface connecting PufferDrive to nuPlan, which, for the first time, enables policies trained via RL at scale to be evaluated on an established planning benchmark for autonomous driving. Complementarily, we offer an evaluation framework that allows planners to be benchmarked directly inside the PufferDrive simulation, at a fraction of the time. Regarding Complexity, we observe that today's standardized benchmarks are so simple that near-perfect scores are achievable by straight lane following with collision checking. We extract a meaningful, interaction-rich split from the Waymo Open Motion Dataset (WOMD) on which strong performance is impossible without multi-agent reasoning. Lastly, we address Behavior Diversity. Existing benchmarks commonly evaluate planners against a single rule-based traffic model, the Intelligent Driver Model (IDM). We provide a diverse suite of interactive traffic agents to stress-test policies under heterogeneous behaviors, beyond just using IDM. Overall, our benchmarking analysis uncovers the following insight: despite learning interactive behaviors in an emergent manner, policies trained via pure self-play under standard reward functions overfit to their training opponents and fail to generalize to other traffic agent behaviors. Building on this observation, we propose a hybrid planner that combines a PPO policy with a rule-based planner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。