用更真实的智能交通代理重评自动驾驶规划器,发现旧评估方式高估了性能。
When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarks
- 引入先进学习型交通代理SMART,替代传统规则代理。
- 14个规划器在真实场景中性能普遍下降,多车道交互表现反而提升。
- 建议以SMART反应式仿真作为nuPlan新基准,适合追求真实评估的开发者。
封闭环路仿真中常使用基于规则的交通代理,其行为简单被动,可能掩盖规划器缺陷并扭曲排名。广泛使用的IDM代理仅跟随前车,无法响应邻道车辆,难以测试复杂交互能力。本文首次将最先进的学习型交通代理SMART集成至nuPlan,实现更贴近现实的规划器评估,并量化模拟到现实差距缩小后结论的变化。分析涵盖14个近期规划器与基线方法,结果显示:基于IDM的仿真高估了规划性能,几乎所有评分均下降;而许多规划器在多车道、高交互场景(如变道或转弯)中的表现优于预期,甚至提升。闭环训练的方法展现出最佳且最稳定的驾驶表现。但在增强的边缘场景下,所有学习型规划器会突然退化,而规则代理仍能维持基本合理行为。据此,我们建议将SMART反应式仿真作为nuPlan的新标准封闭环路基准,并已将SMART代理开源,可直接替换IDM:https://github.com/shgd95/InteractiveClosedLoop。
原文摘要 · Abstract (English)
Planner evaluation in closed-loop simulation often uses rule-based traffic agents, whose simplistic and passive behavior can hide planner deficiencies and bias rankings. Widely used IDM agents simply follow a lead vehicle and cannot react to vehicles in adjacent lanes, hindering tests of complex interaction capabilities. We address this issue by integrating the state-of-the-art learned traffic agent model SMART into nuPlan. Thus, we are the first to evaluate planners under more realistic conditions and quantify how conclusions shift when narrowing the sim-to-real gap. Our analysis covers 14 recent planners and established baselines and shows that IDM-based simulation overestimates planning performance: nearly all scores deteriorate. In contrast, many planners interact better than previously assumed and even improve in multi-lane, interaction-heavy scenarios like lane changes or turns. Methods trained in closed-loop demonstrate the best and most stable driving performance. However, when reaching their limits in augmented edge-case scenarios, all learned planners degrade abruptly, whereas rule-based planners maintain reasonable basic behavior. Based on our results, we suggest SMART-reactive simulation as a new standard closed-loop benchmark in nuPlan and release the SMART agents as a drop-in alternative to IDM at https://github.com/shgd95/InteractiveClosedLoop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。