用生成模型让自动驾驶评估更真实,支持周围车辆实时反应。
Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model
- 分离传感器渲染与行为决策,实现真实交互模拟。
- 生成视频时长超过10秒,保持时空一致性,避免画面扭曲。
- 适合研究自动驾驶闭环评估的学者和工程师使用。
针对端到端自动驾驶的评估难题,现有闭环评估多依赖如CARLA等仿真器,真实性不足;而NAVSIM虽使用真实视觉数据,但仅限于固定规划轨迹且假设其他交通参与者无反应。本文提出Bench2Drive-R,一种生成式框架,实现具备交互能力的闭环评估。不同于传统自动驾驶视频生成模型,该框架将传感器渲染与行为推演解耦,通过独立的行为控制器模拟周边车辆的反应。渲染模块专注图像保真度、控制匹配度及时空连贯性。为保证时间一致性,设计噪声调制的时间编码器与高斯模糊机制,支持长达10秒以上的自回归序列生成而不产生分布偏移。为保障空间一致性,引入基于最近邻图像的检索机制,结合3D相对位置编码显式建模目标与参考图像的空间关系,并通过分层采样与无分类器引导缓解参考图像过度依赖问题。在生成质量上优于现有模型,进一步集成至nuPlan进行闭环仿真验证,效果显著。代码将开源。
原文摘要 · Abstract (English)
For end-to-end autonomous driving (E2E-AD), the evaluation system remains an open problem. Existing closed-loop evaluation protocols usually rely on simulators like CARLA being less realistic; while NAVSIM using real-world vision data, yet is limited to fixed planning trajectories in short horizon and assumes other agents are not reactive. We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation. Unlike existing video generative models for AD, the proposed designs are tailored for interactive simulation, where sensor rendering and behavior rollout are decoupled by applying a separate behavioral controller to simulate the reactions of surrounding agents. As a result, the renderer could focus on image fidelity, control adherence, and spatial-temporal coherence. For temporal consistency, due to the step-wise interaction nature of simulation, we design a noise modulating temporal encoder with Gaussian blurring to encourage long-horizon autoregressive rollout of image sequences without deteriorating distribution shifts. For spatial consistency, a retrieval mechanism, which takes the spatially nearest images as references, is introduced to to ensure scene-level rendering fidelity during the generation process. The spatial relations between target and reference are explicitly modeled with 3D relative position encodings and the potential over-reliance of reference images is mitigated with hierarchical sampling and classifier-free guidance. We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance. We further integrate Bench2Drive-R into nuPlan and evaluate the generative qualities with closed-loop simulation results. We will open source our code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。