用轻量级3D渲染增强端到端自动驾驶,提升闭环鲁棒性。
RAP: 3D Rasterization Augmented End-to-End Planning
- 用标注图元的3D光栅化替代昂贵渲染,实现高效数据增广
- 在4个主流基准上达到最优闭环鲁棒性和长尾泛化性能
- 适合追求实用、可扩展的自动驾驶规划系统开发者
端到端自动驾驶的模仿学习仅依赖专家示范训练策略。部署于闭环后,此类策略缺乏纠错数据:微小失误无法修正并迅速累积为失败。一个有前景的方向是生成日志路径之外的视角与轨迹。先前工作通过神经渲染或游戏引擎构建逼真数字孪生,但成本高昂且速度慢,仅用于评估。本文认为训练规划器无需逼真视觉效果,关键在于语义保真度和可扩展性:驾驶依赖几何与动力学,而非纹理或光照。为此,提出3D光栅化,以轻量级光栅化标注图元替代高成本渲染,支持反事实恢复操作和跨智能体视角合成。为有效将合成视图迁移至真实场景,引入光栅到现实特征空间对齐,弥合仿真到现实的差距。上述组件构成可扩展的数据增广管道RAP,显著提升端到端规划的闭环比对鲁棒性与长尾泛化能力,在NAVSIM v1/v2、Waymo Open Dataset Vision-based E2E Driving和Bench2Drive四个主要基准上排名第一。结果表明,结合特征对齐的轻量级光栅化足以支撑端到端训练规模化,提供比逼真渲染更实用的替代方案。
原文摘要 · Abstract (English)
Imitation learning for end-to-end driving trains policies only on expert demonstrations. Once deployed in a closed loop, such policies lack recovery data: small mistakes cannot be corrected and quickly compound into failures. A promising direction is to generate alternative viewpoints and trajectories beyond the logged path. Prior work explores photorealistic digital twins via neural rendering or game engines, but these methods are prohibitively slow and costly, and thus mainly used for evaluation. In this work, we argue that photorealism is unnecessary for training end-to-end planners. What matters is semantic fidelity and scalability: driving depends on geometry and dynamics, not textures or lighting. Motivated by this, we propose 3D Rasterization, which replaces costly rendering with lightweight rasterization of annotated primitives, enabling augmentations such as counterfactual recovery maneuvers and cross-agent view synthesis. To transfer these synthetic views effectively to real-world deployment, we introduce a Raster-to-Real feature-space alignment that bridges the sim-to-real gap. Together, these components form Rasterization Augmented Planning (RAP), a scalable data augmentation pipeline for planning. RAP achieves state-of-the-art closed-loop robustness and long-tail generalization, ranking first on four major benchmarks: NAVSIM v1/v2, Waymo Open Dataset Vision-based E2E Driving, and Bench2Drive. Our results show that lightweight rasterization with feature alignment suffices to scale E2E training, offering a practical alternative to photorealistic rendering. Project page: https://alan-lanfeng.github.io/RAP/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。