通过在线修复与渐进更新,提升复杂变道场景的驾驶环境重建精度。
ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration
- 引入在线修复模块,动态消除新轨迹中的渲染伪影。
- 在复杂变道场景下,NTA-IoU指标提升24.87%,FID降低29.97%。
- 适合需要高精度闭环仿真的自动驾驶系统研发人员使用。
闭环仿真对端到端自动驾驶至关重要。现有传感器仿真方法(如NeRF和3DGS)基于接近训练数据分布的条件重建驾驶场景,但在处理新轨迹(如变道)时表现不佳。近期工作表明,融合世界模型知识可缓解该问题,但仍难以准确表示复杂操作,尤其多车道变道。为此,我们提出ReconDreamer,通过增量式整合世界模型知识提升场景重建能力。具体地,提出DriveRestorer实现在线修复以减少伪影,并设计渐进式数据更新策略,保障复杂操作下的高质量渲染。据我们所知,ReconDreamer是首个能有效渲染大规模变道的方案。实验表明,其在NTA-IoU、NTL-IoU和FID上优于Street Gaussians,相对提升分别为24.87%、6.72%和29.97%;在大尺度变道中,相较DriveDreamer4D(PVG)提升195.87%的NTA-IoU,用户研究亦验证其优越性。
原文摘要 · Abstract (English)
Closed-loop simulation is crucial for end-to-end autonomous driving. Existing sensor simulation methods (e.g., NeRF and 3DGS) reconstruct driving scenes based on conditions that closely mirror training data distributions. However, these methods struggle with rendering novel trajectories, such as lane changes. Recent works have demonstrated that integrating world model knowledge alleviates these issues. Despite their efficiency, these approaches still encounter difficulties in the accurate representation of more complex maneuvers, with multi-lane shifts being a notable example. Therefore, we introduce ReconDreamer, which enhances driving scene reconstruction through incremental integration of world model knowledge. Specifically, DriveRestorer is proposed to mitigate artifacts via online restoration. This is complemented by a progressive data update strategy designed to ensure high-quality rendering for more complex maneuvers. To the best of our knowledge, ReconDreamer is the first method to effectively render in large maneuvers. Experimental results demonstrate that ReconDreamer outperforms Street Gaussians in the NTA-IoU, NTL-IoU, and FID, with relative improvements by 24.87%, 6.72%, and 29.97%. Furthermore, ReconDreamer surpasses DriveDreamer4D with PVG during large maneuver rendering, as verified by a relative improvement of 195.87% in the NTA-IoU metric and a comprehensive user study.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。