用外部记忆库替换真实车辆，实现更逼真的自动驾驶场景生成
MADrive: Memory-Augmented Driving Scene Modeling
- 通过外部记忆库检索相似3D车辆模型并替换原场景
- 基于7万+辆汽车视频重建3D资产，支持多视角呈现
- 适合需要生成多样化驾驶场景的研究者与开发者
近期的场景重建进展推动了自动驾驶环境的高保真建模，采用3D高斯点阵技术。然而，现有重建结果仍紧密依赖原始观测数据，难以支持显著改变或全新驾驶场景的逼真合成。本文提出MADrive，一种基于记忆增强的重建框架，通过从大规模外部记忆库中检索视觉相似的3D车辆资产，替代原始观测中的车辆。我们发布了MAD-Cars数据集，包含约70,000个真实世界采集的360°汽车视频，并设计了检索模块：在记忆库中定位最相似车辆实例，从视频重建对应3D资产，并通过姿态对齐与再打光技术整合进目标场景。替换后的车辆具备完整的多视角表示，显著提升了复杂场景配置下的逼真合成能力，实验验证了其有效性。
原文摘要 · Abstract (English)
Recent advances in scene reconstruction have pushed toward highly realistic modeling of autonomous driving (AD) environments using 3D Gaussian splatting. However, the resulting reconstructions remain closely tied to the original observations and struggle to support photorealistic synthesis of significantly altered or novel driving scenarios. This work introduces MADrive, a memory-augmented reconstruction framework designed to extend the capabilities of existing scene reconstruction methods by replacing observed vehicles with visually similar 3D assets retrieved from a large-scale external memory bank. Specifically, we release MAD-Cars, a curated dataset of ${\sim}70$K 360° car videos captured in the wild and present a retrieval module that finds the most similar car instances in the memory bank, reconstructs the corresponding 3D assets from video, and integrates them into the target scene through orientation alignment and relighting. The resulting replacements provide complete multi-view representations of vehicles in the scene, enabling photorealistic synthesis of substantially altered configurations, as demonstrated in our experiments. Project page: https://yandex-research.github.io/madrive/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。