arXiv:2606.31918cs.CV2026-06中稿 · ECCV

用点云控制生成车辆,让自动驾驶仿真更真实可控。

DriveWeaver: Point-Conditioned Video Inpainting for Controllable Vehicle Insertion in Autonomous Driving Simulation

论文配图:DriveWeaver: Point-Conditioned Video Inpainting for Controllable Vehicle Insertion in Autonomous Driving Simulation
图 1 · 摘自论文原文
  • 以点云为条件进行视频修复,实现车辆无缝插入
  • 在多个数据集上视觉真实感和几何一致性超越基线
  • 支持长期生成与实时渲染,适合大规模仿真场景

自动驾驶仿真中插入具有预设轨迹的前景车辆是关键步骤,可提升场景多样性并生成测试用的极端情况。现有方法依赖预重建的3D资产,常导致光照不一致,且受限于人工标注资产,难以规模化。为此,我们提出DriveWeaver,一种基于视频修复的可控车辆插入框架。针对被遮挡区域,该方法以车辆点云为条件生成高质量、时序一致的车辆图像,实现前景与背景自然融合;点云条件具备良好泛化能力。为支持长时生成,设计全局到局部的分层修复策略,保障插入车辆身份与外观一致性。同时,通过城市重建流程提取显式3D高斯表示,实现车辆的实时渲染。跨多种数据集的实验表明,本方法在视觉真实感与几何一致性方面均优于现有基线,为可扩展的自动驾驶场景增强提供可靠工具。

原文摘要 · Abstract (English)

A pivotal step in autonomous driving simulation involves inserting foreground vehicles with predefined trajectories into simulated scenes. This process enhances scene diversity and facilitates the creation of various corner cases for testing and improving autonomous driving models. However, existing methods often rely on pre-reconstructed 3D assets, which frequently lead to lighting inconsistencies between the inserted foreground and the background. Moreover, the reliance on limited, manually-curated 3D assets hinders large-scale deployment. To address these challenges, we propose DriveWeaver, a novel framework for controllable vehicle insertion in autonomous driving simulation. Specifically, for a masked target insertion area, DriveWeaver performs video inpainting conditioned on vehicle point clouds to generate high-quality, temporally consistent vehicles. This video-inpainting-based approach ensures seamless blending between the foreground and background, while the readily available point cloud conditions enable superior generalization. To support long-term generation, we further design a global-to-local hierarchical inpainting strategy, ensuring the consistent identity and appearance of the inserted vehicles. Meanwhile, we extract explicit 3D Gaussian representations of the inserted vehicles through an urban reconstruction pipeline to enable real-time rendering for autonomous driving simulation. Extensive experiments across diverse datasets demonstrate that our method outperforms existing baselines in visual realism and geometric consistency, providing a robust tool for scalable autonomous driving scene augmentation.

自动驾驶视频修复3D生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。