arXiv:2609.06948cs.CV2026-09

用重建证据引导生成模型,实现高保真驾驶视图合成。

PRG-Fusion: Orchestrating Generative Priors with Reconstruction Evidence for Driving View Synthesis

论文配图:PRG-Fusion: Orchestrating Generative Priors with Reconstruction Evidence for Driving View Synthesis
图 1 · 摘自论文原文
  • 通过重建误差生成保留、修复、生成三类区域标签
  • 在新轨迹上合成视频时保持几何一致性和视觉真实感
  • 适合自动驾驶仿真与视觉一致性要求高的场景

沿指定轨迹生成逼真的驾驶视频对可扩展闭环仿真至关重要。基于重建的方法能保持几何一致性,但在偏离训练轨迹时易出现伪影和内容缺失;生成模型虽可跨任意轨迹合成真实画面,却难以保证帧间时空一致性。为此,我们提出PRG-Fusion框架,利用重建证据协调生成先验。具体地,从重建的驾驶场景中提取区域级退化证据,并转化为保留(Preserve)、修复(Repair)、生成(Generate)标签。推理时,这些标签作为统一路由策略,分别驱动3DGS外观保持、LiDAR引导的结构修正以及视频先验的内容补全。采用两阶段训练:先通过稀疏LiDAR投影建立几何控制,再通过密集3DGS渲染学习外观控制。在Waymo数据集上的大量实验表明,PRG-Fusion在新轨迹视频合成上达到领先性能,视觉质量与几何保真度优异,且在大幅轨迹偏移下仍保持良好视图一致性。

原文摘要 · Abstract (English)

Synthesizing photorealistic driving videos along specified trajectories is essential for scalable closed-loop simulation. Reconstruction-based methods leverage neural rendering to synthesize geometrically consistent views, but often exhibit diverse artifacts and missing content when the viewpoint deviates from the training trajectory. In contrast, generative models can synthesize realistic views along arbitrary trajectories from vehicle sensor data, yet often struggle to maintain temporal and geometric consistency across frames. To combine the strengths of both, we propose PRG-Fusion, a framework for driving view synthesis that uses reconstruction evidence to orchestrate generative priors across regions. Specifically, we extract region-wise degradation evidence from reconstructed driving scenes and convert it into Preserve, Repair, and Generate (PRG) labels. At inference, these labels serve as a unified routing policy for region-aware spatiotemporal synthesis, orchestrating 3DGS appearance preservation, LiDAR-guided structural correction, and video-prior-driven content completion across Preserve, Repair, and Generate regions, respectively. We then follow a two-stage training paradigm, first establish geometric control from sparse LiDAR projections and subsequently learning appearance control from dense 3DGS renderings. Extensive experiments on Waymo demonstrate that PRG-Fusion achieves state-of-the-art overall performance in novel trajectory video synthesis, with superior visual quality and geometric fidelity while maintaining competitive view consistency under large trajectory shifts.

视图合成生成模型自动驾驶3DGS

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。