arXiv:2606.16278cs.CVcs.AI2026-06

让编辑过的3D模拟视频更真实,提升自动驾驶安全性测试可信度。

RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos

论文配图:RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos
图 1 · 摘自论文原文
  • 用多模态信号引导视频基础模型,自适应融合修复信息。
  • 在CARLA和真实视频数据集上显著降低渲染瑕疵与光影不一致。
  • 适合自动驾驶仿真系统开发者及高阶视觉算法研究者。

长尾危险场景对安全导向的自动驾驶至关重要,但难以大规模采集。可编辑的3D高斯点云(3DGS)仿真通过真实场景重建和可控编辑提供了可扩展替代方案。然而,编辑后的3DGS渲染视频常存在显著的模拟到现实差距,表现为渲染伪影、前景质量下降、光照不匹配和时间闪烁。解决这些耦合缺陷需同时恢复局部外观、调和编辑内容并保持时间一致性,而现有方法通常仅处理部分需求。为此,我们提出RealityBridge,一个视频修复与调和框架,可在保留模拟器定义结构、编辑内容和动态的前提下,将编辑后的3DGS渲染转化为逼真驾驶画面。RealityBridge以互补模态信号为条件,通过轻量级GateNet自适应控制其在骨干网络中的注入。我们还构建了面向任务的数据清洗流水线,并设计四阶段监督训练策略,随后进行奖励引导的后训练。大量实验表明,RealityBridge在修复与调和方面优于现有方法,同时保持强时间一致性。

原文摘要 · Abstract (English)

Long-tail hazardous scenarios are essential for safety-oriented autonomous driving, yet they are difficult to collect at scale. Editable 3D Gaussian Splatting (3DGS) simulation offers a scalable alternative through real-scene reconstruction and controllable editing. However, edited 3DGS-rendered videos often exhibit a significant Sim-to-Real gap, manifested as rendering artifacts, degraded foreground assets, illumination mismatch, and temporal flickering. Addressing these coupled defects requires jointly restoring local appearance, harmonizing edited content, and maintaining temporal consistency, whereas existing methods typically address only a subset of these requirements. To fill this gap, we propose RealityBridge, a video restoration and harmonization framework that converts edited 3DGS renderings into realistic driving footage while preserving simulator-defined structure, edits, and dynamics. RealityBridge conditions a video foundation model on complementary modality signals, with a lightweight GateNet adaptively controlling their injection across backbone blocks. We further develop a task-oriented curation pipeline to construct training data, and design a four-stage supervised training strategy followed by reward-guided post-training. Extensive experiments demonstrate that RealityBridge outperforms existing methods in restoration and harmonization while preserving strong temporal consistency.

3D高斯视频修复自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。