arXiv:2604.04331cs.CVcs.AI2026-04

用生成模型修复动态物体遮挡区域,提升静态场景重建精度

GA-GS: Generation-Assisted Gaussian Splatting for Static Scene Reconstruction

  • 通过运动感知模块分离动态物体,用扩散模型补全遮挡区
  • 在DAVIS和自建数据集上实现当前最佳重建效果,尤其擅长大范围遮挡
  • 适合需要高精度静态场景重建的VR、自动驾驶等应用

从带动态物体的单目视频中重建静态三维场景对虚拟现实、自动驾驶等应用至关重要。现有方法通常依赖背景信息进行重建,难以恢复被动态物体遮挡的区域。本文提出GA-GS:一种生成辅助的高斯点阵静态场景重建方法。核心思想是利用生成模型辅助恢复遮挡区域。首先通过运动感知模块分割并移除动态区域,再使用扩散模型对遮挡区域进行图像修复,生成伪真值监督信号。为平衡真实背景与生成内容的贡献,我们为每个高斯原始引入可学习的真实性标量,在点阵渲染时动态调节不透明度,实现真实性感知的渲染与监督。由于现有数据集缺乏带动态物体的静态场景真值,我们构建了名为Trajectory-Match的数据集,通过固定路径机器人在有/无动态物体情况下拍摄同一场景,支持遮挡区域重建的定量评估。在DAVIS及自建数据集上的大量实验表明,GA-GS在静态场景重建上达到当前最优性能,尤其在大规模持久遮挡的挑战性场景中表现突出。

原文摘要 · Abstract (English)

Reconstructing static 3D scene from monocular video with dynamic objects is important for numerous applications such as virtual reality and autonomous driving. Current approaches typically rely on background for static scene reconstruction, limiting the ability to recover regions occluded by dynamic objects. In this paper, we propose GA-GS, a Generation-Assisted Gaussian Splatting method for Static Scene Reconstruction. The key innovation of our work lies in leveraging generation to assist in reconstructing occluded regions. We employ a motion-aware module to segment and remove dynamic regions, and thenuse a diffusion model to inpaint the occluded areas, providing pseudo-ground-truth supervision. To balance contributions from real background and generated region, we introduce a learnable authenticity scalar for each Gaussian primitive, which dynamically modulates opacity during splatting for authenticity-aware rendering and supervision. Since no existing dataset provides ground-truth static scene of video with dynamic objects, we construct a dataset named Trajectory-Match, using a fixed-path robot to record each scene with/without dynamic objects, enabling quantitative evaluation in reconstruction of occluded regions. Extensive experiments on both the DAVIS and our dataset show that GA-GS achieves state-of-the-art performance in static scene reconstruction, especially in challenging scenarios with large-scale, persistent occlusions.

3D重建生成模型高斯点阵遮挡修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。