arXiv:2504.04679cs.CV2025-04CVPR被引 1

不依赖生成模型,用优化方法修复被遮挡的3D场景。

DeclutterNeRF: Generative-Free 3D Scene Recovery for Occlusion Removal

  • 通过多视角联合优化和渐进式遮挡正则化消除遮挡。
  • 在新数据集上显著优于现有方法,重建无伪影、细节更真实。
  • 适合做3D重建与视觉修复的研究者参考。

近期的新视角合成技术,包括神经辐射场(NeRF)和3D高斯溅射(3DGS),已在高质量渲染和真实细节恢复方面取得显著进展。有效去除遮挡并保留场景细节,可进一步提升这些技术的鲁棒性和应用性。然而,现有物体与遮挡去除方法大多依赖生成先验,虽能填充空洞,却引入新伪影和模糊。此外,现有评估遮挡去除方法的基准数据集缺乏真实复杂度和视角变化。为此,我们提出DeclutterSet,一个包含丰富场景的新数据集,其前景、中景和背景均存在明显遮挡,并在不同视角间表现出显著相对运动。我们进一步提出DeclutterNeRF,一种无需生成先验的遮挡去除方法。该方法结合可学习相机参数的联合多视图优化、遮挡退火正则化,并采用可解释的随机结构相似性损失,从不完整图像中实现高质量、无伪影的重建。实验表明,DeclutterNeRF在所提的DeclutterSet上显著优于现有最先进方法,为未来研究建立了强基准。

原文摘要 · Abstract (English)

Recent novel view synthesis (NVS) techniques, including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have greatly advanced 3D scene reconstruction with high-quality rendering and realistic detail recovery. Effectively removing occlusions while preserving scene details can further enhance the robustness and applicability of these techniques. However, existing approaches for object and occlusion removal predominantly rely on generative priors, which, despite filling the resulting holes, introduce new artifacts and blurriness. Moreover, existing benchmark datasets for evaluating occlusion removal methods lack realistic complexity and viewpoint variations. To address these issues, we introduce DeclutterSet, a novel dataset featuring diverse scenes with pronounced occlusions distributed across foreground, midground, and background, exhibiting substantial relative motion across viewpoints. We further introduce DeclutterNeRF, an occlusion removal method free from generative priors. DeclutterNeRF introduces joint multi-view optimization of learnable camera parameters, occlusion annealing regularization, and employs an explainable stochastic structural similarity loss, ensuring high-quality, artifact-free reconstructions from incomplete images. Experiments demonstrate that DeclutterNeRF significantly outperforms state-of-the-art methods on our proposed DeclutterSet, establishing a strong baseline for future research.

3D重建遮挡修复优化方法无生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。