arXiv:2601.11030cs.CVcs.AI2026-01中稿 · ACM-MM23被引 8

用2D检测器去除3D场景中多种干扰物,实现端到端重建。

IDDR-NGP: Incorporating Detectors for Distractor Removal with Instant Neural Radiance Field

  • 结合2D检测与隐式3D表示,统一处理多种干扰物。
  • 在真实与合成场景中均实现高精度去干扰,效果接近当前最佳去雪方法。
  • 适用于视频/图像修复、3D重建等需去杂项的领域。

本文提出首个统一的去干扰方法IDDR-NGP,直接作用于即时神经辐射场(Instant-NGP)。该方法可有效去除雪片、彩纸、落叶、花瓣等多种类型干扰物,而现有方法通常仅针对单一类型。通过融合隐式3D表示与2D检测器,实现从多幅受污染图像中高效恢复高质量3D场景。设计了学习感知图像块相似性(LPIPS)损失与多视角补偿损失(MVCL),联合优化渲染结果,充分聚合多视角信息。所有模块可端到端训练。为推动隐式3D表示中的去干扰研究,构建了一个包含合成与真实干扰物的新基准数据集,并在真实与合成场景中添加对应标注标签。大量实验表明,IDDR-NGP在去除多种干扰物方面具有优异有效性与鲁棒性,其去雪效果媲美当前最优方法,且能准确处理真实与合成干扰物。

原文摘要 · Abstract (English)

This paper presents the first unified distractor removal method, named IDDR-NGP, which directly operates on Instant-NPG. The method is able to remove a wide range of distractors in 3D scenes, such as snowflakes, confetti, defoliation and petals, whereas existing methods usually focus on a specific type of distractors. By incorporating implicit 3D representations with 2D detectors, we demonstrate that it is possible to efficiently restore 3D scenes from multiple corrupted images. We design the learned perceptual image patch similarity~( LPIPS) loss and the multi-view compensation loss (MVCL) to jointly optimize the rendering results of IDDR-NGP, which could aggregate information from multi-view corrupted images. All of them can be trained in an end-to-end manner to synthesize high-quality 3D scenes. To support the research on distractors removal in implicit 3D representations, we build a new benchmark dataset that consists of both synthetic and real-world distractors. To validate the effectiveness and robustness of IDDR-NGP, we provide a wide range of distractors with corresponding annotated labels added to both realistic and synthetic scenes. Extensive experimental results demonstrate the effectiveness and robustness of IDDR-NGP in removing multiple types of distractors. In addition, our approach achieves results comparable with the existing SOTA desnow methods and is capable of accurately removing both realistic and synthetic distractors.

3D重建去干扰神经渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。