用语义引导分步修复3D场景中移除物体的空缺区域
Semantic-Guided Progressive Object Removal with Gaussian Splatting
- 利用多视角语义匹配，分步修复缺失区域并保持跨视图一致
- 在ScanNet和Mip-NeRF360上实现更高感知质量和几何连贯性
- 适合需要高质量3D重建的AR/VR与内容创作场景
从重建的3D场景中移除不需要的物体是计算机视觉中的重要任务，广泛应用于AR/VR、机器人和数字内容创作。现有方法通常在单步内完成掩码区域填充，且未能有效利用其他视角的语义信息，导致难以处理复杂的几何细节和纹理。本文提出一种新框架，结合语义引导块匹配（SBM）与区域级渐进式优化（RPR），实现高质量3D物体移除。首先，通过DINOv2编码多视角语义信息，将最佳匹配特征解码至目标视图以填补缺失区域，保持跨视图一致性；其次，引入RPR策略，将目标掩码分割为多个子区域，并对视觉质量较差的部分进行选择性精细化修复。本方法基于高斯点阵（Gaussian Splatting），在保证高保真度的同时具备高效计算能力。实验表明，该方法在扫描数据集ScanNet和基准测试Mip-NeRF360上，显著优于现有基于高斯的方法，在感知质量与几何连贯性方面均表现更优。
原文摘要 · Abstract (English)
Removing unwanted objects from reconstructed 3D scenes is an important task in computer vision, supporting applications in AR/VR, robotics, and digital content creation. Existing methods typically complete the entire masked region in a single step and without effectively utilizing semantic information from other views, leading to difficulties in handling complex geometric details and textures. In this work, we propose a novel framework that integrates Semantic-guided Block Matching (SBM) and Region-Wise Progressive Refinement (RPR) for high-quality 3D object removal. First, we leverage DINOv2 to encode semantic guidance from multi-view observations, and the best match tokens are decoded to complete missing regions in the target view while maintaining cross-view consistency. Second, we introduce a RPR strategy that segments the target mask into multiple subregions and selectively refines those with poor visual quality. Our method is built upon Gaussian Splatting, ensuring high-fidelity scene reconstruction with efficient computation. Experimental results demonstrate that our approach outperforms existing Gaussian-based methods in terms of perceptual quality and coherence in 3D object removal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。