用结构化网格指导全景图去家具化，生成更清晰的室内复刻场景。
Defurnishing with X-Ray Vision: Joint Removal of Furniture from Panoramas and Mesh
- 基于简化网格生成'X光'式结构引导，确保去家具化全局一致性。
- 通过边缘控制实现全景图精准修复，避免传统方法的幻觉问题。
- 适合需要高保真室内重建的3D资产制作与数字孪生应用。
我们提出一种生成室内空间去家具化复刻版本的流水线，输入为带纹理的网格和多视角全景图像。首先对网格中的家具进行分割与移除，扩展平面并填充空洞，得到简化的去家具化网格（SDM）。该网格作为场景结构的“X光”视图，指导后续处理。从SDM渲染深度与法向图，提取Canny边缘作为控制信号，利用ControlNet图像修复技术去除全景图中的家具。该控制信号保留了被遮挡的全局几何信息，提升修复质量。修复后的全景图用于重新贴图网格。实验表明，本方法生成的资产在清晰度和细节表现上优于依赖神经辐射场的方法（易模糊、低分辨率）或仅使用RGB-D修复的方法（易产生幻觉）。
原文摘要 · Abstract (English)
We present a pipeline for generating defurnished replicas of indoor spaces represented as textured meshes and corresponding multi-view panoramic images. To achieve this, we first segment and remove furniture from the mesh representation, extend planes, and fill holes, obtaining a simplified defurnished mesh (SDM). This SDM acts as an ``X-ray'' of the scene's underlying structure, guiding the defurnishing process. We extract Canny edges from depth and normal images rendered from the SDM. We then use these as a guide to remove the furniture from panorama images via ControlNet inpainting. This control signal ensures the availability of global geometric information that may be hidden from a particular panoramic view by the furniture being removed. The inpainted panoramas are used to texture the mesh. We show that our approach produces higher quality assets than methods that rely on neural radiance fields, which tend to produce blurry low-resolution images, or RGB-D inpainting, which is highly susceptible to hallucinations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。