利用镜子反射提升复杂场景3D重建质量
Seeing Through Reflections: Advancing 3D Scene Reconstruction in Mirror-Containing Environments with Gaussian Splatting
- 将镜面反射视为互补视角而非干扰因素
- 在镜面环境中实现更高精度与更快训练速度
- 适合需要高保真3D重建的视觉应用
含镜面环境对3D重建和新视角合成（NVS）带来独特挑战，因反射表面引入视点依赖的畸变与不一致。尽管神经辐射场（NeRF）和3D高斯溅射（3DGS）在常规场景中表现优异，但在镜面存在时性能下降。现有方法多通过对称映射处理镜面，却忽视了反射所携带的丰富信息。这些反射可提供互补视角，填补缺失细节，显著提升重建质量。为此，我们构建了MirrorScene3D数据集，包含多样室内场景、1256张高质量图像及标注的镜面掩码，为反射场景下的重建方法提供基准。基于此，我们提出ReflectiveGS，作为3DGS的扩展，将镜面反射用作互补视角，增强场景几何并恢复缺失细节。在MirrorScene3D上的实验表明，ReflectiveGS在SSIM、PSNR、LPIPS指标及训练速度上均优于现有方法，建立了镜面丰富环境下的3D重建新基准。
原文摘要 · Abstract (English)
Mirror-containing environments pose unique challenges for 3D reconstruction and novel view synthesis (NVS), as reflective surfaces introduce view-dependent distortions and inconsistencies. While cutting-edge methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) excel in typical scenes, their performance deteriorates in the presence of mirrors. Existing solutions mainly focus on handling mirror surfaces through symmetry mapping but often overlook the rich information carried by mirror reflections. These reflections offer complementary perspectives that can fill in absent details and significantly enhance reconstruction quality. To advance 3D reconstruction in mirror-rich environments, we present MirrorScene3D, a comprehensive dataset featuring diverse indoor scenes, 1256 high-quality images, and annotated mirror masks, providing a benchmark for evaluating reconstruction methods in reflective settings. Building on this, we propose ReflectiveGS, an extension of 3D Gaussian Splatting that utilizes mirror reflections as complementary viewpoints rather than simple symmetry artifacts, enhancing scene geometry and recovering absent details. Experiments on MirrorScene3D show that ReflectiveGaussian outperforms existing methods in SSIM, PSNR, LPIPS, and training speed, setting a new benchmark for 3D reconstruction in mirror-rich environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。