让3D高斯点云正确渲染镜面反射，避免虚空间错觉。
Gaussian Splatting in Mirrors: Reflection-Aware Rendering via Virtual Camera Optimization
- 用物理对称虚拟相机模拟镜面反射，替代传统错误建模。
- 在真实场景镜面数据集上实现高质量多视角一致渲染。
- 训练速度更快，适合需要实时渲染的工业应用。
3D高斯点云（3D-GS）在新视角合成中实现了实时高质量渲染，但在包含镜面等反射表面的场景中，常将反射误认为虚拟空间，导致镜内图像模糊且视角不一致。本文提出一种新方法，通过将反射建模为基于物理的虚拟相机来提升镜面渲染质量。利用3D-GS输出的深度与法线估计镜面平面，并在镜面对称位置设置虚拟相机以解释反射内容。针对镜面平面估计误差，设计了一种简单有效的虚拟相机优化策略，显著提升反射清晰度与一致性。构建了一个包含三个真实场景的新镜面数据集，用于更全面评估。在Mirror-Nerf及自建数据集上的实验表明，本方法在渲染质量上达到或优于当前最优，同时大幅降低训练时间。
原文摘要 · Abstract (English)
Recent advancements in 3D Gaussian Splatting (3D-GS) have revolutionized novel view synthesis, facilitating real-time, high-quality image rendering. However, in scenarios involving reflective surfaces, particularly mirrors, 3D-GS often misinterprets reflections as virtual spaces, resulting in blurred and inconsistent multi-view rendering within mirrors. Our paper presents a novel method aimed at obtaining high-quality multi-view consistent reflection rendering by modelling reflections as physically-based virtual cameras. We estimate mirror planes with depth and normal estimates from 3D-GS and define virtual cameras that are placed symmetrically about the mirror plane. These virtual cameras are then used to explain mirror reflections in the scene. To address imperfections in mirror plane estimates, we propose a straightforward yet effective virtual camera optimization method to enhance reflection quality. We collect a new mirror dataset including three real-world scenarios for more diverse evaluation. Experimental validation on both Mirror-Nerf and our real-world dataset demonstrate the efficacy of our approach. We achieve comparable or superior results while significantly reducing training time compared to previous state-of-the-art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。