arXiv:2410.20686cs.CV2024-10NeurIPS被引 23

用新投影方法实现全景图3D重建,速度比传统方法快100倍。

ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings

  • 设计切平面投影机制,解决全景图畸变问题。
  • 训练与渲染速度比NeRF快100倍,重建质量更优。
  • 适合需要快速高精度3D重建的场景应用。

全景(360度)图像因可单图渲染完整场景,正广泛用于3D应用。现有基于神经辐射场的方法在第一人称视频上表现良好,但训练和渲染耗时长。近期3D高斯溅射因其快速优化和实时渲染受到关注。然而,直接使用透视投影器处理全景图像会因两种图像域光学特性差异导致严重畸变。本文提出ODGS,一种针对全景图像的新式光栅化管线,具有几何解释性。对每个高斯,定义一个与单位球相切且垂直于指向高斯中心射线的切平面,再用透视相机投影器将高斯投射到对应切平面。投影后的高斯经变换与合并生成全景图像,完成光栅化。该方法揭示了隐含假设,通过数学证明验证。整个过程使用CUDA并行化,实现优化与渲染速度比基于NeRF的方法快100倍。全面实验表明,ODGS在多个数据集上均取得最佳重建与感知质量。漫游数据集结果还显示,即使在大场景下也能有效还原细节。源代码见项目页面(https://github.com/esw0116/ODGS)。

原文摘要 · Abstract (English)

Omnidirectional (or 360-degree) images are increasingly being used for 3D applications since they allow the rendering of an entire scene with a single image. Existing works based on neural radiance fields demonstrate successful 3D reconstruction quality on egocentric videos, yet they suffer from long training and rendering times. Recently, 3D Gaussian splatting has gained attention for its fast optimization and real-time rendering. However, directly using a perspective rasterizer to omnidirectional images results in severe distortion due to the different optical properties between two image domains. In this work, we present ODGS, a novel rasterization pipeline for omnidirectional images, with geometric interpretation. For each Gaussian, we define a tangent plane that touches the unit sphere and is perpendicular to the ray headed toward the Gaussian center. We then leverage a perspective camera rasterizer to project the Gaussian onto the corresponding tangent plane. The projected Gaussians are transformed and combined into the omnidirectional image, finalizing the omnidirectional rasterization process. This interpretation reveals the implicit assumptions within the proposed pipeline, which we verify through mathematical proofs. The entire rasterization process is parallelized using CUDA, achieving optimization and rendering speeds 100 times faster than NeRF-based methods. Our comprehensive experiments highlight the superiority of ODGS by delivering the best reconstruction and perceptual quality across various datasets. Additionally, results on roaming datasets demonstrate that ODGS restores fine details effectively, even when reconstructing large 3D scenes. The source code is available on our project page (https://github.com/esw0116/ODGS).

3D重建全景图高斯溅射加速渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。