为鱼眼相机优化3D高斯点云渲染，提升画质与效率。
Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras
- 重新设计鱼眼相机的投影变换及梯度计算
- 相比去畸变后训练，视觉质量明显提升
- 模块化轻量设计，可无缝接入其他渲染系统
最近，3D高斯点云（3DGS）因其高保真和实时渲染能力受到关注。然而，将3DGS适配到不同相机模型，特别是鱼眼镜头，面临挑战，主要源于独特的3D到2D投影计算。此外，基于区块的点云绘制在鱼眼镜头的极端曲率和宽视场下存在效率问题，制约了其在真实场景中的广泛应用。为此，我们提出Fisheye-GS。该方法重新计算鱼眼相机的投影变换及其梯度，可作为模块无缝集成至其他高效3D渲染方法中，具有良好的扩展性、轻量化和模块化设计。由于仅修改投影部分，该方法也可轻松适配其他相机模型。相比在去畸变后训练的方法，我们的方案在视觉质量上表现出明显优势。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3DGS) has garnered attention for its high fidelity and real-time rendering. However, adapting 3DGS to different camera models, particularly fisheye lenses, poses challenges due to the unique 3D to 2D projection calculation. Additionally, there are inefficiencies in the tile-based splatting, especially for the extreme curvature and wide field of view of fisheye lenses, which are crucial for its broader real-life applications. To tackle these challenges, we introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods, emphasizing its extensibility, lightweight nature, and modular design. Since we only modified the projection component, it can also be easily adapted for use with different camera models. Compared to methods that train after undistortion, our approach demonstrates a clear improvement in visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。