用更灵活的3D Gabor基元替代高斯函数,提升3D场景细节渲染质量与效率。
3DGabSplat: 3D Gabor Splatting for Frequency-adaptive Radiance Field Rendering
- 采用多方向频率响应的3D Gabor基元构建滤波器组,增强高频细节表达能力。
- 在真实与合成场景中实现最高1.35 dB PSNR提升,同时减少70%以上基元数量和内存占用。
- 支持即插即用,适用于现有3DGS框架,适合需要高质量实时渲染的研究者。
3D高斯点阵(3DGS)虽实现了实时渲染与高保真新视角生成,但其固有的低通特性限制了对3D场景高频细节的表达,导致冗余基元、训练与渲染效率下降及内存开销过大。为此,本文提出3D Gabor点阵(3DGabSplat),采用一种新型3D Gabor基元,具备多方向3D频率响应,通过多视图图像监督进行辐射场建模。该基元构成包含多个不同频率的3D Gabor核的滤波器组,显著提升捕捉精细3D结构的灵活性与效率。进一步设计高效的CUDA光栅化器,将3D Gabor基元的多方向频率分量投影至2D图像平面,并引入频率自适应机制实现基元的联合优化。3DGabSplat可无缝集成至现有3DGS框架中,实现更高效率与质量的新视角渲染。大量实验表明,3DGabSplat优于3DGS及其使用其他基元的变体,在真实世界与合成场景中均达到当前最优渲染效果;相比3DGS,PSNR最高提升1.35 dB,同时基元数量与内存消耗显著降低。
原文摘要 · Abstract (English)
Recent prominence in 3D Gaussian Splatting (3DGS) has enabled real-time rendering while maintaining high-fidelity novel view synthesis. However, 3DGS resorts to the Gaussian function that is low-pass by nature and is restricted in representing high-frequency details in 3D scenes. Moreover, it causes redundant primitives with degraded training and rendering efficiency and excessive memory overhead. To overcome these limitations, we propose 3D Gabor Splatting (3DGabSplat) that leverages a novel 3D Gabor-based primitive with multiple directional 3D frequency responses for radiance field representation supervised by multi-view images. The proposed 3D Gabor-based primitive forms a filter bank incorporating multiple 3D Gabor kernels at different frequencies to enhance flexibility and efficiency in capturing fine 3D details. Furthermore, to achieve novel view rendering, an efficient CUDA-based rasterizer is developed to project the multiple directional 3D frequency components characterized by 3D Gabor-based primitives onto the 2D image plane, and a frequency-adaptive mechanism is presented for adaptive joint optimization of primitives. 3DGabSplat is scalable to be a plug-and-play kernel for seamless integration into existing 3DGS paradigms to enhance both efficiency and quality of novel view synthesis. Extensive experiments demonstrate that 3DGabSplat outperforms 3DGS and its variants using alternative primitives, and achieves state-of-the-art rendering quality across both real-world and synthetic scenes. Remarkably, we achieve up to 1.35 dB PSNR gain over 3DGS with simultaneously reduced number of primitives and memory consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。