用小型神经网络压缩3D高斯点的外观表示,显著降低内存占用并提升渲染效率。
Compact Neural Appearance Models for Efficient Gaussian Splatting

- 用共享小MLP解码每点的紧凑隐向量,替代传统球谐函数。
- 相比三阶球谐函数,每点存储从192字节降至28字节,优化速度提升1.3倍。
- 适合追求高效渲染与低内存占用的实时3D重建应用。
显式基元辐射场如3D高斯点阵通常使用低阶球谐函数(SH)建模视角相关外观。尽管计算高效,但SH系数占用了大量每基元存储与内存流量,且其带限基底限制了角度细节。本文系统比较了SH与近期球面外观模型,并提出一种隐式替代方案:通过微型共享MLP解码紧凑的每基元隐代码。我们将所有模型集成至同一优化流水线,融合前向与反向传播至可微分的CUDA光栅器,并提供适用于笔记本与移动GPU的便携式WebGL查看器。评估显示,近期球面模型在重建质量、内存占用、优化与渲染性能上表现最优。我们的神经表示是所评估中最紧凑的,相比三阶SH,每基元外观占用从192字节降至28字节,优化加速1.3倍,同时提升重建质量。我们进一步分析外观参数化对优化的影响,揭示不同模型恢复几何的差异及高表达模型吸收非静态内容的倾向。整体框架与分析为超越图像指标的SH替代提供了实用指导。
原文摘要 · Abstract (English)
Explicit primitive-based radiance fields such as 3D Gaussian Splatting typically model view-dependent appearance using low-order spherical harmonics (SH). Although efficient to evaluate, SH coefficients dominate per-primitive storage and memory traffic, while their band-limited basis restricts angular detail. We present a thorough, end-to-end comparison of SH and recent spherical appearance models and introduce an implicit alternative that decodes compact per-primitive latent codes using a tiny shared MLP. We integrate all models into the same optimized pipeline, fusing their forward and backward passes into a differentiable CUDA rasterizer and provide a portable WebGL viewer for laptop and mobile GPUs. Our evaluation across reconstruction quality, memory use, and optimization and rendering performance shows that recent spherical models offer the strongest overall quality-efficiency trade-off. Our neural representation is the most compact model evaluated and, compared to third-degree SH, reduces the per-primitive appearance footprint from 192 to 28 bytes, accelerates optimization by 1.3$\times$, while improving reconstruction quality. We further analyze how appearance parametrization shapes optimization, identifying differences in recovered geometry and the tendency of expressive models to absorb non-static scene content. Together, our framework and analysis provide practical guidance for replacing SH beyond what image metrics alone can capture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。