用可训练的代码本压缩3D高斯点云,大幅降低显存占用。
ContraGS: Codebook-Condensed and Trainable Gaussian Splatting for Fast, Memory-Efficient Reconstruction
- 用代码本动态存储高斯参数,训练时直接压缩
- 训练峰值显存降低3.49倍,速度提升1.36倍
- 适合资源有限但需高质量重建的场景
3D高斯点云(3DGS)是实现高质量实时渲染的先进方法。通常需要大量高斯点以获得高保真度,但会显著增加显存消耗。大模型依赖高性能显卡,且训练/渲染延迟高。本文提出ContraGS,可在不减少高斯点数量的前提下直接对压缩表示进行训练,仅轻微损失质量。该方法利用代码本紧凑存储高斯参数向量,大幅降低内存开销。尽管代码本在训练后压缩中效果显著,但直接在压缩表示上训练仍是未解难题。ContraGS通过将参数估计建模为贝叶斯推断问题,采用MCMC采样从后验分布中抽样,解决了代码本不可微参数的学习问题。实验表明,与基线相比,ContraGS平均降低3.49倍峰值显存,训练加速1.36倍,渲染加速1.88倍,同时保持接近最先进水平的重建质量。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) is a state-of-art technique to model real-world scenes with high quality and real-time rendering. Typically, a higher quality representation can be achieved by using a large number of 3D Gaussians. However, using large 3D Gaussian counts significantly increases the GPU device memory for storing model parameters. A large model thus requires powerful GPUs with high memory capacities for training and has slower training/rendering latencies due to the inefficiencies of memory access and data movement. In this work, we introduce ContraGS, a method to enable training directly on compressed 3DGS representations without reducing the Gaussian Counts, and thus with a little loss in model quality. ContraGS leverages codebooks to compactly store a set of Gaussian parameter vectors throughout the training process, thereby significantly reducing memory consumption. While codebooks have been demonstrated to be highly effective at compressing fully trained 3DGS models, directly training using codebook representations is an unsolved challenge. ContraGS solves the problem of learning non-differentiable parameters in codebook-compressed representations by posing parameter estimation as a Bayesian inference problem. To this end, ContraGS provides a framework that effectively uses MCMC sampling to sample over a posterior distribution of these compressed representations. With ContraGS, we demonstrate that ContraGS significantly reduces the peak memory during training (on average 3.49X) and accelerated training and rendering (1.36X and 1.88X on average, respectively), while retraining close to state-of-art quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。