通过动态增删高斯点,实现低内存占用的3D高斯溅射训练。
Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training

- 训练中交替进行低效点删除与自适应新增,控制内存波动。
- 在严苛内存限制下,峰值内存降低80%仍保持高质量渲染。
- 适合边缘设备部署,如Jetson AGX Xavier等资源受限场景。
3D高斯溅射(3DGS)通过数百万个3D高斯原语的连续聚合,实现了高质量的新视角合成。然而,其训练过程存在显著的内存占用问题,尤其因未受控的密集化导致内存峰值过高,严重制约了在内存受限的边缘设备上的部署。现有方法虽可在训练后剪枝冗余高斯点,但无法缓解训练初期高斯点激增带来的内存高峰。为此,本文提出一种系统性的内存约束训练框架,通过迭代式增长与剪枝动态优化高斯点分布。该框架在增量剪枝低贡献点的同时,结合自适应补偿机制策略性地生成新原语,在持续提升渲染质量的同时维持近似恒定的低内存使用。我们在多个真实世界数据集上进行了全面评估,结果表明,相比现有最先进方法有显著提升。特别地,本方法首次使3DGS在NVIDIA Jetson AGX Xavier上实现高效内存训练,峰值内存消耗降低高达80%,同时保持相近的视觉质量。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it suffers from a substantial memory footprint, particularly during training due to uncontrolled densification, posing a critical bottleneck for deployment on memory-constrained edge devices. While existing methods prune redundant Gaussians post-training, they fail to address the peak memory spikes caused by the abrupt growth of Gaussians early in the training process. To solve the training memory consumption problem, we propose a systematic memory-bounded training framework that dynamically optimizes Gaussians through iterative growth and pruning. In other words, the proposed framework alternates between incremental pruning of low-impact Gaussians and strategic growing of new primitives with an adaptive Gaussian compensation, maintaining a near-constant low memory usage while progressively refining rendering fidelity. We comprehensively evaluate the proposed training framework on various real-world datasets under strict memory constraints, showing significant improvements over existing state-of-the-art methods. Particularly, our proposed method practically enables memory-efficient 3DGS training on NVIDIA Jetson AGX Xavier, achieving similar visual quality with up to 80% lower peak training memory consumption than the original 3DGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。