让3D高斯点云训练速度提升10倍以上,接近二阶收敛。
3DGS$^2$: Near Second-order Converging 3D Gaussian Splatting
- 按高斯核属性分组优化,用小规模牛顿法实现局部快速收敛。
- 训练迭代次数减少10倍以上,图像质量不降反升。
- 适合需要快速重建的科研与工业应用,尤其看重效率的场景。
3D Gaussian Splatting(3DGS)已成为新颖视图合成与三维重建的主流方法。通过显式使用高斯核集合编码三维场景,3DGS实现了高质量渲染和优异效率。作为一种学习方法,3DGS训练通常采用标准随机梯度下降(SGD),最多仅具备线性收敛速度,因此训练常需数十分钟,即使在GPU加速下亦然。本文提出一种(近似)二阶收敛的3DGS训练算法,利用其独特性质。核心思路有二:首先,高斯核属性对图像空间损失贡献独立,支持局部优化。我们按单个核属性分组,为每组参数解析构建小型牛顿系统,并在GPU线程上高效求解,实现单张图像级别的牛顿收敛,无需全局海森矩阵;其次,核在不同输入图像间呈现稀疏且结构化耦合,可利用空间信息抑制随机训练中的过冲现象。本方法相比标准基于GPU的3DGS训练收敛快一个数量级,所需迭代次数超10倍减少,同时保持或超越原有重建质量。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as a mainstream solution for novel view synthesis and 3D reconstruction. By explicitly encoding a 3D scene using a collection of Gaussian kernels, 3DGS achieves high-quality rendering with superior efficiency. As a learning-based approach, 3DGS training has been dealt with the standard stochastic gradient descent (SGD) method, which offers at most linear convergence. Consequently, training often requires tens of minutes, even with GPU acceleration. This paper introduces a (near) second-order convergent training algorithm for 3DGS, leveraging its unique properties. Our approach is inspired by two key observations. First, the attributes of a Gaussian kernel contribute independently to the image-space loss, which endorses isolated and local optimization algorithms. We exploit this by splitting the optimization at the level of individual kernel attributes, analytically constructing small-size Newton systems for each parameter group, and efficiently solving these systems on GPU threads. This achieves Newton-like convergence per training image without relying on the global Hessian. Second, kernels exhibit sparse and structured coupling across input images. This property allows us to effectively utilize spatial information to mitigate overshoot during stochastic training. Our method converges an order faster than standard GPU-based 3DGS training, requiring over $10\times$ fewer iterations while maintaining or surpassing the quality of the compared with the SGD-based 3DGS reconstructions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。