提升3D高斯点云渲染效率，实现多视角联合训练
Efficient multi-view training for 3D Gaussian Splatting
- 改进渲染流程，降低多视角训练开销
- 引入距离感知的D-SSIM损失，提升图像一致性
- 适配多视角的自适应密度控制，适合大规模场景
3D高斯点云（3DGS）因渲染速度优势，已成为逆向渲染领域的热门选择。当前主流采用单视角小批量训练，每轮仅处理一张图像，而NeRF则使用多视角训练。我们发现单视角训练因小批量梯度方差大，导致优化效果不佳，亟需多视角训练。然而，直接在3DGS中进行多视角训练存在显著开销，且现有方法依赖单视角假设，影响高斯点密度优化。为此，我们改进光栅化流程以减少开销，并提出距离感知的D-SSIM损失与多视角自适应密度控制机制，更适配多视角场景。实验表明，所提方法显著提升3DGS及其变体性能，使其摆脱单视角训练限制。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as a preferred choice alongside Neural Radiance Fields (NeRF) in inverse rendering due to its superior rendering speed. Currently, the common approach in 3DGS is to utilize "single-view" mini-batch training, where only one image is processed per iteration, in contrast to NeRF's "multi-view" mini-batch training, which leverages multiple images. We observe that such single-view training can lead to suboptimal optimization due to increased variance in mini-batch stochastic gradients, highlighting the necessity for multi-view training. However, implementing multi-view training in 3DGS poses challenges. Simply rendering multiple images per iteration incurs considerable overhead and may result in suboptimal Gaussian densification due to its reliance on single-view assumptions. To address these issues, we modify the rasterization process to minimize the overhead associated with multi-view training and propose a 3D distance-aware D-SSIM loss and multi-view adaptive density control that better suits multi-view scenarios. Our experiments demonstrate that the proposed methods significantly enhance the performance of 3DGS and its variants, freeing 3DGS from the constraints of single-view training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。