通过代理模型提升3D高斯点云的遮挡感知,兼顾训练与渲染效率。
Proxy-GS: Unified Occlusion Priors for Training and Inference in Structured 3D Gaussian Splatting
- 用1毫秒内生成1000x1000遮挡深度图的代理系统,实现多视角遮挡感知。
- 在MatrixCity Streets数据集上实现2.5倍以上速度提升,同时显著提高渲染质量。
- 适合追求高效高保真3D渲染的场景,尤其适用于复杂遮挡环境。
3D高斯点云(3DGS)已成为实现照片级渲染的高效方法。近期基于MLP的变体进一步提升了视觉保真度,但渲染时带来显著解码开销。为缓解计算负担,已有剪枝策略和细节层级(LOD)技术用于减少大规模场景中的高斯素数量。然而,我们分析发现,因缺乏遮挡感知,仍存在大量冗余。本文提出Proxy-GS,一种利用代理实现任意视角遮挡感知的新流程。核心是快速代理系统,可在1毫秒内生成1000×1000分辨率的精确遮挡深度图。该代理兼具双重作用:其一,指导锚点与高斯素剔除,加速渲染;其二,引导训练阶段向表面密集化,避免遮挡区域不一致,提升渲染质量。在高度遮挡场景(如MatrixCity Streets数据集)中,Proxy-GS不仅赋予基于MLP的高斯点云更强渲染能力,且实现更快速度,相较Octree-GS提速超2.5倍,并持续提供更高渲染质量。代码将在论文接收后公开。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as an efficient approach for achieving photorealistic rendering. Recent MLP-based variants further improve visual fidelity but introduce substantial decoding overhead during rendering. To alleviate computation cost, several pruning strategies and level-of-detail (LOD) techniques have been introduced, aiming to effectively reduce the number of Gaussian primitives in large-scale scenes. However, our analysis reveals that significant redundancy still remains due to the lack of occlusion awareness. In this work, we propose Proxy-GS, a novel pipeline that exploits a proxy to introduce Gaussian occlusion awareness from any view. At the core of our approach is a fast proxy system capable of producing precise occlusion depth maps at a resolution of 1000x1000 under 1ms. This proxy serves two roles: first, it guides the culling of anchors and Gaussians to accelerate rendering speed. Second, it guides the densification towards surfaces during training, avoiding inconsistencies in occluded regions, and improving the rendering quality. In heavily occluded scenarios, such as the MatrixCity Streets dataset, Proxy-GS not only equips MLP-based Gaussian splatting with stronger rendering capability but also achieves faster rendering speed. Specifically, it achieves more than 2.5x speedup over Octree-GS, and consistently delivers substantially higher rendering quality. Code will be public upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。