让手机实时渲染高质量3D高斯点云,速度与体积双优化。
Mobile-GS: Real-time Gaussian Splatting for Mobile Devices
- 用无序深度渲染替代耗时排序,突破性能瓶颈
- 通过神经视图增强修复重叠区域透明伪影
- 结合量化与剪枝,模型压缩至适合手机部署
3D高斯点云(3DGS)在多种应用中展现出强大的高质量渲染能力,但其高计算需求和大存储开销使其难以部署于移动设备。本文提出面向移动端的实时3D高斯点云方法Mobile-GS,实现边缘设备上的高效推理。首先,识别出阿尔法混合是主要性能瓶颈,因其依赖耗时的高斯深度排序。为此,提出一种深度感知的无序渲染方案,消除排序需求,显著提升渲染速度。尽管该方案加速了渲染,但在重叠几何区域可能因渲染顺序缺失引入透明伪影。为解决此问题,设计一种神经视图依赖增强策略,基于视角方向、3D高斯几何和外观属性更准确建模视图依赖效果。同时,为适应内存受限的移动平台,引入一阶球谐函数蒸馏、神经向量量化及基于贡献的剪枝策略,结合神经网络降低高斯原语数量并压缩3D高斯表示。大量实验表明,Mobile-GS在保持高视觉质量的同时实现实时渲染且模型紧凑,非常适合移动端应用。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as a powerful representation for high-quality rendering across a wide range of applications.However, its high computational demands and large storage costs pose significant challenges for deployment on mobile devices. In this work, we propose a mobile-tailored real-time Gaussian Splatting method, dubbed Mobile-GS, enabling efficient inference of Gaussian Splatting on edge devices. Specifically, we first identify alpha blending as the primary computational bottleneck, since it relies on the time-consuming Gaussian depth sorting process. To solve this issue, we propose a depth-aware order-independent rendering scheme that eliminates the need for sorting, thereby substantially accelerating rendering. Although this order-independent rendering improves rendering speed, it may introduce transparency artifacts in regions with overlapping geometry due to the scarcity of rendering order. To address this problem, we propose a neural view-dependent enhancement strategy, enabling more accurate modeling of view-dependent effects conditioned on viewing direction, 3D Gaussian geometry, and appearance attributes. In this way, Mobile-GS can achieve both high-quality and real-time rendering. Furthermore, to facilitate deployment on memory-constrained mobile platforms, we also introduce first-order spherical harmonics distillation, a neural vector quantization technique, and a contribution-based pruning strategy to reduce the number of Gaussian primitives and compress the 3D Gaussian representation with the assistance of neural networks. Extensive experiments demonstrate that our proposed Mobile-GS achieves real-time rendering and compact model size while preserving high visual quality, making it well-suited for mobile applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。