让手机在有限内存和时间下,实现高质量3D场景的实时建模。
PocketGS: On-Device Training of 3D Gaussian Splatting for High Perceptual Modeling
- 设计三类协同优化操作,平衡训练效率与建模精度。
- 在移动端实现媲美工作站的3D高保真重建效果。
- 适合移动设备端实时3D内容创作与个性化建模。
尽管3D高斯溅射(3DGS)可实现实时渲染,其训练需工作站级算力与内存,在分钟级时间预算和有限峰值内存约束下,难以在移动端部署。本文提出PocketGS,一种面向移动端的3DGS训练范式,在严苛资源限制下仍保持高保真重建。通过三个协同设计的操作:$$\mathcal{G}$$ 构建几何忠实的点云先验;$$\mathcal{I}$$ 注入局部表面统计信息以生成各向异性高斯,缩小早期条件差距;$$\mathcal{T}$$ 采用缓存中间结果的alpha合成展开与索引映射梯度散射,确保移动端反向传播稳定。大量实验表明,PocketGS在移动端预算下超越主流工作站3DGS基线,实现高质量重建,支持端到端的实用化捕获-渲染工作流。
原文摘要 · Abstract (English)
While 3D Gaussian Splatting (3DGS) enables real-time rendering, its training demands workstation-level compute and memory, making mobile deployment impractical under minute-scale time budgets and limited peak memory. We present PocketGS, a mobile scene modeling paradigm that enables on-device 3DGS training under these tightly coupled constraints while preserving high-fidelity reconstruction. PocketGS resolves the fundamental tension between training efficiency, memory compactness, and modeling quality through three co-designed operators: $\mathcal{G}$ builds geometry-faithful point-cloud priors; $\mathcal{I}$ injects local surface statistics to seed anisotropic Gaussians, thereby reducing early conditioning gaps; and $\mathcal{T}$ unrolls alpha compositing with cached intermediates and index-mapped gradient scattering for stable mobile backpropagation. Extensive experiments demonstrate that PocketGS outperforms the powerful mainstream workstation 3DGS baseline under mobile budgets, delivering high-quality reconstructions and enabling a fully on-device, practical capture-to-rendering workflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。