让3D场景重建模型在手机级设备上持续学习,内存降低88%。
Improving Continual Learning for Gaussian Splatting based Environments Reconstruction on Commercial Off-the-Shelf Edge Devices
- 按内存和延迟热点优化计算流程,减少中间数据占用
- 自动分配精度,在保证质量前提下将训练时间缩短74%
- 首次实现在嵌入式设备上的实时3D重建训练
新视角合成(NVS)在边缘机器人中日益重要,需在有限内存与延迟下实现紧凑且可增量更新的3D场景建模。变分贝叶斯高斯点云(VBGS)通过概率建模实现无重放持续更新,但其高精度计算与大中间张量导致设备端训练不可行。本文提出一种精度自适应优化框架,保持原变分形式不变,通过(i)分析VBGS性能瓶颈,(ii)融合内存密集型算子以减少中间张量,(iii)基于相对误差约束的混合精度搜索自动分配操作精度。在Blender、Habitat和Replica数据集上,峰值内存从9.44 GB降至1.11 GB,训练时间由约234分钟缩减至约61分钟,同时维持甚至提升现有最优VBGS重建质量。首次在商业嵌入式平台Jetson Orin Nano上实现NVS训练,单帧延迟降低19倍,相较3DGS。
原文摘要 · Abstract (English)
Novel view synthesis (NVS) is increasingly relevant for edge robotics, where compact and incrementally updatable 3D scene models are needed for SLAM, navigation, and inspection under tight memory and latency budgets. Variational Bayesian Gaussian Splatting (VBGS) enables replay-free continual updates for the 3DGS algorithm by maintaining a probabilistic scene model, but its high-precision computations and large intermediate tensors make on-device training impractical. We present a precision-adaptive optimization framework that enables VBGS training on resource-constrained hardware without altering its variational formulation. We (i) profile VBGS to identify memory/latency hotspots, (ii) fuse memory-dominant kernels to reduce materialized intermediate tensors, and (iii) automatically assign operation-level precisions via a mixed-precision search with bounded relative error. Across the Blender, Habitat, and Replica datasets, our optimised pipeline reduces peak memory from 9.44 GB to 1.11 GB and training time from ~234 min to ~61 min on an A5000 GPU, while preserving (and in some cases improving) reconstruction quality of the state-of-the-art VBGS baseline. We also enable for the first time NVS training on a commercial embedded platform, the Jetson Orin Nano, reducing per-frame latency by 19x compared to 3DGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。