用动量自蒸馏提升大场景重建质量,减少块数依赖显卡数量
Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction
- 通过动量教师网络为每个块提供全局一致性指导
- 在更少分块下实现12.8%的LPIPS提升,优于现有方法
- 适合大规模3D场景重建,尤其适用于显卡资源有限的场景
3D高斯点阵在大场景重建中表现优异,但面临训练内存消耗高和存储开销大的挑战。混合表示结合隐式与显式特征可缓解此问题,但在并行分块训练时出现两个关键问题:独立训练各块导致数据多样性下降,重建精度降低;并行训练受限于可用GPU数量,无法自由划分块数。为此,我们提出Momentum-GS,一种基于动量自蒸馏的新方法,通过维护一个带动量更新的教师高斯解码器,在训练过程中提供稳定全局参考。该教师网络以自蒸馏方式为各块提供全局引导,增强空间一致性。为进一步确保块间一致性,引入块权重机制,根据各块重建精度动态调整权重。大量实验表明,本方法在大场景上持续优于现有技术,在较少分块条件下较CityGaussian提升12.8% LPIPS,建立新基准。项目页:https://jixuan-fan.github.io/Momentum-GS_Page/
原文摘要 · Abstract (English)
3D Gaussian Splatting has demonstrated notable success in large-scale scene reconstruction, but challenges persist due to high training memory consumption and storage overhead. Hybrid representations that integrate implicit and explicit features offer a way to mitigate these limitations. However, when applied in parallelized block-wise training, two critical issues arise since reconstruction accuracy deteriorates due to reduced data diversity when training each block independently, and parallel training restricts the number of divided blocks to the available number of GPUs. To address these issues, we propose Momentum-GS, a novel approach that leverages momentum-based self-distillation to promote consistency and accuracy across the blocks while decoupling the number of blocks from the physical GPU count. Our method maintains a teacher Gaussian decoder updated with momentum, ensuring a stable reference during training. This teacher provides each block with global guidance in a self-distillation manner, promoting spatial consistency in reconstruction. To further ensure consistency across the blocks, we incorporate block weighting, dynamically adjusting each block's weight according to its reconstruction accuracy. Extensive experiments on large-scale scenes show that our method consistently outperforms existing techniques, achieving a 12.8% improvement in LPIPS over CityGaussian with much fewer divided blocks and establishing a new state of the art. Project page: https://jixuan-fan.github.io/Momentum-GS_Page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。