arXiv:2506.06517cs.CV2025-06被引 5

首个可泛化的稀疏高斯泼溅语义SLAM系统,速度快10倍且用10倍少的高斯点。

GS4: Generalizable Sparse Splatting Semantic SLAM

  • 用前馈网络从视频帧预测稀疏高斯参数,统一融合颜色与语义信息
  • 仅需1-5次联合优化即可校正位姿漂移,实现比现有方法快10倍、少10倍高斯点
  • 在多个数据集上零样本迁移表现优异,适合需要快速构建高质量语义地图的场景

传统SLAM算法在相机跟踪上表现良好,但生成的地图通常不完整且分辨率低,且与语义预测集成度不高。近期工作将高斯泼溅(GS)引入SLAM以实现稠密、逼真的3D建图,但现有基于GS的SLAM方法需对每场景进行优化,速度慢且高斯点数量过多。我们提出GS4,首个可泛化的基于GS的语义SLAM系统。相比先前方法,GS4运行速度快10倍,使用的高斯点减少10倍,并在颜色、深度、语义映射和相机跟踪方面达到最先进性能。从RGB-D视频流中,GS4通过前馈网络增量式构建并更新一组3D高斯点。首先,高斯预测模型从输入帧估计稀疏高斯参数,使用同一骨干网络整合颜色与语义预测;其次,高斯精炼网络将新高斯点与现有集合合并,避免冗余;最后,当检测到显著位姿变化时,仅执行1-5次联合高斯-位姿优化以校正漂移、去除浮动点并进一步提升跟踪精度。在真实世界扫描数据集ScanNet和ScanNet++上的实验表明,其达到最先进的语义SLAM性能,零样本迁移至NYUv2和TUM RGB-D数据集也展现出强泛化能力。

原文摘要 · Abstract (English)

Traditional SLAM algorithms excel at camera tracking, but typically produce incomplete and low-resolution maps that are not tightly integrated with semantics prediction. Recent work integrates Gaussian Splatting (GS) into SLAM to enable dense, photorealistic 3D mapping, yet existing GS-based SLAM methods require per-scene optimization that is slow and consumes an excessive number of Gaussians. We present GS4, the first generalizable GS-based semantic SLAM system. Compared with prior approaches, GS4 runs 10x faster, uses 10x fewer Gaussians, and achieves state-of-the-art performance across color, depth, semantic mapping and camera tracking. From an RGB-D video stream, GS4 incrementally builds and updates a set of 3D Gaussians using a feed-forward network. First, the Gaussian Prediction Model estimates a sparse set of Gaussian parameters from input frame, which integrates both color and semantic prediction with the same backbone. Then, the Gaussian Refinement Network merges new Gaussians with the existing set while avoiding redundancy. Finally, when significant pose changes are detected, we perform only 1-5 iterations of joint Gaussian-pose optimization to correct drift, remove floaters, and further improve tracking accuracy. Experiments on the real-world ScanNet and ScanNet++ benchmarks demonstrate state-of-the-art semantic SLAM performance, with strong generalization capability shown through zero-shot transfer to the NYUv2 and TUM RGB-D datasets.

SLAM高斯泼溅语义建图实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。