实时融合语言信息的3D高斯点云建图，速度超10帧/秒
LEG-SLAM: Real-Time Language-Enhanced Gaussian Splatting for SLAM
- 用DINOv2+PCA压缩提取视觉语言特征，实现语义增强
- 在Replica上达10+帧/秒，ScanNet上达18帧/秒
- 无需预处理，适合机器人与AR等实时交互场景
现代高斯点云渲染方法在实时生成逼真3D图像方面表现优异，但如何在保持实时性的同时融入语义信息仍是挑战，尤其在SLAM应用中。本文提出LEG-SLAM，将优化的高斯点云实现与基于DINOv2的视觉-语言特征提取结合，再通过可学习的主成分分析压缩器处理，支持在线稠密SLAM。该方法同步生成高质量逼真图像与语义标注场景图，在Replica数据集上实现超过10帧/秒，在ScanNet上达18帧/秒。实验表明，该方法在重建速度上显著优于现有最优方案，同时保持竞争性渲染质量。系统无需预先准备相机位姿或静态语义地图。其在自主机器人、增强现实等交互领域的应用潜力巨大，是实时语义3D高斯SLAM的重要进展。
原文摘要 · Abstract (English)
Modern Gaussian Splatting methods have proven highly effective for real-time photorealistic rendering of 3D scenes. However, integrating semantic information into this representation remains a significant challenge, especially in maintaining real-time performance for SLAM (Simultaneous Localization and Mapping) applications. In this work, we introduce LEG-SLAM -- a novel approach that fuses an optimized Gaussian Splatting implementation with visual-language feature extraction using DINOv2 followed by a learnable feature compressor based on Principal Component Analysis, while enabling an online dense SLAM. Our method simultaneously generates high-quality photorealistic images and semantically labeled scene maps, achieving real-time scene reconstruction with more than 10 fps on the Replica dataset and 18 fps on ScanNet. Experimental results show that our approach significantly outperforms state-of-the-art methods in reconstruction speed while achieving competitive rendering quality. The proposed system eliminates the need for prior data preparation such as camera's ego motion or pre-computed static semantic maps. With its potential applications in autonomous robotics, augmented reality, and other interactive domains, LEG-SLAM represents a significant step forward in real-time semantic 3D Gaussian-based SLAM. Project page: https://titrom025.github.io/LEG-SLAM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。