用可扩展的高斯点云实现长期沉浸式场景语义补全
TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion
- 构建持久化高斯记忆，通过双时序编码器融合历史与当前特征
- 仅用更少的高斯点实现更高精度，长时场景一致性更强
- 适合需要持续感知的机器人、AR/VR等实时3D重建场景
沉浸式3D语义场景补全（Embodied SSC）从连续的自身视角观测中推断出稠密的几何与语义信息。现有基于高斯的方法通常在预定义空间范围内随机初始化大量原始元素，导致冗余且难以扩展至无界场景。近期的深度引导方法虽缓解此问题，但仍局限于局部范围，随规模增长出现延迟与内存开销上升。为此，本文提出TGSFormer，一种面向沉浸式场景补全的可扩展时序高斯点云框架。该框架维护一个持久化的高斯记忆，无需依赖图像一致性或帧缓存。通过双时序编码器联合处理当前与历史高斯特征，采用置信度感知的交叉注意力机制进行时序融合。随后，置信度感知体素融合模块将重叠的原始元素合并为体素对齐表示，调控密度并保持紧凑性。大量实验表明，TGSFormer在局部与沉浸式场景补全基准上均达到领先性能，在显著减少原始元素数量的同时，保持了更高的精度与更好的可扩展性，并维持长期场景完整性。代码将在论文接收后发布。
原文摘要 · Abstract (English)
Embodied 3D Semantic Scene Completion (SSC) infers dense geometry and semantics from continuous egocentric observations. Most existing Gaussian-based methods rely on random initialization of many primitives within predefined spatial bounds, resulting in redundancy and poor scalability to unbounded scenes. Recent depth-guided approach alleviates this issue but remains local, suffering from latency and memory overhead as scale increases. To overcome these challenges, we propose TGSFormer, a scalable Temporal Gaussian Splatting framework for embodied SSC. It maintains a persistent Gaussian memory for temporal prediction, without relying on image coherence or frame caches. For temporal fusion, a Dual Temporal Encoder jointly processes current and historical Gaussian features through confidence-aware cross-attention. Subsequently, a Confidence-aware Voxel Fusion module merges overlapping primitives into voxel-aligned representations, regulating density and maintaining compactness. Extensive experiments demonstrate that TGSFormer achieves state-of-the-art results on both local and embodied SSC benchmarks, offering superior accuracy and scalability with significantly fewer primitives while maintaining consistent long-term scene integrity. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。