StreamSplat实现流式3D高斯溅射,支持持续更新的实时场景重建。
StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting

- 采用体素对齐因果缓存,按场景几何增长内存,避免随输入变长而爆炸
- 在1024帧长流下仍保持高质量渲染,比固定视图方法更稳定
- 适合在线场景重建,尤其适用于移动设备或连续采集的实时应用
前馈式3D高斯溅射可实现无需每场景优化的高效新视角合成,但现有方法多假设上下文视图固定且联合处理,难以应用于校准视图逐次到达的在线场景。本文提出StreamSplat,一种流式前馈3DGS框架,能增量维护持久的几何基础场景状态,并在每个输入块后解码为可渲染的3D高斯。核心是体素对齐因果缓存(VACC),将历史3D令牌存储于内存受限的体素结构中,使内存增长与探索的场景几何相关,而非流长度。为更好利用历史信息进行因果预测,引入历史投影深度锚定(HPDA)以提供深度引导,以及缓存引导特征注入(CGFI)将缓存潜在证据注入高斯令牌回归。在DL3DV、RealEstate10K和ScanNet上的实验表明,即使在稀疏因果输入下,StreamSplat仍与最先进的前馈3DGS方法竞争,且不依赖未来视图或全场景上下文。更重要的是,其可扩展至256、512和1024帧长流,在固定视图基线因内存不足失效时仍保持性能提升,随观测增加持续改善新视角合成质量。代码将在录用后公开。
原文摘要 · Abstract (English)
Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally. We present \emph{StreamSplat}, a streaming feed-forward 3DGS framework that incrementally maintains a persistent geometry-grounded scene state and decodes it into renderable 3D Gaussians after each input chunk. StreamSplat centers on a \textbf{Voxel-Aligned Causal Cache (VACC)}, which stores historical 3D tokens in a memory-bounded voxel structure so that memory grows with explored scene geometry rather than stream length. To better reuse history during causal prediction, we introduce \textbf{History-Projected Depth Anchoring (HPDA)} to project cached geometry as depth guidance for current cost-volume estimation, and \textbf{Cache-Guided Feature Injection (CGFI)} to inject cached latent evidence into Gaussian-token regression. Experiments on DL3DV, RealEstate10K, and ScanNet show that StreamSplat remains competitive with state-of-the-art feed-forward 3DGS methods under sparse causal inputs, despite not using future views or full-scene context. More importantly, it scales to long input streams with 256, 512, and 1024 views where fixed-view baselines run out of memory, yielding sustained improvements in novel-view synthesis quality as more observations arrive. The code will be made publicly available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。