让3D高斯点云实时处理长序列视频,内存占用降低44%。
LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images
- 用图像化表示法动态更新点云,边进边压缩冗余数据。
- 实时重建下高斯点数减少44%,画质接近最优水平。
- 适合需要持续建模的自动驾驶、机器人视觉场景。
3D高斯点云渲染能实现高保真新视角合成,但在在线长序列场景中应用受限。现有方法或依赖慢速逐场景优化,或无法高效增量更新,难以持续保持性能。本文提出LongSplat,一个针对长序列图像输入的在线实时3D高斯重建框架。核心是流式更新机制:在融合当前视图观测的同时,选择性压缩历史冗余高斯点。关键在于提出的高斯-图像表征(GIR),将3D高斯参数编码为结构化的2D图像格式,实现当前与历史高斯的高效融合及身份感知的冗余压缩。该机制使模型可在不显著增加内存与计算开销的前提下适应长序列。此外,借助已有图像压缩方法指导生成更紧凑、更高品质的3D高斯点。大量实验表明,LongSplat在实时新视角合成中达到顶尖的效率-质量平衡,实现实时重建的同时,相比现有逐像素高斯预测方法,高斯数量减少44%。
原文摘要 · Abstract (English)
3D Gaussian Splatting achieves high-fidelity novel view synthesis, but its application to online long-sequence scenarios is still limited. Existing methods either rely on slow per-scene optimization or fail to provide efficient incremental updates, hindering continuous performance. In this paper, we propose LongSplat, an online real-time 3D Gaussian reconstruction framework designed for long-sequence image input. The core idea is a streaming update mechanism that incrementally integrates current-view observations while selectively compressing redundant historical Gaussians. Crucial to this mechanism is our Gaussian-Image Representation (GIR), a representation that encodes 3D Gaussian parameters into a structured, image-like 2D format. GIR simultaneously enables efficient fusion of current-view and historical Gaussians and identity-aware redundancy compression. These functions enable online reconstruction and adapt the model to long sequences without overwhelming memory or computational costs. Furthermore, we leverage an existing image compression method to guide the generation of more compact and higher-quality 3D Gaussians. Extensive evaluations demonstrate that LongSplat achieves state-of-the-art efficiency-quality trade-offs in real-time novel view synthesis, delivering real-time reconstruction while reducing Gaussian counts by 44\% compared to existing per-pixel Gaussian prediction methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。