实时处理任意长度视频，在线重建动态3D场景
StreamSplat: Towards Online Dynamic 3D Reconstruction from Uncalibrated Video Streams
- 基于概率采样与双向变形场，实现无标定视频的在线3D高斯点云生成
- 在标准数据集上达到顶尖重建质量，相比传统方法提速1200倍
- 适合需要低延迟、高内存效率的实时3D重建应用
从无标定视频流中实时重建动态3D场景，需在严格延迟和内存约束下，从稀疏观测中恢复场景动态。现有方法多依赖全序列优化，耗时数小时，难以部署。本文提出StreamSplat，一种全前馈框架，可即时将任意长度的未校准视频流转化为动态3D高斯点阵（3DGS）表示。核心技术包括：1）概率采样机制，从无标定输入中稳健预测3D高斯分布；2）双向变形场，实现帧间可靠关联并抑制长期误差累积；3）自适应高斯融合操作，持续追踪稳定点，处理新出现与消失的点。在标准动态与静态基准测试中，StreamSplat实现顶尖重建质量与动态建模能力。独特优势在于支持任意长视频流的在线重建，相比优化方法提速1200倍。代码与模型见https://streamsplat3d.github.io/。
原文摘要 · Abstract (English)
Real-time reconstruction of dynamic 3D scenes from uncalibrated video streams demands robust online methods that recover scene dynamics from sparse observations under strict latency and memory constraints. Yet most dynamic reconstruction methods rely on hours of per-scene optimization under full-sequence access, limiting practical deployment. In this work, we introduce StreamSplat, a fully feed-forward framework that instantly transforms uncalibrated video streams of arbitrary length into dynamic 3D Gaussian Splatting (3DGS) representations in an online manner. It is achieved via three key technical innovations: 1) a probabilistic sampling mechanism that robustly predicts 3D Gaussians from uncalibrated inputs; 2) a bidirectional deformation field that yields reliable associations across frames and mitigates long-term error accumulation; 3) an adaptive Gaussian fusion operation that propagates persistent Gaussians while handling emerging and vanishing ones. Extensive experiments on standard dynamic and static benchmarks demonstrate that StreamSplat achieves state-of-the-art reconstruction quality and dynamic scene modeling. Uniquely, our method supports the online reconstruction of arbitrarily long video streams with a 1200x speedup over optimization-based methods. Our code and models are available at https://streamsplat3d.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。