实时重建动态场景，2秒内完成每帧，误差更小。
Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting
- 用锚点驱动3D高斯运动，单次推理生成帧间变化。
- 关键帧引导策略使每帧重建时间平均仅2秒以上。
- 适合需要快速响应的动态场景视频生成应用。
以流式方式构建自由视角视频相比离线训练方法具有更快的响应速度，显著提升用户体验。然而，现有流式方法存在每帧重建时间长（超过10秒）和误差累积问题，限制了其广泛应用。本文提出 Instant Gaussian Stream（IGS），一种快速且通用的流式动态场景重建框架。首先，提出广义的锚点驱动高斯运动网络，将多视角2D运动特征投影至3D空间，利用锚点驱动所有高斯点的运动，实现单次推理即可生成目标帧的高斯运动。其次，提出关键帧引导流式策略，对每个关键帧进行精炼，有效实现复杂时序场景的准确重建并缓解误差累积。我们在多个域内与跨域场景上进行了广泛评估，结果表明该方法可实现平均每帧重建时间低于2秒的流式处理，同时显著提升视图合成质量。
原文摘要 · Abstract (English)
Building Free-Viewpoint Videos in a streaming manner offers the advantage of rapid responsiveness compared to offline training methods, greatly enhancing user experience. However, current streaming approaches face challenges of high per-frame reconstruction time (10s+) and error accumulation, limiting their broader application. In this paper, we propose Instant Gaussian Stream (IGS), a fast and generalizable streaming framework, to address these issues. First, we introduce a generalized Anchor-driven Gaussian Motion Network, which projects multi-view 2D motion features into 3D space, using anchor points to drive the motion of all Gaussians. This generalized Network generates the motion of Gaussians for each target frame in the time required for a single inference. Second, we propose a Key-frame-guided Streaming Strategy that refines each key frame, enabling accurate reconstruction of temporally complex scenes while mitigating error accumulation. We conducted extensive in-domain and cross-domain evaluations, demonstrating that our approach can achieve streaming with a average per-frame reconstruction time of 2s+, alongside a enhancement in view synthesis quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。