arXiv:2505.16533cs.CV2025-05NeurIPS被引 6

通过关键点追踪运动,实现视频重建的超低存储占用。

Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction

  • 用关键点表示物体运动,动态传播到周围点
  • 存储压缩超159倍,视觉质量与速度不降
  • 适合实时自由视角视频传输与渲染

3D高斯点云(3DGS)已成为一种高保真、高效的在线自由视角视频(FVV)重建范式,提供快速响应与沉浸体验。然而,现有方法因逐点建模无法利用运动特性,导致存储开销巨大。为此,我们提出一种新型紧凑高斯流框架ComGS,利用动态场景中运动的局部性与一致性,通过关键点驱动的方式建模对象一致的高斯点运动。仅传输关键点属性即可实现高效建模。首先,基于视图空间梯度差策略,在运动区域中定位稀疏的敏感关键点;随后,设计自适应运动驱动机制,通过预测空间影响场,将关键点运动传播至邻近运动一致的高斯点;此外,采用误差感知校正策略,在关键帧重建中选择性修正错误区域,有效抑制误差累积且无额外开销。整体上,ComGS相比3DGStream实现超过159倍的存储压缩,相较当前最优方法QUEEN减少14倍,同时保持优异的视觉保真度与渲染速度。

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) has emerged as a high-fidelity and efficient paradigm for online free-viewpoint video (FVV) reconstruction, offering viewers rapid responsiveness and immersive experiences. However, existing online methods face challenge in prohibitive storage requirements primarily due to point-wise modeling that fails to exploit the motion properties. To address this limitation, we propose a novel Compact Gaussian Streaming (ComGS) framework, leveraging the locality and consistency of motion in dynamic scene, that models object-consistent Gaussian point motion through keypoint-driven motion representation. By transmitting only the keypoint attributes, this framework provides a more storage-efficient solution. Specifically, we first identify a sparse set of motion-sensitive keypoints localized within motion regions using a viewspace gradient difference strategy. Equipped with these keypoints, we propose an adaptive motion-driven mechanism that predicts a spatial influence field for propagating keypoint motion to neighboring Gaussian points with similar motion. Moreover, ComGS adopts an error-aware correction strategy for key frame reconstruction that selectively refines erroneous regions and mitigates error accumulation without unnecessary overhead. Overall, ComGS achieves a remarkable storage reduction of over 159 X compared to 3DGStream and 14 X compared to the SOTA method QUEEN, while maintaining competitive visual fidelity and rendering speed.

3DGS视频重建运动建模压缩传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。