arXiv:2507.23704cs.CVcs.AI2025-07

用光流优化高斯视频重建,解决动态场景模糊与轨迹不稳问题。

Enhanced Velocity Field Modeling for Gaussian Video Reconstruction

  • 引入光流引导的运动场建模,提升动态区域轨迹精度
  • 在真实数据集上实现超过2.5 dB的PSNR提升,减少动态纹理模糊
  • 适合需要高质量动态场景重建的VR/AR开发者

高保真3D视频重建对虚拟现实与增强现实中的实时动态场景渲染至关重要。基于深度变形网络的3D高斯点阵方法虽已实现近摄影级效果,但在复杂运动与显著尺度变化下,变形网络常过度拟合不规则高斯轨迹,导致视觉质量下降。此外,为静态场景设计的梯度密度化策略难以应对动态内容缺失。为此,我们提出专用于高斯视频重建的流动增强速度场建模方案——FlowGaussian-VR,包含两项核心组件:基于光流优化的速度场渲染(VFR)管道,以及在动态区域自适应调整高斯数量与大小的流辅助密度化(FAD)策略。我们在多个包含挑战性运动场景的真实世界多视角动态重建与新视角合成数据集上验证了模型有效性,不仅实现显著的视觉提升(PSNR提升超2.5 dB),动态纹理更少模糊,且高斯轨迹更规律、可追踪。

原文摘要 · Abstract (English)

High-fidelity 3D video reconstruction is essential for enabling real-time rendering of dynamic scenes with realistic motion in virtual and augmented reality (VR/AR). The deformation field paradigm of 3D Gaussian splatting has achieved near-photorealistic results in video reconstruction due to the great representation capability of deep deformation networks. However, in videos with complex motion and significant scale variations, deformation networks often overfit to irregular Gaussian trajectories, leading to suboptimal visual quality. Moreover, the gradient-based densification strategy designed for static scene reconstruction proves inadequate to address the absence of dynamic content. In light of these challenges, we propose a flow-empowered velocity field modeling scheme tailored for Gaussian video reconstruction, dubbed FlowGaussian-VR. It consists of two core components: a velocity field rendering (VFR) pipeline which enables optical flow-based optimization, and a flow-assisted adaptive densification (FAD) strategy that adjusts the number and size of Gaussians in dynamic regions. We validate our model's effectiveness on multi-view dynamic reconstruction and novel view synthesis with multiple real-world datasets containing challenging motion scenarios, demonstrating not only notable visual improvements (over 2.5 dB gain in PSNR) and less blurry artifacts in dynamic textures, but also regularized and trackable per-Gaussian trajectories.

视频重建高斯点阵光流VR/AR

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。