arXiv:2412.06424cs.CV2024-12AAAI被引 26

用模糊单目视频重建高质量4D模型,解决运动模糊导致的失真问题。

Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video

  • 将动态表示估计转化为曝光时间估计,提升建模精度。
  • 在合成与真实数据上,四项任务均优于当前最优方法。
  • 适用于去模糊、插帧、视频稳定等多场景应用。

近期4D重建方法虽取得显著成果,但依赖清晰视频作为监督信号。然而,因相机抖动和物体运动常导致视频出现运动模糊,现有方法在使用此类视频重建4D模型时会产生模糊结果。尽管已有少数方法尝试解决该问题,但由于难以准确估计曝光时间内连续动态表征,仍难获得高质量输出。受3D高斯点云(3DGS)在3D运动轨迹建模中的启发,本文采用3DGS作为场景表示方式,提出Deblur4DGS,从模糊单目视频中重建高质量4D模型。具体地,将曝光时间内的连续动态表示估计转化为曝光时间估计,并引入曝光正则项、多帧及多分辨率一致性正则项以避免平凡解。此外,为更好表达大运动物体,提出感知模糊的可变标准高斯分布。除新视角合成外,Deblur4DGS还可用于多角度改善模糊视频,包括去模糊、帧插值与视频稳定。在合成与真实数据上的广泛实验表明,该方法在上述四项任务中均超越现有最先进4D重建方法。代码已开源:https://github.com/ZcsrenlongZ/Deblur4DGS。

原文摘要 · Abstract (English)

Recent 4D reconstruction methods have yielded impressive results but rely on sharp videos as supervision. However, motion blur often occurs in videos due to camera shake and object movement, while existing methods render blurry results when using such videos for reconstructing 4D models. Although a few approaches attempted to address the problem, they struggled to produce high-quality results, due to the inaccuracy in estimating continuous dynamic representations within the exposure time. Encouraged by recent works in 3D motion trajectory modeling using 3D Gaussian Splatting (3DGS), we take 3DGS as the scene representation manner, and propose Deblur4DGS to reconstruct a high-quality 4D model from blurry monocular video. Specifically, we transform continuous dynamic representations estimation within an exposure time into the exposure time estimation. Moreover, we introduce the exposure regularization term, multi-frame, and multi-resolution consistency regularization term to avoid trivial solutions. Furthermore, to better represent objects with large motion, we suggest blur-aware variable canonical Gaussians. Beyond novel-view synthesis, Deblur4DGS can be applied to improve blurry video from multiple perspectives, including deblurring, frame interpolation, and video stabilization. Extensive experiments in both synthetic and real-world data on the above four tasks show that Deblur4DGS outperforms state-of-the-art 4D reconstruction methods. The codes are available at https://github.com/ZcsrenlongZ/Deblur4DGS.

4D重建去模糊视频处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。