统一处理模糊与运动模糊视频，实现高质量动态点云渲染。
Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos
- 用神经网络预测每像素模糊核，结合场景与相机信息增强准确性。
- 在模糊视频上生成逼真新视角图像，超越当前最优方法。
- 适合需要从低质量单目视频重建3D场景的研究者和开发者。
本文提出一种统一框架，可从模糊和运动模糊的单目视频中生成高质量动态高斯点云。由于散焦模糊与运动模糊形成机制差异显著，现有方法通常仅针对其中一类，难以同时处理两者。尽管两者可联合建模为基于模糊核的卷积，但准确估计模糊核仍是核心难点。为此，本文提出一种模糊预测网络，利用与模糊相关的场景与相机信息，并施加模糊感知稀疏性约束，以估计可靠每像素模糊核。此外，引入动态高斯密集化策略，弥补不完整区域的高斯缺失问题，并通过未见视图信息约束场景优化，提升新视角合成效果。大量实验表明，本方法在模糊与运动模糊单目视频上生成逼真新视角的效果优于当前最优方法。代码已开源：https://github.com/hhhddddddd/dydeblur。
原文摘要 · Abstract (English)
This paper presents a unified framework that allows high-quality dynamic Gaussian Splatting from both defocused and motion-blurred monocular videos. Due to the significant difference between the formation processes of defocus blur and motion blur, existing methods are tailored for either one of them, lacking the ability to simultaneously deal with both of them. Although the two can be jointly modeled as blur kernel-based convolution, the inherent difficulty in estimating accurate blur kernels greatly limits the progress in this direction. In this work, we go a step further towards this direction. Particularly, we propose to estimate per-pixel reliable blur kernels using a blur prediction network that exploits blur-related scene and camera information and is subject to a blur-aware sparsity constraint. Besides, we introduce a dynamic Gaussian densification strategy to mitigate the lack of Gaussians for incomplete regions, and boost the performance of novel view synthesis by incorporating unseen view information to constrain scene optimization. Extensive experiments show that our method outperforms the state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos. Our code is available at https://github.com/hhhddddddd/dydeblur.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。