用事件流和扩散先验提升模糊图像的3D重建质量
DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting
- 结合事件流积分与扩散先验，分两阶段优化3D高斯点云
- 在真实和合成数据上显著改善新视角图像清晰度
- 适合做运动模糊修复与高质量3D重建的研究者
从多视角模糊图像中重建清晰的3D表示是计算机视觉中的长期挑战。近期工作尝试利用事件相机的高动态范围和微秒级时间分辨率来提升运动模糊下的高质量新视角合成效果。然而，这些方法常在颜色恢复不准确或丢失细粒度细节方面表现不佳。本文提出DiET-GS，一种基于扩散先验与事件流辅助的运动去模糊3D高斯点云（3DGS）框架。该框架通过两阶段训练策略，有效融合无模糊事件流与扩散先验。具体而言，我们引入新型事件双积分约束机制，实现准确的颜色还原与清晰的细节保留；同时提出简单技术，利用扩散先验进一步增强边缘细节。在合成与真实世界数据上的定性和定量实验表明，相较于现有基线方法，DiET-GS能生成显著更优的新视角图像质量。
原文摘要 · Abstract (English)
Reconstructing sharp 3D representations from blurry multi-view images are long-standing problem in computer vision. Recent works attempt to enhance high-quality novel view synthesis from the motion blur by leveraging event-based cameras, benefiting from high dynamic range and microsecond temporal resolution. However, they often reach sub-optimal visual quality in either restoring inaccurate color or losing fine-grained details. In this paper, we present DiET-GS, a diffusion prior and event stream-assisted motion deblurring 3DGS. Our framework effectively leverages both blur-free event streams and diffusion prior in a two-stage training strategy. Specifically, we introduce the novel framework to constraint 3DGS with event double integral, achieving both accurate color and well-defined details. Additionally, we propose a simple technique to leverage diffusion prior to further enhance the edge details. Qualitative and quantitative results on both synthetic and real-world data demonstrate that our DiET-GS is capable of producing significantly better quality of novel views compared to the existing baselines. Our project page is https://diet-gs.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。