用事件流和光流提升模糊图像的3D重建清晰度
EvFlow-GS: Event Enhanced Motion Deblurring with Optical Flow for 3D Gaussian Splatting
- 结合事件流与光流，联合优化双积分、相机位姿与3D高斯点云
- 在RealEgo-4D数据集上达到12.89dB PSNR，优于现有方法
- 适合做运动去模糊与3D重建的开发者参考
仅从运动模糊图像实现清晰3D重建极具挑战性，促使近期研究引入事件相机以利用其微秒级时间分辨率。然而，这些方法因事件双积分先验不准确及事件噪声和模糊导致残余伪影与纹理模糊。本文提出EvFlow-GS，一个统一框架，通过事件流与光流联合优化可学习双积分（LDI）、相机位姿与3D高斯点阵（3DGS）。首先利用光流提取事件边缘信息，并设计新型事件损失分别作用于不同模块；其次引入事件残差先验，强化3DGS渲染图像间强度变化的监督；最后将3DGS与LDI输出整合进联合损失，实现双向优化。实验表明，EvFlow-GS在RealEgo-4D数据集上达到12.89dB PSNR，性能领先。
原文摘要 · Abstract (English)
Achieving sharp 3D reconstruction from motion-blurred images alone becomes challenging, motivating recent methods to incorporate event cameras, benefiting from microsecond temporal resolution. However, they suffer from residual artifacts and blurry texture details due to misleading supervision from inaccurate event double integral priors and noisy, blurry events. In this study, we propose EvFlow-GS, a unified framework that leverages event streams and optical flow to optimize an end-to-end learnable double integral (LDI), camera poses, and 3D Gaussian Splatting (3DGS) jointly on-the-fly. Specifically, we first extract edge information from the events using optical flow and then formulate a novel event-based loss applied separately to different modules. Additionally, we exploit a novel event-residual prior to strengthen the supervision of intensity changes between images rendered from 3DGS. Finally, we integrate the outputs of both 3DGS and LDI into a joint loss, enabling their optimization to mutually facilitate each other. Experiments demonstrate the leading performance of our EvFlow-GS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。