用事件流和模糊图像重建清晰3D场景,提升运动模糊下的视觉质量。
EBAD-Gaussian: Event-driven Bundle Adjusted Deblur Gaussian Splatting
- 结合事件流与模糊图像,联合优化3D高斯分布与相机运动轨迹。
- 在真实数据集上实现优于现有方法的重建精度,模糊图像下仍保持细节。
- 适合处理高速运动或低光环境下的3D重建,尤其适合事件相机用户。
尽管3D高斯泼溅(3D-GS)能实现逼真的新视角合成,但在运动模糊下性能下降。在快速运动或低光照条件下,现有基于RGB的去模糊方法难以建模曝光期间的相机位姿与辐射变化,导致重建精度降低。事件相机通过捕捉曝光期间连续的亮度变化,可有效辅助建模运动模糊并提升重建质量。为此,我们提出事件驱动的束调整去模糊高斯泼溅(EBAD-Gaussian),从事件流和严重模糊的图像中重建清晰的3D高斯分布。该方法在恢复相机运动轨迹的同时联合学习高斯参数。具体而言,首先通过合成曝光期内多个潜在清晰图像构建模糊损失函数,最小化真实与合成模糊图像之间的差异;随后利用事件流监督任意时刻潜在清晰图像间的光强变化,弥补RGB图像中丢失的动态信息;此外,基于事件双积分(EDI)先验,优化中间曝光时刻的潜在清晰图像,施加一致性约束以增强细节与纹理。在合成与真实世界数据集上的大量实验表明,EBAD-Gaussian能在模糊图像与事件流输入下实现高质量3D场景重建。
原文摘要 · Abstract (English)
While 3D Gaussian Splatting (3D-GS) achieves photorealistic novel view synthesis, its performance degrades with motion blur. In scenarios with rapid motion or low-light conditions, existing RGB-based deblurring methods struggle to model camera pose and radiance changes during exposure, reducing reconstruction accuracy. Event cameras, capturing continuous brightness changes during exposure, can effectively assist in modeling motion blur and improving reconstruction quality. Therefore, we propose Event-driven Bundle Adjusted Deblur Gaussian Splatting (EBAD-Gaussian), which reconstructs sharp 3D Gaussians from event streams and severely blurred images. This method jointly learns the parameters of these Gaussians while recovering camera motion trajectories during exposure time. Specifically, we first construct a blur loss function by synthesizing multiple latent sharp images during the exposure time, minimizing the difference between real and synthesized blurred images. Then we use event stream to supervise the light intensity changes between latent sharp images at any time within the exposure period, supplementing the light intensity dynamic changes lost in RGB images. Furthermore, we optimize the latent sharp images at intermediate exposure times based on the event-based double integral (EDI) prior, applying consistency constraints to enhance the details and texture information of the reconstructed images. Extensive experiments on synthetic and real-world datasets show that EBAD-Gaussian can achieve high-quality 3D scene reconstruction under the condition of blurred images and event stream inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。