提出新方法,从模糊事件数据重建清晰三维场景。
Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions
- 基于物理的像素带宽模型,精准建模高速/低光下的事件模糊。
- 在真实与仿真数据上实现更少模糊的NeRF重建,提升视觉质量。
- 适合高动态、低光照或高速运动场景的三维重建研究者。
事件相机因设计哲学差异,在高速、高动态范围和低光条件下表现优于传统相机。然而,其在这些极端条件下仍存在运动模糊问题,主要源于像素带宽受限且与光照强度成正比。为充分发挥事件相机在上述场景的优势,需在下游任务中考虑事件运动模糊,尤其在三维重建中。但现有基于事件重建神经辐射场(NeRF)的方法及事件模拟器均未全面考虑该效应。为此,本文提出Deblur e-NeRF,一种直接从高速或低光条件下生成的运动模糊事件中高效重建低模糊NeRF的新方法。核心是提出一个物理准确的像素带宽模型,可适应任意速度与光照条件下的事件模糊。同时引入阈值归一化的总变差损失,增强对大块无纹理区域的正则化效果。在真实数据和新构建的逼真仿真序列上验证了方法有效性。代码、事件模拟器及合成事件数据集将开源。
原文摘要 · Abstract (English)
The stark contrast in the design philosophy of an event camera makes it particularly ideal for operating under high-speed, high dynamic range and low-light conditions, where standard cameras underperform. Nonetheless, event cameras still suffer from some amount of motion blur, especially under these challenging conditions, in contrary to what most think. This is attributed to the limited bandwidth of the event sensor pixel, which is mostly proportional to the light intensity. Thus, to ensure that event cameras can truly excel in such conditions where it has an edge over standard cameras, it is crucial to account for event motion blur in downstream applications, especially reconstruction. However, none of the recent works on reconstructing Neural Radiance Fields (NeRFs) from events, nor event simulators, have considered the full effects of event motion blur. To this end, we propose, Deblur e-NeRF, a novel method to directly and effectively reconstruct blur-minimal NeRFs from motion-blurred events generated under high-speed motion or low-light conditions. The core component of this work is a physically-accurate pixel bandwidth model proposed to account for event motion blur under arbitrary speed and lighting conditions. We also introduce a novel threshold-normalized total variation loss to improve the regularization of large textureless patches. Experiments on real and novel realistically simulated sequences verify our effectiveness. Our code, event simulator and synthetic event dataset will be open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。