融合事件相机与RGB图像，修复高速运动下的模糊神经辐射场。
LSE-NeRF: Learning Sensor Modeling Errors for Deblured Neural Radiance Fields with RGB-Event Stereo
- 将相机建模误差作为可学习嵌入，提升重建精度。
- 在自建双目采集数据集上实现更清晰的三维场景重建。
- 适合做动态场景重建或多传感器融合的研究者。
我们提出一种方法，在快速相机运动下仍能重建清晰的神经辐射场（NeRF）。为解决模糊问题，利用双目配置中捕获的模糊RGB图像和事件相机数据。关键在于，重建清晰NeRF时，将简单针孔相机模型带来的建模误差视为每个相机测量的可学习嵌入，并进一步学习一个映射器，连接事件相机与RGB数据。由于此前无适用于该双目设置的数据集，我们引入了一个基于3D打印立体结构的事件相机数据集。实验评估了所提数据集及EVIMOv2，结果表明本方法显著提升重建质量。代码与数据集已开源于https://github.com/ubc-vision/LSENeRF。
原文摘要 · Abstract (English)
We present a method for reconstructing a clear Neural Radiance Field (NeRF) even with fast camera motions. To address blur artifacts, we leverage both (blurry) RGB images and event camera data captured in a binocular configuration. Importantly, when reconstructing our clear NeRF, we consider the camera modeling imperfections that arise from the simple pinhole camera model as learned embeddings for each camera measurement, and further learn a mapper that connects event camera measurements with RGB data. As no previous dataset exists for our binocular setting, we introduce an event camera dataset with captures from a 3D-printed stereo configuration between RGB and event cameras. Empirically, we evaluate our introduced dataset and EVIMOv2 and show that our method leads to improved reconstructions. Our code and dataset are available at https://github.com/ubc-vision/LSENeRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。