arXiv:2411.06757cs.CV2024-11NeurIPS被引 17

解决手持低光照片重建中噪点与模糊耦合问题,实现清晰三维场景重建。

LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes

  • 分步建模图像噪声与运动模糊,利用多视角一致性与频率信息解耦
  • 在合成与真实数据上均超越现有方法,显著提升低光场景重建质量
  • 适合从事低光三维重建、图像恢复与NeRF优化的研究者

神经辐射场(NeRF)在高质量图像生成新视角图像方面表现优异。然而,手持低光拍摄会同时导致可见度低、噪声大和相机抖动,挑战了现有NeRF方法。现有方法通常仅能处理低光或运动模糊,直接组合或引入图像增强手段效果不佳,因这些退化因素高度耦合。我们观察到低光图像中的噪声始终锐利,而相机抖动造成模糊,暗示图像形成过程中存在隐含的退化顺序。为此,本文提出新模型LuSh-NeRF,可从一组手持低光图像中重建出清晰干净的三维场景。核心思想是通过多视角特征一致性建模噪声,利用NeRF的频率信息建模模糊。具体包括:新颖的场景-噪声分解(SND)模块分离噪声与场景表征,以及基于低频场景信息估计相机轨迹的相机运动预测(CTP)模块。为支持训练与评估,构建了包含合成与真实图像的新数据集。实验表明,LuSh-NeRF优于现有方法。代码与数据集见:https://github.com/quzefan/LuSh-NeRF。

原文摘要 · Abstract (English)

Neural Radiance Fields (NeRFs) have shown remarkable performances in producing novel-view images from high-quality scene images. However, hand-held low-light photography challenges NeRFs as the captured images may simultaneously suffer from low visibility, noise, and camera shakes. While existing NeRF methods may handle either low light or motion, directly combining them or incorporating additional image-based enhancement methods does not work as these degradation factors are highly coupled. We observe that noise in low-light images is always sharp regardless of camera shakes, which implies an implicit order of these degradation factors within the image formation process. To this end, we propose in this paper a novel model, named LuSh-NeRF, which can reconstruct a clean and sharp NeRF from a group of hand-held low-light images. The key idea of LuSh-NeRF is to sequentially model noise and blur in the images via multi-view feature consistency and frequency information of NeRF, respectively. Specifically, LuSh-NeRF includes a novel Scene-Noise Decomposition (SND) module for decoupling the noise from the scene representation and a novel Camera Trajectory Prediction (CTP) module for the estimation of camera motions based on low-frequency scene information. To facilitate training and evaluations, we construct a new dataset containing both synthetic and real images. Experiments show that LuSh-NeRF outperforms existing approaches. Our code and dataset can be found here: https://github.com/quzefan/LuSh-NeRF.

NeRF低光重建图像去噪三维重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。