arXiv:2412.08200cs.CVeess.IV2024-12中稿 · publication at BMV…被引 4

用多视角信息无监督消除镜头眩光,让模糊图像变清晰。

GN-FR:Generalizable Neural Radiance Fields for Flare Removal

  • 基于多视角建模眩光,利用邻近视角恢复被遮挡细节。
  • 在782张真实眩光图像上实现无监督去眩光,效果优于单图方法。
  • 适合图像修复、自动驾驶等需要高精度成像的场景。

眩光是由镜头系统内非期望散射和反射引起的光学现象,表现为耀斑、光晕、色溢和雾霾等多种形态,严重干扰成像质量。现有传统与学习方法因依赖单张图像处理,面临病态问题,效果有限。本文将去眩光任务转化为多视角图像问题,利用眩光随视角变化的特性,通过邻近视角信息恢复单图中被遮挡内容。提出通用神经辐射场框架GN-FR(Generalizable Neural Radiance Fields for Flare Removal),可从少量受眩光影响的输入图像中渲染无眩光视图,并在不同场景下实现无监督泛化。该框架集成于通用NeRF Transformer(GNT)架构,包含眩光占用掩码生成(FMG)、视图采样器(VS)与点采样器(PS)。为避免获取真实无眩光数据的困难,设计了基于掩码的无监督损失函数。同时构建首个3D多视角眩光数据集,包含17个真实场景、782张图像、80种真实眩光模式及其标注的眩光占用掩码。本工作首次在NeRF框架下解决眩光去除问题。

原文摘要 · Abstract (English)

Flare, an optical phenomenon resulting from unwanted scattering and reflections within a lens system, presents a significant challenge in imaging. The diverse patterns of flares, such as halos, streaks, color bleeding, and haze, complicate the flare removal process. Existing traditional and learning-based methods have exhibited limited efficacy due to their reliance on single-image approaches, where flare removal is highly ill-posed. We address this by framing flare removal as a multi-view image problem, taking advantage of the view-dependent nature of flare artifacts. This approach leverages information from neighboring views to recover details obscured by flare in individual images. Our proposed framework, GN-FR (Generalizable Neural Radiance Fields for Flare Removal), can render flare-free views from a sparse set of input images affected by lens flare and generalizes across different scenes in an unsupervised manner. GN-FR incorporates several modules within the Generalizable NeRF Transformer (GNT) framework: Flare-occupancy Mask Generation (FMG), View Sampler (VS), and Point Sampler (PS). To overcome the impracticality of capturing both flare-corrupted and flare-free data, we introduce a masking loss function that utilizes mask information in an unsupervised setting. Additionally, we present a 3D multi-view flare dataset, comprising 17 real flare scenes with 782 images, 80 real flare patterns, and their corresponding annotated flare-occupancy masks. To our knowledge, this is the first work to address flare removal within a Neural Radiance Fields (NeRF) framework.

图像修复神经辐射场眩光去除多视角

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。