无需标注数据，一键修复水下图像并生成新视角。
U2NeRF: Unsupervised Underwater Image Restoration and Neural Radiance Fields
- 用Transformer模型同时学习水下图像修复与三维渲染。
- 单场景优化下，修复效果比基线提升11% LPIPS、5% UIQM、4% UCIQE。
- 适合做水下视觉重建的研究者和开发者参考。
水下图像因光线吸收、折射与散射导致色彩偏移、对比度低和雾状模糊，修复问题备受关注。本文提出无监督水下神经辐射场U2NeRF，一种基于Transformer的架构，可同时根据多视角几何信息生成并修复新视图。由于缺乏监督信号，我们通过隐式方式将修复能力嵌入NeRF流程，解耦预测颜色为场景辐射、直接透射图、后向散射透射图和全局背景光，并在自监督框架下重构水下图像。此外，我们发布了包含12个水下场景的合成与真实数据混合的水下视图合成数据集UVS。实验表明，在单场景优化下，U2NeRF平均优于多个基线方法：LPIPS降低11%，UIQM提升5%，UCIQE提升4%，展现出更强的渲染与修复能力。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Underwater images suffer from colour shifts, low contrast, and haziness due to light absorption, refraction, scattering and restoring these images has warranted much attention. In this work, we present Unsupervised Underwater Neural Radiance Field U2NeRF, a transformer-based architecture that learns to render and restore novel views conditioned on multi-view geometry simultaneously. Due to the absence of supervision, we attempt to implicitly bake restoring capabilities onto the NeRF pipeline and disentangle the predicted color into several components - scene radiance, direct transmission map, backscatter transmission map, and global background light, and when combined reconstruct the underwater image in a self-supervised manner. In addition, we release an Underwater View Synthesis UVS dataset consisting of 12 underwater scenes, containing both synthetically-generated and real-world data. Our experiments demonstrate that when optimized on a single scene, U2NeRF outperforms several baselines by as much LPIPS 11%, UIQM 5%, UCIQE 4% (on average) and showcases improved rendering and restoration capabilities. Code will be made available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。