用新模型和真实感数据生成,实现快速高质水下图像修复
Single-Step Latent Diffusion for Underwater Image Restoration
- 结合预训练潜空间扩散模型与显式场景分解,建模光线衰减与散射
- 在合成数据上提升3 dB PSNR,速度比现有方法快200倍以上
- 适合需要实时处理的水下摄影、生态监测等应用
水下图像修复旨在恢复水下拍摄场景的颜色、对比度和外观,广泛应用于海洋生态、水产养殖、水下建筑与考古等领域。现有基于像素域扩散的修复方法虽能处理简单场景,但计算开销大,且在复杂几何和显著深度变化的场景中常产生不自然伪影。本文提出SLURPP新架构,融合预训练潜空间扩散模型(具备强几何与深度先验)与显式场景分解,以建模并校正光衰减与后向散射效应。为训练该模型,设计了基于物理的水下图像合成流程,将多样且真实的退化效果施加于现有陆地图像数据集,生成带有密集介质/退化标注的多样化训练数据。在合成与真实世界基准上全面评估表明,该方法性能达到当前最优。尤其在合成基准上,相比现有扩散方法,PSNR提升约3 dB,速度提高200倍以上;在真实数据上亦呈现显著的定性改进。
原文摘要 · Abstract (English)
Underwater image restoration algorithms seek to restore the color, contrast, and appearance of a scene that is imaged underwater. They are a critical tool in applications ranging from marine ecology and aquaculture to underwater construction and archaeology. While existing pixel-domain diffusion-based image restoration approaches are effective at restoring simple scenes with limited depth variation, they are computationally intensive and often generate unrealistic artifacts when applied to scenes with complex geometry and significant depth variation. In this work we overcome these limitations by combining a novel network architecture (SLURPP) with an accurate synthetic data generation pipeline. SLURPP combines pretrained latent diffusion models -- which encode strong priors on the geometry and depth of scenes -- with an explicit scene decomposition -- which allows one to model and account for the effects of light attenuation and backscattering. To train SLURPP we design a physics-based underwater image synthesis pipeline that applies varied and realistic underwater degradation effects to existing terrestrial image datasets. This approach enables the generation of diverse training data with dense medium/degradation annotations. We evaluate our method extensively on both synthetic and real-world benchmarks and demonstrate state-of-the-art performance. Notably, SLURPP is over 200X faster than existing diffusion-based methods while offering ~ 3 dB improvement in PSNR on synthetic benchmarks. It also offers compelling qualitative improvements on real-world data. Project website https://tianfwang.github.io/slurpp/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。