首个无需参考图的水下场景风格迁移框架,能保留结构同时变换水体风格。
UStyle: Waterbody Style Transfer of Underwater Scenes by Depth-Guided Feature Synthesis
- 基于深度引导的特征合成,融合物理模型实现风格迁移
- 在7种水体风格上超越现有方法,保持颜色、结构与高频特征一致
- 适合水下图像处理、海洋视觉研究者使用
水下场景的水体风格迁移在图像与视觉领域仍属空白。传统图像风格迁移多关注艺术化与写实融合,难以在高散射介质中保持物体与场景几何。波长依赖的非线性衰减和深度相关的后向散射干扰了从无配对数据中学习水下风格迁移。本文提出UStyle,首个无需参考图像或场景信息的数据驱动水下风格迁移框架。设计深度感知的白化与着色变换(DA-WCT)机制,结合物理基础的水体合成,确保感知一致性的同时保留场景结构。通过精心设计的损失函数,引导模型维持色彩丰富度、亮度、结构完整性、频域特性及VGG与CLIP特征空间中的高层内容。针对特定领域挑战,UStyle提供稳健的无参考水下图像风格迁移方案,性能优于仅依赖端到端重建损失的现有方法。此外,我们构建了UF7D数据集,包含7种不同水体风格的高分辨率水下图像,为未来研究提供基准。UStyle推理流程与UF7D数据集已开源:https://github.com/uf-robopi/UStyle。
原文摘要 · Abstract (English)
The concept of waterbody style transfer remains largely unexplored in the underwater imaging and vision literature. Traditional image style transfer (STx) methods primarily focus on artistic and photorealistic blending, often failing to preserve object and scene geometry in images captured in high-scattering mediums such as underwater. The wavelength-dependent nonlinear attenuation and depth-dependent backscattering artifacts further complicate learning underwater image STx from unpaired data. This paper introduces UStyle, the first data-driven learning framework for transferring waterbody styles across underwater images without requiring prior reference images or scene information. We propose a novel depth-aware whitening and coloring transform (DA-WCT) mechanism that integrates physics-based waterbody synthesis to ensure perceptually consistent stylization while preserving scene structure. To enhance style transfer quality, we incorporate carefully designed loss functions that guide UStyle to maintain colorfulness, lightness, structural integrity, and frequency-domain characteristics, as well as high-level content in VGG and CLIP (contrastive language-image pretraining) feature spaces. By addressing domain-specific challenges, UStyle provides a robust framework for no-reference underwater image STx, surpassing state-of-the-art (SOTA) methods that rely solely on end-to-end reconstruction loss. Furthermore, we introduce the UF7D dataset, a curated collection of high-resolution underwater images spanning seven distinct waterbody styles, establishing a benchmark to support future research in underwater image STx. The UStyle inference pipeline and UF7D dataset are released at: https://github.com/uf-robopi/UStyle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。