统一处理水下图像多种退化问题,提升真实场景修复效果。
UniUIR: Considering Underwater Image Restoration as An All-in-One Learner
- 设计Mamba混合专家模块,分治不同退化类型并保持全局特征。
- 引入空间-频率先验生成器,自适应选择任务提示,提升修复精度。
- 融合深度信息感知区域差异,更好应对局部退化问题。
现有水下图像修复(UIR)方法通常仅处理色彩失真或联合处理色彩与雾效问题,却常忽略真实水下场景中更复杂的混合退化。为此,我们提出通用型水下图像修复方法UniUIR,将现实世界中的复杂混合退化作为整体学习任务。为解耦特定退化问题并探索各类退化间的关联性,设计了基于Mamba的混合专家模块(Mamba Mixture-of-Experts),使各专家可识别不同退化类型,并协同提取任务特定先验,同时维持线性复杂度的全局特征表示。在此基础上,引入空间-频率先验生成器,从空间与频域提取退化先验信息,并根据图像内容自适应选择最适任务提示,从而提高修复准确性。最后,通过引入大规模预训练深度预测模型获取的深度信息,使网络能够感知并利用图像区域间的深度变化,有效处理依赖区域的复杂退化。大量实验表明,UniUIR在定性与定量对比中均表现更优,且泛化能力优于当前最优方法。
原文摘要 · Abstract (English)
Existing underwater image restoration (UIR) methods generally only handle color distortion or jointly address color and haze issues, but they often overlook the more complex degradations that can occur in underwater scenes. To address this limitation, we propose a Universal Underwater Image Restoration method, termed as UniUIR, considering the complex scenario of real-world underwater mixed distortions as an all-in-one manner. To decouple degradation-specific issues and explore the inter-correlations among various degradations in UIR task, we designed the Mamba Mixture-of-Experts module. This module enables each expert to identify distinct types of degradation and collaboratively extract task-specific priors while maintaining global feature representation based on linear complexity. Building upon this foundation, to enhance degradation representation and address the task conflicts that arise when handling multiple types of degradation, we introduce the spatial-frequency prior generator. This module extracts degradation prior information in both spatial and frequency domains, and adaptively selects the most appropriate task-specific prompts based on image content, thereby improving the accuracy of image restoration. Finally, to more effectively address complex, region-dependent distortions in UIR task, we incorporate depth information derived from a large-scale pre-trained depth prediction model, thereby enabling the network to perceive and leverage depth variations across different image regions to handle localized degradation. Extensive experiments demonstrate that UniUIR can produce more attractive results across qualitative and quantitative comparisons, and shows strong generalization than state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。