一个模型搞定雨雪雾霾多种天气图像修复，还能灵活切换模式。
MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers
- 用自适应参数调节机制，单架构处理多种天气退化。
- 在多天气基准上超越现有方法，计算开销小。
- 支持不重训练下切换单一或混合天气修复，适合实际应用。
恶劣天气下的图像恢复是众多计算机视觉任务的基础。然而，现有方法通常仅针对单一退化类型，难以应对真实场景中如雨雪、雨雾等复合天气。为此，我们提出多天气视觉Transformer——MWFormer，一种统一架构，可同时处理多种天气引起的图像退化。通过对比学习训练辅助网络，提取与内容无关的、感知退化类型的特征嵌入（支持多类共存），并利用超网络和特征加权线性调制模块，使主网络根据预测的天气类型自适应调整参数，实现局部与全局特征处理。该设计允许在不重新训练的前提下，灵活切换至单一天气或混合天气修复模式，控制性更强。实验表明，MWFormer在多个多天气恢复基准上显著优于现有最先进方法，且计算成本低。此外，超网络策略可推广至其他网络结构，进一步提升性能。代码已开源：https://github.com/taco-group/MWFormer
原文摘要 · Abstract (English)
Restoring images captured under adverse weather conditions is a fundamental task for many computer vision applications. However, most existing weather restoration approaches are only capable of handling a specific type of degradation, which is often insufficient in real-world scenarios, such as rainy-snowy or rainy-hazy weather. Towards being able to address these situations, we propose a multi-weather Transformer, or MWFormer for short, which is a holistic vision Transformer that aims to solve multiple weather-induced degradations using a single, unified architecture. MWFormer uses hyper-networks and feature-wise linear modulation blocks to restore images degraded by various weather types using the same set of learned parameters. We first employ contrastive learning to train an auxiliary network that extracts content-independent, distortion-aware feature embeddings that efficiently represent predicted weather types, of which more than one may occur. Guided by these weather-informed predictions, the image restoration Transformer adaptively modulates its parameters to conduct both local and global feature processing, in response to multiple possible weather. Moreover, MWFormer allows for a novel way of tuning, during application, to either a single type of weather restoration or to hybrid weather restoration without any retraining, offering greater controllability than existing methods. Our experimental results on multi-weather restoration benchmarks show that MWFormer achieves significant performance improvements compared to existing state-of-the-art methods, without requiring much computational cost. Moreover, we demonstrate that our methodology of using hyper-networks can be integrated into various network architectures to further boost their performance. The code is available at: https://github.com/taco-group/MWFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。