半监督框架实现端到端恶劣天气去除,少标注也能高效复现。
SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal
- 基于教师-学生结构与去噪扩散模型,支持多类天气统一修复。
- 在真实和合成数据集上均优于全监督方法,视觉质量高。
- 通过质量评估筛选伪标签,避免错误引导,适合数据稀缺场景。
恶劣天气去除旨在恢复恶劣天气下的清晰视觉。现有方法大多针对特定天气类型,且严重依赖大量标注数据。为解决这两项局限,本文提出首个基于教师-学生网络与去噪扩散模型(DDM)作为主干的半监督端到端恶劣天气去除框架,命名为SemiDDM-Weather。在DDM主干设计上,采用当前最优的小波扩散模型Wavediff,结合定制输入与损失函数,以促进多对一映射分布的学习,实现有限标注数据下的高效全类型天气修复。为缓解半监督学习中教师网络生成潜在不准确伪标签带来的训练误导风险,引入质量评估与内容一致性约束,筛选教师输出中的“最优”结果作为伪标签,更有效地指导学生网络训练。实验表明,在合成与真实世界数据集上,SemiDDM-Weather持续保持高视觉质量,性能甚至超越全监督对手。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Adverse weather removal aims to restore clear vision under adverse weather conditions. Existing methods are mostly tailored for specific weather types and rely heavily on extensive labeled data. In dealing with these two limitations, this paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network with a Denoising Diffusion Model (DDM) as the backbone, termed SemiDDM-Weather. As for the design of DDM backbone in our SemiDDM-Weather, we adopt the SOTA Wavelet Diffusion Model-Wavediff with customized inputs and loss functions, devoted to facilitating the learning of many-to-one mapping distributions for efficient all-in-one adverse weather removal with limited label data. To mitigate the risk of misleading model training due to potentially inaccurate pseudo-labels generated by the teacher network in semi-supervised learning, we introduce quality assessment and content consistency constraints to screen the "optimal" outputs from the teacher network as the pseudo-labels, thus more effectively guiding the student network training with unlabeled data. Experimental results show that on both synthetic and real-world datasets, our SemiDDM-Weather consistently delivers high visual quality and superior adverse weather removal, even when compared to fully supervised competitors. Our code and pre-trained model are available at this repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。