用可组合提示池自适应恢复复杂天气，效率显著提升。
Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint
- 构建提示池，自动组合子提示生成天气特征。
- 在多个数据集上超越现有扩散模型，计算效率更高。
- 适合需要高效处理真实复杂天气的图像修复场景。
近期在恶劣天气图像恢复方面取得进展，但现实世界中天气退化类型和组合难以预测，给模型带来挑战。以往方法难以动态处理复杂的退化组合，且背景重建不精确，导致性能与泛化能力受限。受提示学习和‘因材施教’理念启发，本文提出新框架 T3-DiffWeather。通过提示池机制，网络可自主组合子提示构建天气提示，灵活应对未见天气输入；从场景建模角度，引入 Depth-Anything 特征约束的通用提示，为扩散过程提供场景特定条件；同时，采用对比提示损失，以相互推斥策略确保两类提示表征的区分性。实验表明，该方法在多种合成与真实世界数据集上达到领先性能，尤其在计算效率方面显著优于现有扩散技术。
原文摘要 · Abstract (English)
Recent advancements in adverse weather restoration have shown potential, yet the unpredictable and varied combinations of weather degradations in the real world pose significant challenges. Previous methods typically struggle with dynamically handling intricate degradation combinations and carrying on background reconstruction precisely, leading to performance and generalization limitations. Drawing inspiration from prompt learning and the "Teaching Tailored to Talent" concept, we introduce a novel pipeline, T3-DiffWeather. Specifically, we employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts, harnessing the necessary attributes to adaptively tackle unforeseen weather input. Moreover, from a scene modeling perspective, we incorporate general prompts constrained by Depth-Anything feature to provide the scene-specific condition for the diffusion process. Furthermore, by incorporating contrastive prompt loss, we ensures distinctive representations for both types of prompts by a mutual pushing strategy. Experimental results demonstrate that our method achieves state-of-the-art performance across various synthetic and real-world datasets, markedly outperforming existing diffusion techniques in terms of computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。