一个模型搞定多种天气模糊,视频修复更真实。
Removing Multiple Hybrid Adverse Weather in Video via a Unified Model
- 用先验引导模块和动态路由聚合,分别处理空间与时间上的复杂天气干扰。
- 在15种混合天气下表现优异,比现有方法提升3.2~5.7个点(PSNR)。
- 适合需要统一处理多种真实复杂天气的视频修复场景。
真实世界中的视频常受多种混合天气影响,导致退化特征分布不均。现有方法仅针对单一退化类型设计,难以应对复杂的随机天气组合,且因缺乏成对数据,训练困难。为此,本文提出统一模型UniWRV,一次性修复多种异质天气退化。针对空间特征异质性,设计天气先验引导模块,为不同区域生成专属提示以优化特征表征;针对时间特征异质性,提出动态路由聚合模块,自动选择最优融合路径实现时序特征自适应整合。此外,构建新合成数据集HWVideo,包含15种混合天气,共1500对带噪/干净视频片段,并采集真实混合天气视频用于评估泛化能力。大量实验表明,UniWRV在多种混合退化场景中均表现稳健且领先,适用于包括天气去除在内的多种通用视频恢复任务。
原文摘要 · Abstract (English)
Videos captured under real-world adverse weather conditions typically suffer from uncertain hybrid weather artifacts with heterogeneous degradation distributions. However, existing algorithms only excel at specific single degradation distributions due to limited adaption capacity and have to deal with different weather degradations with separately trained models, thus may fail to handle real-world stochastic weather scenarios. Besides, the model training is also infeasible due to the lack of paired video data to characterize the coexistence of multiple weather. To ameliorate the aforementioned issue, we propose a novel unified model, dubbed UniWRV, to remove multiple heterogeneous video weather degradations in an all-in-one fashion. Specifically, to tackle degenerate spatial feature heterogeneity, we propose a tailored weather prior guided module that queries exclusive priors for different instances as prompts to steer spatial feature characterization. To tackle degenerate temporal feature heterogeneity, we propose a dynamic routing aggregation module that can automatically select optimal fusion paths for different instances to dynamically integrate temporal features. Additionally, we managed to construct a new synthetic video dataset, termed HWVideo, for learning and benchmarking multiple hybrid adverse weather removal, which contains 15 hybrid weather conditions with a total of 1500 adverse-weather/clean paired video clips. Real-world hybrid weather videos are also collected for evaluating model generalizability. Comprehensive experiments demonstrate that our UniWRV exhibits robust and superior adaptation capability in multiple heterogeneous degradations learning scenarios, including various generic video restoration tasks beyond weather removal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。