arXiv:2603.02560cs.CV2026-03被引 5

首个统一处理多天气退化与红外可见光融合的模型。

CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

  • 用联合权重端到端同时完成图像融合与多种天气修复。
  • 在复合天气下优于现有方法,尤其在雨+雾、雨+雪场景表现突出。
  • 适合自动驾驶和无人机监控等真实恶劣环境感知任务。

多模态图像融合(MMIF)通过整合不同模态的互补信息生成更清晰、更具信息量的融合图像,在自动驾驶和无人机监测中尤为重要。然而,现有恶劣天气融合方法通常仅处理单一退化类型(如雾霾、雨或雪),当多种退化共存(如雾霾+雨、雨+雪)时性能下降。为此,我们提出首个端到端框架CAWM-Mamba,统一实现图像融合与复合天气修复。网络包含三个关键模块:(1)天气感知预处理模块(WAPM),增强退化可见特征并提取全局天气嵌入;(2)跨模态特征交互模块(CFIM),促进异质模态对齐与互补特征交换;(3)小波域状态块(WSSB),利用小波分解解耦多频段退化。WSSB引入频域状态机(Freq-SSM)建模各向异性高频退化,避免冗余,并具备统一退化表征机制,提升复杂复合天气下的泛化能力。在AWMM-100K基准及三个标准融合数据集上的大量实验表明,CAWM-Mamba在复合与单天气场景中均持续领先于最先进方法。此外,其融合结果在语义分割与目标检测等下游任务中表现优异,验证了其在真实恶劣天气感知中的实用价值。源代码将发布于https://github.com/Feecuin/CAWM-Mamba。

原文摘要 · Abstract (English)

Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at https://github.com/Feecuin/CAWM-Mamba.

图像融合多天气修复小波域建模自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。