将图像退化建模与融合过程耦合，提升低质红外可见光图像融合效果。
Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion
- 用视觉语言模型感知退化特征，实现退化抑制与特征提取同步进行。
- 在多种退化场景下，性能显著优于现有方法。
- 适合处理真实环境中质量不佳的红外与可见光图像融合任务。
现有红外与可见光图像融合（IVIF）方法通常假设输入图像质量良好，但在处理退化图像时，需手动切换不同预处理技术，导致退化处理与融合过程脱节，性能大幅下降。本文提出一种新型的视觉语言模型引导的退化耦合融合网络（VGDCFusion），将退化建模与融合过程紧密耦合，并利用视觉语言模型（VLM）实现退化感知与引导抑制。具体地，提出的特定提示退化耦合提取器（SPDCE）实现模态特异性退化感知，并联合建模退化抑制与模态内特征提取；同时，联合提示退化耦合融合模块（JPDCF）促进跨模态退化感知，将残余退化滤波与互补跨模态特征融合耦合。大量实验表明，所提方法在多种退化图像场景下显著优于现有最先进融合方法。代码已开源：https://github.com/Lmmh058/VGDCFusion。
原文摘要 · Abstract (English)
Existing Infrared and Visible Image Fusion (IVIF) methods typically assume high-quality inputs. However, when handing degraded images, these methods heavily rely on manually switching between different pre-processing techniques. This decoupling of degradation handling and image fusion leads to significant performance degradation. In this paper, we propose a novel VLM-Guided Degradation-Coupled Fusion network (VGDCFusion), which tightly couples degradation modeling with the fusion process and leverages vision-language models (VLMs) for degradation-aware perception and guided suppression. Specifically, the proposed Specific-Prompt Degradation-Coupled Extractor (SPDCE) enables modality-specific degradation awareness and establishes a joint modeling of degradation suppression and intra-modal feature extraction. In parallel, the Joint-Prompt Degradation-Coupled Fusion (JPDCF) facilitates cross-modal degradation perception and couples residual degradation filtering with complementary cross-modal feature fusion. Extensive experiments demonstrate that our VGDCFusion significantly outperforms existing state-of-the-art fusion approaches under various degraded image scenarios. Our code is available at https://github.com/Lmmh058/VGDCFusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。