用红外图辅助去雾并融合可见光图像,一步完成清晰化处理。
Infrared-Assisted Single-Stage Framework for Joint Restoration and Fusion of Visible and Infrared Images under Hazy Conditions
- 引入提示生成机制,调节红外与可见光特征差异。
- 根据雾霾密度选择候选特征,实现去雾与融合同步完成。
- 单阶段框架更轻量,适合实际部署,效果优于两阶段方法。
红外与可见光(IR-VIS)图像融合因广泛应用价值备受关注。然而,现有方法常忽略红外图在雾霾条件下对可见光图像特征的恢复作用。为此,本文提出一种联合学习框架,利用红外图实现雾霾环境下IR-VIS图像的联合去雾与融合。为缓解红外与可见光图像间特征差异问题,设计提示生成机制,从非共享图像信息中构建提示选择矩阵,并通过提示池生成提示嵌入,用于生成去雾候选特征。进一步提出红外辅助特征恢复机制,依据雾霾密度选择候选特征,实现在单阶段框架内的同时去雾与融合。为提升融合质量,构建多阶段提示嵌入融合模块,利用提示生成模块提供的特征补全。所提方法有效融合并去除雾霾,生成清晰无雾融合结果。相比先去雾后融合的两阶段方法,本方案在单阶段框架中实现协同训练,模型更轻量,适用于实际部署。实验验证了其有效性,并优于现有方法。
原文摘要 · Abstract (English)
Infrared and visible (IR-VIS) image fusion has gained significant attention for its broad application value. However, existing methods often neglect the complementary role of infrared image in restoring visible image features under hazy conditions. To address this, we propose a joint learning framework that utilizes infrared image for the restoration and fusion of hazy IR-VIS images. To mitigate the adverse effects of feature diversity between IR-VIS images, we introduce a prompt generation mechanism that regulates modality-specific feature incompatibility. This creates a prompt selection matrix from non-shared image information, followed by prompt embeddings generated from a prompt pool. These embeddings help generate candidate features for dehazing. We further design an infrared-assisted feature restoration mechanism that selects candidate features based on haze density, enabling simultaneous restoration and fusion within a single-stage framework. To enhance fusion quality, we construct a multi-stage prompt embedding fusion module that leverages feature supplementation from the prompt generation module. Our method effectively fuses IR-VIS images while removing haze, yielding clear, haze-free fusion results. In contrast to two-stage methods that dehaze and then fuse, our approach enables collaborative training in a single-stage framework, making the model relatively lightweight and suitable for practical deployment. Experimental results validate its effectiveness and demonstrate advantages over existing methods. The source code of the paper is available at \href{https://github.com/fangjiaqi0909/IASSF}{\textcolor{blue}{https://github.com/fangjiaqi0909/IASSF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。