解决光照变化下红外与可见光图像融合的失真问题
Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion
- 动态门控机制自动选择亮度相关特征进行归一化
- 在不同亮度条件下保持图像信息完整性和视觉质量
- 适合需要稳定融合效果的夜视、安防等场景
红外与可见光图像融合旨在整合多模态优势，生成视觉增强、信息丰富的图像。然而，真实场景中可见光成像易受动态环境亮度变化影响，导致纹理退化。现有融合方法对亮度扰动缺乏鲁棒性，严重降低融合图像的视觉保真度。为此，本文提出亮度自适应多模态动态融合框架（BA-Fusion），可在亮度波动下实现鲁棒融合。具体地，设计亮度自适应门控（BAG）模块，动态选择亮度相关通道进行归一化，同时保留源图像中与亮度无关的结构信息；并引入亮度一致性损失函数优化BAG模块。整个框架采用交替训练策略进行端到端优化。大量实验表明，本方法在保留多模态信息和视觉保真度方面优于现有先进方法，并在不同亮度水平下表现出显著鲁棒性。代码已公开：https://github.com/SunYM2020/BA-Fusion。
原文摘要 · Abstract (English)
Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。