提出角度感知框架,提升多模态图像融合的细节与边缘精度。
AngularFuse: A Closer Look at Angle-based Perception for Spatial-Sensitive Multi-Modality Image Fusion
- 通过跨模态互补掩码学习模态间差异信息。
- 融合后图像在三个公开数据集上显著优于主流方法。
- 适合需要高精度夜视与自动驾驶融合图像的场景。
可见光-红外图像融合在自动驾驶和夜间监控等关键应用中至关重要,目标是整合多模态信息以生成更适合下游任务的增强图像。尽管基于深度学习的方法已取得进展,现有无监督方法仍面临实际应用挑战。主流方法依赖人工设计损失函数,但存在参考图像细节不足、亮度不均的问题,且常用梯度损失仅关注梯度幅值。为此,本文提出角感知感知框架AngularFuse。首先设计跨模态互补掩码模块,促使网络学习模态间的互补信息;其次引入细粒度参考图像生成策略,结合拉普拉斯边缘增强与自适应直方图均衡化,生成细节更丰富、亮度更均衡的参考图像;最后提出角度感知损失,首次在梯度域中同时约束梯度幅值与方向,确保融合图像既保留纹理强度又保持正确边缘方向。在MSRS、RoadScene和M3FD三个公开数据集上的全面实验表明,AngularFuse显著优于现有主流方法。视觉对比进一步证实,本方法在复杂场景下生成的图像更锐利、细节更丰富,展现出优越的融合能力。
原文摘要 · Abstract (English)
Visible-infrared image fusion is crucial in key applications such as autonomous driving and nighttime surveillance. Its main goal is to integrate multimodal information to produce enhanced images that are better suited for downstream tasks. Although deep learning based fusion methods have made significant progress, mainstream unsupervised approaches still face serious challenges in practical applications. Existing methods mostly rely on manually designed loss functions to guide the fusion process. However, these loss functions have obvious limitations. On one hand, the reference images constructed by existing methods often lack details and have uneven brightness. On the other hand, the widely used gradient losses focus only on gradient magnitude. To address these challenges, this paper proposes an angle-based perception framework for spatial-sensitive image fusion (AngularFuse). At first, we design a cross-modal complementary mask module to force the network to learn complementary information between modalities. Then, a fine-grained reference image synthesis strategy is introduced. By combining Laplacian edge enhancement with adaptive histogram equalization, reference images with richer details and more balanced brightness are generated. Last but not least, we introduce an angle-aware loss, which for the first time constrains both gradient magnitude and direction simultaneously in the gradient domain. AngularFuse ensures that the fused images preserve both texture intensity and correct edge orientation. Comprehensive experiments on the MSRS, RoadScene, and M3FD public datasets show that AngularFuse outperforms existing mainstream methods with clear margin. Visual comparisons further confirm that our method produces sharper and more detailed results in challenging scenes, demonstrating superior fusion capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。