解决红外与可见光图像融合中的配准难题,提升精度与效率。
FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration
- 通过学习模态错位特征而非强制对齐,增强鲁棒性。
- 在三个数据集上实现更优细节对齐与抗干扰能力。
- 适合作为通用模块嵌入各类融合方法,无需额外预处理。
跨模态空间配准是真实世界感知中多模态图像融合的关键挑战。现有基于配准的融合方法通常需大量预处理操作,影响效率。为此,本文提出一种基于视觉先验的通用跨模态配准方法FusionRegister,用于红外与可见光图像融合。该方法不强制对齐所有差异,而是学习跨模态错位表示,确保在复杂输入下仍具稳定输出。FusionRegister直接作用于融合结果,显式表达并有效处理错位信息,可无缝集成至多种融合方法,保持其原有特性。同时,利用主干融合模型作为自然视觉先验,引导配准聚焦于错位区域,避免冗余计算。在三个数据集上的实验表明,FusionRegister不仅继承了先进方法的融合质量,还在细节对齐和鲁棒性方面表现更优,适用于红外与可见光图像融合任务。代码将发布于https://github.com/bociic/FusionRegister。
原文摘要 · Abstract (English)
Spatial registration across different visual modalities is a critical but formidable step in multi-modality image fusion for real-world perception. Although several methods are proposed to address this issue, the existing registration-based fusion methods typically require extensive pre-registration operations, limiting their efficiency. To overcome these limitations, a general cross-modality registration method guided by visual priors is proposed for infrared and visible image fusion task, termed FusionRegister. Firstly, FusionRegister achieves robustness by learning cross-modality misregistration representations rather than forcing alignment of all differences, ensuring stable outputs even under challenging input conditions. Moreover, FusionRegister demonstrates strong generality by operating directly on fused results, where misregistration is explicitly represented and effectively handled, enabling seamless integration with diverse fusion methods while preserving their intrinsic properties. In addition, its efficiency is further enhanced by serving the backbone fusion method as a natural visual prior provider, which guides the registration process to focus only on mismatch regions, thereby avoiding redundant operations. Extensive experiments on three datasets demonstrate that FusionRegister not only inherits the fusion quality of state-of-the-art methods, but also delivers superior detail alignment and robustness, making it highly suitable for infrared and visible image fusion method. The code will be available at https://github.com/bociic/FusionRegister.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。