arXiv:2605.13049cs.CV2026-05

解决红外与可见光图像未对齐时的融合误差问题,提升多模态图像质量。

Uncertainty-aware Spatial-Frequency Registration and Fusion for Infrared and Visible Images

论文配图:Uncertainty-aware Spatial-Frequency Registration and Fusion for Infrared and Visible Images
图 1 · 摘自论文原文
  • 分尺度迭代注册,结合不确定性估计动态修正配准误差。
  • 引入热辐射分布一致性作为频域监督信号,提升全局对齐精度。
  • 双分支融合模块同时利用空间与频域信息,生成视觉更优的融合图像。

红外与可见光图像融合(IVIF)在复杂环境下展现出巨大潜力,但未对齐条件下的融合存在固有错位问题。现有方法多采用粗略到精细的变形参数预测或多层次变形场估计进行配准,但忽略了配准过程中的累积误差,这些误差会污染融合阶段,严重降低图像质量。本文提出空间-频率联合注册与融合框架(SFRF),将不确定性估计和红外热辐射分布一致性整合进统一流程,实现跨空间与频域的鲁棒注册与融合。具体地,SFRF构建了多尺度迭代注册(MIR)框架,在各尺度上迭代优化变形场,并通过不确定性估计动态抑制误差累积,提升对齐精度;为确保红外热辐射分布准确对齐,引入热辐射分布一致性作为频域监督信号,促进频域全局一致性。基于空间-频率对齐结果,SFRF进一步设计双分支空间-频率融合(DSFF)模块,融合空间几何特征与频域分布信息,重建视觉效果更佳的图像。SFRF在多个数据集上均取得优异表现。

原文摘要 · Abstract (English)

Infrared and Visible Image Fusion (IVIF) has shown promise in visual tasks under challenging environments, but fusion under unregistered conditions faces inherent misalignments. Current studies to solve them either predict the deformation parameters coarse-to-fine (i.e., coarse registration and fine registration) or estimate the deformation fields in multi-scales for registration. Though straightforward, they overlook the cumulative errors in registration, which contaminate the fusion stage and severely deteriorate the resulting images. We introduce the Spatial-Frequency Registration and Fusion (SFRF) framework, which incorporates uncertainty estimation and infrared thermal radiation distribution consistency into a unified pipeline to handle the error accumulation for robust registration and fusion across both spatial and frequency domains. Specifically, SFRF constructs a Multi-scale Iterative Registration (MIR) framework that iteratively refines the deformation field across scales, leveraging uncertainty estimation at each stage to mitigate error accumulation and enhance alignment accuracy dynamically. To ensure the accurate alignment of infrared thermal distributions during registration, thermal radiation distribution consistency is employed as a frequency-domain supervisory signal, promoting global consistency in the frequency domain. Based on the spatial-frequency alignment, SFRF further adopts a Dual-branch Spatial-Frequency Fusion (DSFF) module, which incorporates spatial geometric features and frequency distribution information to reconstruct visually appealing images. SFRF achieves impressive performance across diverse datasets.

图像融合红外可见光多模态不确定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。