arXiv:2504.12869cs.CV2025-04被引 5

提出SC3EF框架,提升可见光与热成像图像配准精度。

SC3EF: A Joint Self-Correlation and Cross-Correspondence Estimation Framework for Visible and Thermal Image Registration

  • 融合局部特征与全局上下文,联合估计跨模态对应关系。
  • 在多个RGB-T数据集上超越当前最先进方法,误差更低。
  • 适用于大视差、遮挡、恶劣天气等复杂场景,泛化性强。

多光谱成像在智能交通系统中至关重要,涵盖高级驾驶辅助系统(ADAS)、交通监控和夜视应用。然而,由于可见光与热成像之间存在显著模态差异,准确的可见光-热图像配准仍具挑战性。本文提出一种新型联合自相关与跨对应估计框架(SC3EF),通过结合局部代表性特征与全局上下文信息,有效生成可见光与热图像间的对应关系。为此,设计了一种基于卷积-变换器的流水线,提取局部特征并编码同模态的全局相关性,用于异模态对应估计。在融合局部与全局对应估计结果后,进一步采用分层光流估计解码器逐步优化密集对应图。大量实验表明,所提方法在代表性RGB-T数据集上优于现有最先进方法;同时在大视差、严重遮挡、恶劣天气及跨模态数据集(如RGB-N、RGB-D)中也展现出良好泛化能力。

原文摘要 · Abstract (English)

Multispectral imaging plays a critical role in a range of intelligent transportation applications, including advanced driver assistance systems (ADAS), traffic monitoring, and night vision. However, accurate visible and thermal (RGB-T) image registration poses a significant challenge due to the considerable modality differences. In this paper, we present a novel joint Self-Correlation and Cross-Correspondence Estimation Framework (SC3EF), leveraging both local representative features and global contextual cues to effectively generate RGB-T correspondences. For this purpose, we design a convolution-transformer-based pipeline to extract local representative features and encode global correlations of intra-modality for inter-modality correspondence estimation between unaligned visible and thermal images. After merging the local and global correspondence estimation results, we further employ a hierarchical optical flow estimation decoder to progressively refine the estimated dense correspondence maps. Extensive experiments demonstrate the effectiveness of our proposed method, outperforming the current state-of-the-art (SOTA) methods on representative RGB-T datasets. Furthermore, it also shows competitive generalization capabilities across challenging scenarios, including large parallax, severe occlusions, adverse weather, and other cross-modal datasets (e.g., RGB-N and RGB-D).

图像配准跨模态热成像视觉融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。