arXiv:2509.22450cs.CV2025-09被引 2

无需分割标签,通过自监督学习实现可见光与红外图像融合

SSVIF: Self-Supervised Segmentation-Oriented Visible and Infrared Image Fusion

论文配图:SSVIF: Self-Supervised Segmentation-Oriented Visible and Infrared Image Fusion
图 1 · 摘自论文原文
  • 利用特征级与像素级分割一致性设计自监督任务
  • 在无标注数据下融合效果超越传统方法,接近有监督模型
  • 适合缺乏标注数据的多模态图像融合场景

可见光与红外图像融合(VIF)在场景分割、目标检测等任务中应用广泛。传统VIF方法仅关注融合图像质量,而面向应用的VIF方法通过引入下游任务损失提升性能,但需依赖语义分割或目标检测等标注数据,数据获取成本高。为此,本文提出一种面向分割任务的自监督可见光-红外图像融合框架(SSVIF)。基于特征级融合分割与像素级融合分割的一致性,设计任务交叉分割一致性自监督机制,使模型在无分割标签情况下学习高层语义特征。同时,采用两阶段训练策略与动态权重调整方法,实现高效联合学习。在多个公开数据集上的大量实验表明,尽管仅使用未标注的可见光-红外图像对进行训练,SSVIF的融合性能超过传统方法,并可媲美有监督的分割导向方法。代码将在论文接收后开源。

原文摘要 · Abstract (English)

Visible and infrared image fusion (VIF) has gained significant attention in recent years due to its wide application in tasks such as scene segmentation and object detection. VIF methods can be broadly classified into traditional VIF methods and application-oriented VIF methods. Traditional methods focus solely on improving the quality of fused images, while application-oriented VIF methods additionally consider the performance of downstream tasks on fused images by introducing task-specific loss terms during training. However, compared to traditional methods, application-oriented VIF methods require datasets labeled for downstream tasks (e.g., semantic segmentation or object detection), making data acquisition labor-intensive and time-consuming. To address this issue, we propose a self-supervised training framework for segmentation-oriented VIF methods (SSVIF). Leveraging the consistency between feature-level fusion-based segmentation and pixel-level fusion-based segmentation, we introduce a novel self-supervised task-cross-segmentation consistency-that enables the fusion model to learn high-level semantic features without the supervision of segmentation labels. Additionally, we design a two-stage training strategy and a dynamic weight adjustment method for effective joint learning within our self-supervised framework. Extensive experiments on public datasets demonstrate the effectiveness of our proposed SSVIF. Remarkably, although trained only on unlabeled visible-infrared image pairs, our SSVIF outperforms traditional VIF methods and rivals supervised segmentation-oriented ones. Our code will be released upon acceptance.

图像融合自监督学习多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。