arXiv:2505.06665cs.CV2025-05被引 2

用多任务学习让融合图像同时生成分割结果,更高效。

MultiTaskVIF: Segmentation-oriented visible and infrared image fusion via multi-task learning

  • 用多任务头解码器同时输出融合图和分割图
  • 无需完整分割模型,融合网络直接学语义特征
  • 结构更简洁,适合需要图像融合与分割的场景

可见光与红外图像融合(VIF)近年来受到广泛关注。传统方法主要关注生成高质量视觉融合图像,而近期进展逐渐强调在训练中引入语义信息。然而,多数面向分割的VIF方法采用分离的融合与分割模型级联结构,导致网络复杂且冗余。本文提出一种简洁通用的多任务训练框架MultiTaskVIF,通过引入多任务头解码器(MTH),在训练时同时输出融合图像与分割结果。相比以往需联合训练完整分割模型的级联框架,MultiTaskVIF仅需将原解码器替换为MTH,即可使融合模型直接学习语义特征。大量实验验证了该方法的有效性。代码将在录用后公开。

原文摘要 · Abstract (English)

Visible and infrared image fusion (VIF) has attracted significant attention in recent years. Traditional VIF methods primarily focus on generating fused images with high visual quality, while recent advancements increasingly emphasize incorporating semantic information into the fusion model during training. However, most existing segmentation-oriented VIF methods adopt a cascade structure comprising separate fusion and segmentation models, leading to increased network complexity and redundancy. This raises a critical question: can we design a more concise and efficient structure to integrate semantic information directly into the fusion model during training-Inspired by multi-task learning, we propose a concise and universal training framework, MultiTaskVIF, for segmentation-oriented VIF models. In this framework, we introduce a multi-task head decoder (MTH) to simultaneously output both the fused image and the segmentation result during training. Unlike previous cascade training frameworks that necessitate joint training with a complete segmentation model, MultiTaskVIF enables the fusion model to learn semantic features by simply replacing its decoder with MTH. Extensive experimental evaluations validate the effectiveness of the proposed method. Our code will be released upon acceptance.

图像融合多任务学习分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。