arXiv:2504.05164cs.CV2025-04ICCV被引 5

TITA框架动态平衡任务通用与特异性,提升多源图像融合性能与泛化能力。

Balancing Task-invariant Interaction and Task-specific Adaptation for Unified Image Fusion

  • 引入IPA模块增强像素级交互,提取多源互补信息。
  • OAF模块根据任务特性动态调整操作权重,实现自适应融合。
  • 支持未见融合任务的泛化,适合多场景图像融合应用。

统一图像融合旨在通过统一框架整合多源图像的互补信息,提升图像质量。现有方法虽能处理不同融合任务,但依赖显式任务识别,限制了对未见任务的泛化能力。为此,我们提出TITA框架,动态平衡任务通用交互与任务特定适应。IPA模块增强像素级交互以更好提取多源互补信息;OAF模块基于任务特性动态调整操作权重;同时采用快速自适应多任务优化(FAMO)缓解联合训练中的梯度冲突。大量实验表明,TITA在三种图像融合场景下表现媲美专用方法,并具备强泛化能力。代码已开源:https://github.com/huxingyuabc/TITA。

原文摘要 · Abstract (English)

Unified image fusion aims to integrate complementary information from multi-source images, enhancing image quality through a unified framework applicable to diverse fusion tasks. While treating all fusion tasks as a unified problem facilitates task-invariant knowledge sharing, it often overlooks task-specific characteristics, thereby limiting the overall performance. Existing general image fusion methods incorporate explicit task identification to enable adaptation to different fusion tasks. However, this dependence during inference restricts the model's generalization to unseen fusion tasks. To address these issues, we propose a novel unified image fusion framework named "TITA", which dynamically balances both Task-invariant Interaction and Task-specific Adaptation. For task-invariant interaction, we introduce the Interaction-enhanced Pixel Attention (IPA) module to enhance pixel-wise interactions for better multi-source complementary information extraction. For task-specific adaptation, the Operation-based Adaptive Fusion (OAF) module dynamically adjusts operation weights based on task properties. Additionally, we incorporate the Fast Adaptive Multitask Optimization (FAMO) strategy to mitigate the impact of gradient conflicts across tasks during joint training. Extensive experiments demonstrate that TITA not only achieves competitive performance compared to specialized methods across three image fusion scenarios but also exhibits strong generalization to unseen fusion tasks. The source codes are released at https://github.com/huxingyuabc/TITA.

图像融合自适应多任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。