让红外可见光融合模型能自动适配不同任务,无需重训练。
Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion

- 通过闭环反馈机制,根据任务需求动态调整融合网络结构。
- 在三个数据集上同时保持高质量融合与多任务适应能力。
- 适合需要灵活应对多种下游任务的图像融合场景。
红外-可见光图像融合旨在整合互补信息以实现鲁棒的视觉理解,但现有方法难以同时适应多个下游任务。为此,我们提出闭环动态网络(CLDyN),可自适应响应多样下游任务的语义需求,实现任务定制化图像融合。具体而言,CLDyN引入闭环优化机制,建立语义传输链,通过需求驱动的语义补偿(RSC)模块,将下游任务的反馈显式传递回融合网络。RSC模块利用基向量库(BVB)和架构自适应语义注入(A2SI)块,根据任务需求定制网络结构,实现特定语义补偿,使融合网络能在不重新训练的前提下主动适应各类任务。为促进语义补偿,引入奖励-惩罚策略,依据任务性能变化对RSC模块进行奖励或惩罚。在M3FD、FMB和VT5000数据集上的实验表明,CLDyN不仅维持了高融合质量,还展现出强大的多任务适应能力。代码已开源:https://github.com/YR0211/CLDyN。
原文摘要 · Abstract (English)
Infrared-visible image fusion aims to integrate complementary information for robust visual understanding, but existing fusion methods struggle with simultaneously adapting to multiple downstream tasks. To address this issue, we propose a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks for task-customized image fusion. Specifically, CLDyN introduces a closed-loop optimization mechanism that establishes a semantic transmission chain to achieve explicit feedback from downstream tasks to the fusion network through a Requirement-driven Semantic Compensation (RSC) module. The RSC module leverages a Basis Vector Bank (BVB) and an Architecture-Adaptive Semantic Injection (A2SI) block to customize the network architecture according to task requirements, thereby enabling task-specific semantic compensation and allowing the fusion network to actively adapt to diverse tasks without retraining. To promote semantic compensation, a reward-penalty strategy is introduced to reward or penalize the RSC module based on task performance variations. Experiments on the M3FD, FMB, and VT5000 datasets demonstrate that CLDyN not only maintains high fusion quality but also exhibits strong multi-task adaptability. The code is available at https://github.com/YR0211/CLDyN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。