统一融合框架,跨任务保持源图信息,视觉效果更优。
UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation
- 用DINOv3构建统一语义空间,支持多模态输入
- 引入重建对齐损失,有效保留源图细节
- 双层优化平衡融合与重建,适合真实场景
图像融合旨在整合多源图像的互补信息,生成更具信息量且视觉一致的表示,有助于人类感知和下游视觉任务。尽管已有进展,但多数现有方法针对特定任务(如多模态、多曝光或多焦点融合),在融合过程中难以有效保留源信息。这主要源于任务专用架构及深层传播导致的源信息退化。为此,我们提出UniFusion,一个实现跨任务泛化的统一图像融合框架。首先,利用DINOv3进行模态一致的特征提取,在多样化输入间建立共享语义空间;其次,为保持各源图像的理解,引入重建对齐损失以维持融合输出与输入间的一致性;最后,采用双层优化策略,解耦并联合优化重建与融合目标,有效缓解二者耦合关系,确保平稳收敛。在多个融合任务上的大量实验表明,UniFusion在视觉质量、泛化能力及真实场景适应性方面均表现优越。代码已公开于https://github.com/dusongcheng/UniFusion。
原文摘要 · Abstract (English)
Image fusion aims to integrate complementary information from multiple source images to produce a more informative and visually consistent representation, benefiting both human perception and downstream vision tasks. Despite recent progress, most existing fusion methods are designed for specific tasks (i.e., multi-modal, multi-exposure, or multi-focus fusion) and struggle to effectively preserve source information during the fusion process. This limitation primarily arises from task-specific architectures and the degradation of source information caused by deep-layer propagation. To overcome these issues, we propose UniFusion, a unified image fusion framework designed to achieve cross-task generalization. First, leveraging DINOv3 for modality-consistent feature extraction, UniFusion establishes a shared semantic space for diverse inputs. Second, to preserve the understanding of each source image, we introduce a reconstruction-alignment loss to maintain consistency between fused outputs and inputs. Finally, we employ a bilevel optimization strategy to decouple and jointly optimize reconstruction and fusion objectives, effectively balancing their coupling relationship and ensuring smooth convergence. Extensive experiments across multiple fusion tasks demonstrate UniFusion's superior visual quality, generalization ability, and adaptability to real-world scenarios. Code is available at https://github.com/dusongcheng/UniFusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。