动态融合多模态图像,让彼此增强,效果更优。
Dream-IF: Dynamic Relative EnhAnceMent for Image Fusion
- 根据各模态相对主导区域,动态调整跨模态增强
- 在多个数据集上均超越现有方法,显著提升融合质量
- 适合需要多源图像增强的科研与工业应用
图像融合旨在整合多源图像中的完整信息。然而,不同传感器获取的图像常存在各类退化,影响融合效果。传统方法将图像增强与融合分步处理,忽略了二者内在关联;值得注意的是,融合图像中某一模态的主导区域,往往提示另一模态可从中受益。受此启发,本文提出主导区域增强概念,并构建动态相对增强图像融合框架(Dream-IF)。该框架量化不同层级下各模态的相对主导性,利用此信息实现双向跨模态增强。通过融合过程导出的相对主导性,该方法不仅支持图像复原,还可拓展至更广泛的图像增强任务。此外,采用基于提示的编码机制捕捉退化特异性信息,动态引导修复过程,在多模态图像融合与图像增强场景中实现协同优化。大量实验表明,Dream-IF持续优于现有方法。代码已公开。
原文摘要 · Abstract (English)
Image fusion aims to integrate comprehensive information from images acquired through multiple sources. However, images captured by diverse sensors often encounter various degradations that can negatively affect fusion quality. Traditional fusion methods generally treat image enhancement and fusion as separate processes, overlooking the inherent correlation between them; notably, the dominant regions in one modality of a fused image often indicate areas where the other modality might benefit from enhancement. Inspired by this observation, we introduce the concept of dominant regions for image enhancement and present a Dynamic Relative EnhAnceMent framework for Image Fusion (Dream-IF). This framework quantifies the relative dominance of each modality across different layers and leverages this information to facilitate reciprocal cross-modal enhancement. By integrating the relative dominance derived from image fusion, our approach supports not only image restoration but also a broader range of image enhancement applications. Furthermore, we employ prompt-based encoding to capture degradation-specific details, which dynamically steer the restoration process and promote coordinated enhancement in both multi-modal image fusion and image enhancement scenarios. Extensive experimental results demonstrate that Dream-IF consistently outperforms its counterparts. The code is publicly available.\footnote{ https://github.com/jehovahxu/Dream-IF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。