arXiv:2411.01573cs.CVcs.LG2024-11NeurIPS被引 26

无需训练即可动态融合多源图像,实时响应不同场景需求。

Conditional Controllable Image Fusion

  • 用预训练扩散模型注入动态约束,实现按需融合
  • 跨场景测试中优于现有方法,无需额外训练
  • 适合快速变化环境中的实时图像融合任务

图像融合旨在整合多源输入图像的互补信息以生成新图像。现有方法通常针对特定场景设计固定约束,难以适应动态环境。为此,我们提出条件可控融合(CCF)框架,无需特定训练即可处理通用融合任务。由于样本间存在动态差异,CCF为每个输入动态选择适配的融合约束。利用去噪扩散模型的强大生成能力,将具体约束注入预训练的DDPM中作为自适应融合条件,并在逆向扩散过程中动态调整条件,实现分步条件校准。大量实验表明,该方法在多种场景下的通用融合任务中均优于对比方法,且无需额外训练。

原文摘要 · Abstract (English)

Image fusion aims to integrate complementary information from multiple input images acquired through various sources to synthesize a new fused image. Existing methods usually employ distinct constraint designs tailored to specific scenes, forming fixed fusion paradigms. However, this data-driven fusion approach is challenging to deploy in varying scenarios, especially in rapidly changing environments. To address this issue, we propose a conditional controllable fusion (CCF) framework for general image fusion tasks without specific training. Due to the dynamic differences of different samples, our CCF employs specific fusion constraints for each individual in practice. Given the powerful generative capabilities of the denoising diffusion model, we first inject the specific constraints into the pre-trained DDPM as adaptive fusion conditions. The appropriate conditions are dynamically selected to ensure the fusion process remains responsive to the specific requirements in each reverse diffusion stage. Thus, CCF enables conditionally calibrating the fused images step by step. Extensive experiments validate our effectiveness in general fusion tasks across diverse scenarios against the competing methods without additional training.

图像融合扩散模型条件生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。