用退化与语义双重先验，提升恶劣条件下图像融合效果。
DSPFusion: Image Fusion via Degradation and Semantic Dual-Prior Guidance
- 分离提取退化与语义先验，联合优化融合过程。
- 修复速度比主流扩散模型快20倍以上，计算开销极低。
- 适合低质量图像融合场景，如夜间、雾天视觉任务。
现有融合方法多针对高质量图像设计，难以应对恶劣环境下捕获的退化图像，限制了实际应用。本文提出一种退化与语义双重先验引导的图像融合框架（DSPFusion），利用退化先验和通过扩散模型恢复的高质量场景语义先验，在统一模型中协同指导信息恢复与融合。具体而言，先分别提取模态特定的退化先验，同时联合捕捉全面的低质量语义先验；随后构建扩散模型，在紧凑潜在空间中迭代恢复高质量语义先验，使本方法比主流基于扩散模型的融合方案快20倍以上；最后，通过双先验引导模块和先验引导融合模块，结合退化先验与高质量语义先验，实现信息增强与特征聚合。大量实验表明，DSPFusion能有效缓解典型退化问题，整合互补上下文信息，且计算成本极低，显著拓展了图像融合的应用范围。
原文摘要 · Abstract (English)
Existing fusion methods are tailored for high-quality images but struggle with degraded images captured under harsh circumstances, thus limiting the practical potential of image fusion. This work presents a \textbf{D}egradation and \textbf{S}emantic \textbf{P}rior dual-guided framework for degraded image \textbf{Fusion} (\textbf{DSPFusion}), utilizing degradation priors and high-quality scene semantic priors restored via diffusion models to guide both information recovery and fusion in a unified model. In specific, it first individually extracts modality-specific degradation priors, while jointly capturing comprehensive low-quality semantic priors. Subsequently, a diffusion model is developed to iteratively restore high-quality semantic priors in a compact latent space, enabling our method to be over $20 \times$ faster than mainstream diffusion model-based image fusion schemes. Finally, the degradation priors and high-quality semantic priors are employed to guide information enhancement and aggregation via the dual-prior guidance and prior-guided fusion modules. Extensive experiments demonstrate that DSPFusion mitigates most typical degradations while integrating complementary context with minimal computational cost, greatly broadening the application scope of image fusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。