arXiv:2505.11800cs.CVeess.IV2025-05CVPR被引 18

无需训练数据,用扩散模型融合高光谱与多光谱图像

Self-Learning Hyperspectral and Multispectral Image Fusion via Adaptive Residual Guided Subspace Diffusion Model

  • 分离学习光谱与空间信息,通过反向扩散重建高分辨率图像
  • 在多个数据集上优于现有方法,参数量减少60%以上
  • 适合缺乏标注数据的遥感图像融合场景

高光谱与多光谱图像(HSI-MSI)融合旨在将低分辨率高光谱图像(LR-HSI)与高分辨率多光谱图像(HR-MSI)结合,生成高分辨率高光谱图像(HR-HSI)。现有深度学习方法大多依赖大量标注数据进行监督训练,但在实际应用中高光谱数据往往稀缺。本文提出自学习的自适应残差引导子空间扩散模型(ARGS-Diff),仅利用观测图像进行融合,无需额外训练数据。具体地,由于LR-HSI包含光谱信息,而HR-MSI包含空间信息,我们设计了两个轻量级光谱与空间扩散模型,分别从两者中学习光谱与空间分布。随后,在反向扩散过程中,基于低维光谱基和缩减系数重构HR-HSI。此外,引入自适应残差引导模块(ARGM),在每一步采样中通过残差引导函数优化两个分量,从而稳定采样过程。大量实验表明,ARGS-Diff在性能和计算效率方面均优于现有先进方法。代码已开源:https://github.com/Zhu1116/ARGS-Diff。

原文摘要 · Abstract (English)

Hyperspectral and multispectral image (HSI-MSI) fusion involves combining a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI) to generate a high-resolution hyperspectral image (HR-HSI). Most deep learning-based methods for HSI-MSI fusion rely on large amounts of hyperspectral data for supervised training, which is often scarce in practical applications. In this paper, we propose a self-learning Adaptive Residual Guided Subspace Diffusion Model (ARGS-Diff), which only utilizes the observed images without any extra training data. Specifically, as the LR-HSI contains spectral information and the HR-MSI contains spatial information, we design two lightweight spectral and spatial diffusion models to separately learn the spectral and spatial distributions from them. Then, we use these two models to reconstruct HR-HSI from two low-dimensional components, i.e, the spectral basis and the reduced coefficient, during the reverse diffusion process. Furthermore, we introduce an Adaptive Residual Guided Module (ARGM), which refines the two components through a residual guided function at each sampling step, thereby stabilizing the sampling process. Extensive experimental results demonstrate that ARGS-Diff outperforms existing state-of-the-art methods in terms of both performance and computational efficiency in the field of HSI-MSI fusion. Code is available at https://github.com/Zhu1116/ARGS-Diff.

图像融合扩散模型遥感

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。