无配准高光谱与多光谱图像融合,同时提升空间与光谱分辨率。
Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability
- 耦合光谱解混与潜在空间对抗学习,实现无监督双图像超分辨率
- 理论证明在合理生成模型下可恢复超分辨的多光谱与高光谱图像
- 适用于缺乏配准数据的真实场景,尤其适合遥感影像处理
本文研究了空间未配准的高光谱图像(HSI)与多光谱图像(MSI)融合问题。由于HSI具有高光谱分辨率但低空间分辨率,而MSI则相反,目标是整合两者互补信息以提升二者分辨率。现有方法多聚焦于MSI超分辨率,忽略对HSI的增强;监督深度学习依赖精确训练数据,难以获取。此外,理论分析主要针对配准情形,未注册融合尚无深入理解。本文提出无监督框架,同步实现MSI与HSI的超分辨率。方法结合耦合光谱解混用于MSI超分辨率,以及潜空间对抗学习用于HSI超分辨率。在合理生成模型下建立了超分辨图像的可恢复性理论保证——据我们所知,这是首个针对未注册融合的理论洞察。实验在半真实和真实数据集上验证,覆盖多种环境条件。
原文摘要 · Abstract (English)
This paper addresses the fusion of a pair of spatially unregistered hyperspectral image (HSI) and multispectral image (MSI) covering roughly overlapping regions. HSIs offer high spectral but low spatial resolution, while MSIs provide the opposite. The goal is to integrate their complementary information to enhance both HSI spatial resolution and MSI spectral resolution. While hyperspectral-multispectral fusion (HMF) has been widely studied, the unregistered setting remains challenging. Many existing methods focus solely on MSI super-resolution, leaving HSI unchanged. Supervised deep learning approaches were proposed for HSI super-resolution, but rely on accurate training data, which is often unavailable. Moreover, theoretical analyses largely address the co-registered case, leaving unregistered HMF poorly understood. In this work, an unsupervised framework is proposed to simultaneously super-resolve both MSI and HSI. The method integrates coupled spectral unmixing for MSI super-resolution with latent-space adversarial learning for HSI super-resolution. Theoretical guarantees on the recoverability of the super-resolution MSI and HSI are established under reasonable generative models -- providing, to our best knowledge, the first such insights for unregistered HMF. The approach is validated on semi-real and real HSI-MSI pairs across diverse conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。