无需标注数据,用深度张量分解实现高光谱与多光谱图像融合。
Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning
- 通过共享参数解码器将低分辨率高光谱图与高分辨率多光谱图映射到统一特征空间。
- 引入空间-光谱注意力机制和拉普拉斯流形约束,提升跨模态特征对齐能力。
- 在多个遥感数据集上表现优异,适合无监督遥感图像融合任务。
高光谱与多光谱图像融合旨在通过融合高分辨率多光谱图像(HR-MSI)和低分辨率高光谱图像(LR-HSI),生成高光谱与高空间分辨率的高光谱图像(HR-HSI)。现有方法面临退化参数未知、高维结构与深层图像特征关联利用不充分等挑战。为此,本文提出一种基于张量分解与空间-光谱流形学习的无监督盲融合方法(DTDNML)。设计了一种新型深度张量分解网络,将LR-HSI与HR-MSI映射至一致特征空间,并通过共享参数解码器实现重建。为更好挖掘与融合数据中的空间-光谱特征,引入核心张量融合网络及空间-光谱注意力机制,实现多尺度特征对齐与融合。此外,在共享解码器中引入基于拉普拉斯的空间-光谱流形约束,增强全局信息捕捉能力。大量实验验证了该方法在不同遥感数据集上的融合精度与效率优势。代码已公开于 https://github.com/Shawn-H-Wang/DTDNML。
原文摘要 · Abstract (English)
Hyperspectral and multispectral image fusion aims to generate high spectral and spatial resolution hyperspectral images (HR-HSI) by fusing high-resolution multispectral images (HR-MSI) and low-resolution hyperspectral images (LR-HSI). However, existing fusion methods encounter challenges such as unknown degradation parameters, incomplete exploitation of the correlation between high-dimensional structures and deep image features. To overcome these issues, in this article, an unsupervised blind fusion method for hyperspectral and multispectral images based on Tucker decomposition and spatial spectral manifold learning (DTDNML) is proposed. We design a novel deep Tucker decomposition network that maps LR-HSI and HR-MSI into a consistent feature space, achieving reconstruction through decoders with shared parameter. To better exploit and fuse spatial-spectral features in the data, we design a core tensor fusion network that incorporates a spatial spectral attention mechanism for aligning and fusing features at different scales. Furthermore, to enhance the capacity in capturing global information, a Laplacian-based spatial-spectral manifold constraints is introduced in shared-decoders. Sufficient experiments have validated that this method enhances the accuracy and efficiency of hyperspectral and multispectral fusion on different remote sensing datasets. The source code is available at https://github.com/Shawn-H-Wang/DTDNML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。