无需标注数据，通过解耦多模态信息提升高光谱图像超分辨率质量
Unsupervised Hyperspectral Image Super-Resolution via Self-Supervised Modality Decoupling
- 提出自监督解耦框架，分离跨模态共享与互补特征
- 在多个数据集上超越现有方法，参数更少、推理更快
- 适合遥感图像处理、无监督图像融合方向的研究者
基于融合的高光谱图像超分辨率旨在融合低分辨率高光谱图像（LR-HSIs）和高分辨率多光谱图像（HR-MSIs），以重建高空间与高光谱分辨率图像。现有方法通常直接融合两模态数据而缺乏有效监督，导致对深层模态互补信息感知不全，对跨模态相关性理解有限。为此，本文提出一种简单有效的无监督高光谱-多光谱图像融合（HMIF）方法，揭示模态解耦是提升融合性能的关键。具体地，设计端到端自监督的模态解耦时空融合框架（MossFuse），分离两模态间的共享与互补信息，并聚合紧凑表示以降低模态冗余。同时引入子空间聚类损失，明确指导共享特征与互补特征的解耦。在多个数据集上的系统实验表明，该方法显著优于现有技术，且参数量更少、推理时间更短。
原文摘要 · Abstract (English)
Fusion-based hyperspectral image super-resolution aims to fuse low-resolution hyperspectral images (LR-HSIs) and high-resolution multispectral images (HR-MSIs) to reconstruct high spatial and high spectral resolution images. Current methods typically apply direct fusion from the two modalities without effective supervision, leading to an incomplete perception of deep modality-complementary information and a limited understanding of inter-modality correlations. To address these issues, we propose a simple yet effective solution for unsupervised HMIF, revealing that modality decoupling is key to improving fusion performance. Specifically, we propose an end-to-end self-supervised Modality-Decoupled Spatial-Spectral Fusion (MossFuse) framework that decouples shared and complementary information across modalities and aggregates a concise representation of both LR-HSIs and HR-MSIs to reduce modality redundancy. Also, we introduce the subspace clustering loss as a clear guide to decouple modality-shared features from modality-complementary ones. Systematic experiments over multiple datasets demonstrate that our simple and effective approach consistently outperforms the existing HMIF methods while requiring considerably fewer parameters with reduced inference time. The source source code is in \href{https://github.com/dusongcheng/MossFuse}{MossFuse}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。