让传统模型适应未知模糊,自监督实现盲超分辨融合
Degradation Alchemy: Self-Supervised Unknown-to-Known Transformation for Blind Hyperspectral Image Fusion
- 自监督构建未知到已知的退化转换框架
- 提升5个现成模型在多种模糊下的泛化能力
- 适合处理未标注、未知退化的超光谱图像融合
超光谱图像(HSI)融合通过结合低分辨率超光谱图(LR-HSI)和高分辨率多光谱图(HR-MSI),生成高分辨率超光谱图(HR-HSI)。现有监督学习方法在测试数据退化与训练一致时表现良好,但在未知退化下泛化能力差。为此,本文提出自监督未知到已知退化转换框架(U2K),将未知退化自适应转化为预训练模型可处理的已知退化类型。U2K包含:(1) 空间与光谱退化封装(DW)模块,将真实HR-HSI映射为未知退化的HR-MSI与LR-HSI;(2) 退化转换(DT)模块,将封装后的数据转为预定义退化模式。随后用预训练网络重建目标HR-HSI。通过一致性损失与贪婪交替优化进行自监督训练,显著提升盲融合灵活性。大量实验验证了该框架在多种退化设置下,能有效增强5种现有监督模型的适应性,并优于当前最优盲融合方法。
原文摘要 · Abstract (English)
Hyperspectral image (HSI) fusion is an efficient technique that combines low-resolution HSI (LR-HSI) and high-resolution multispectral images (HR-MSI) to generate high-resolution HSI (HR-HSI). Existing supervised learning methods (SLMs) can yield promising results when test data degradation matches the training ones, but they face challenges in generalizing to unknown degradations. To unleash the potential and generalization ability of SLMs, we propose a novel self-supervised unknown-to-known degradation transformation framework (U2K) for blind HSI fusion, which adaptively transforms unknown degradation into the same type of degradation as those handled by pre-trained SLMs. Specifically, the proposed U2K framework consists of: (1) spatial and spectral Degradation Wrapping (DW) modules that map HR-HSI to unknown degraded HR-MSI and LR-HSI, and (2) Degradation Transformation (DT) modules that convert these wrapped data into predefined degradation patterns. The transformed HR-MSI and LR-HSI pairs are then processed by a pre-trained network to reconstruct the target HR-HSI. We train the U2K framework in a self-supervised manner using consistency loss and greedy alternating optimization, significantly improving the flexibility of blind HSI fusion. Extensive experiments confirm the effectiveness of our proposed U2K framework in boosting the adaptability of five existing SLMs under various degradation settings and surpassing state-of-the-art blind methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。