不需对齐的RGB引导高光谱图像超分辨,提升纹理与光谱一致性。
Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance
- 分两阶段生成对齐:先用微调光流粗对齐,再用扭曲模型修复纹理
- 引入可学习偏移的迭代可变形特征聚合,增强多尺度融合效果
- 设计光谱注意力模块,提升跨波段信息交互,适合遥感与自然场景
高光谱图像超分辨旨在提升空间分辨率,但在高倍率下性能受限。现有方法依赖高分辨率参考图像,但因对齐不准及对齐与融合模块交互不足,难以有效利用参考图信息。本文提出无对齐参考RGB引导的高光谱超分辨框架SSC-HSR,解决对齐不准与模块间互动弱的问题。为保证空间一致性,构建两阶段图像对齐模块:第一阶段通过微调光流模型生成更准确光流,第二阶段利用扭曲模型修复受损纹理。为增强对齐与融合模块间的交互并保障重建时的光谱一致性,提出特征聚合模块和注意力融合模块。特征聚合模块引入迭代可变形特征聚合块,通过多尺度融合结果引导,迭代生成可学习偏移,实现显著特征匹配与纹理聚合;注意力融合模块包含两种基础光谱注意力块,用于建模波段间交互。在三个自然或遥感数据集上的大量实验表明,本方法在定量与定性评价上均优于现有最优方法。
原文摘要 · Abstract (English)
Hyperspectral images super-resolution aims to improve the spatial resolution, yet its performance is often limited at high-resolution ratios. The recent adoption of high-resolution reference images for super-resolution is driven by the poor spatial detail found in low-resolution HSIs, presenting it as a favorable method. However, these approaches cannot effectively utilize information from the reference image, due to the inaccuracy of alignment and its inadequate interaction between alignment and fusion modules. In this paper, we introduce a Spatial-Spectral Concordance Hyperspectral Super-Resolution (SSC-HSR) framework for unaligned reference RGB guided HSI SR to address the issues of inaccurate alignment and poor interactivity of the previous approaches. Specifically, to ensure spatial concordance, i.e., align images more accurately across resolutions and refine textures, we construct a Two-Stage Image Alignment with a synthetic generation pipeline in the image alignment module, where the fine-tuned optical flow model can produce a more accurate optical flow in the first stage and warp model can refine damaged textures in the second stage. To enhance the interaction between alignment and fusion modules and ensure spectral concordance during reconstruction, we propose a Feature Aggregation module and an Attention Fusion module. In the feature aggregation module, we introduce an Iterative Deformable Feature Aggregation block to achieve significant feature matching and texture aggregation with the fusion multi-scale results guidance, iteratively generating learnable offset. Besides, we introduce two basic spectral-wise attention blocks in the attention fusion module to model the inter-spectra interactions. Extensive experiments on three natural or remote-sensing datasets show that our method outperforms state-of-the-art approaches on both quantitative and qualitative evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。