arXiv:2501.01481eess.IVcs.CV2025-01被引 3

通过建模光谱相关性与连续性,提升从RGB图像重建高光谱图像的质量。

Unleashing Correlation and Continuity for Hyperspectral Reconstruction from RGB Images

  • 引入局部相关性与全局连续性建模模块,捕捉光谱特征关系
  • 在NTIRE2020和NTIRE2022数据集上达到当前最优性能
  • 适合关注高光谱图像重建与跨模态融合的研究者

从RGB图像重建高光谱图像(HSI)可在较低成本下获得高空间分辨率的HSI,具有重要应用潜力。本文揭示了光谱特性的局部相关性与全局连续性对重建任务至关重要。为此,我们提出了一种相关性与连续性网络(CCNet)用于从RGB图像重建HSI。针对局部光谱相关性,设计了分组光谱相关性建模(GrSCM)模块,高效建立局部范围内光谱波段的相似性;针对全局光谱连续性,设计了邻域光谱连续性建模(NeSCM)模块,利用记忆单元递归建模全局范围内的渐进变化特性。为探索两模块的内在互补性,设计了逐块自适应融合(PAF)模块,以逐块自适应方式将全局连续性特征融入光谱特征中。在主流数据集NTIRE2020和NTIRE2022上进行充分对比与消融实验,所提方法相比现有先进算法实现当前最优(SOTA)性能。

原文摘要 · Abstract (English)

Reconstructing Hyperspectral Images (HSI) from RGB images can yield high spatial resolution HSI at a lower cost, demonstrating significant application potential. This paper reveals that local correlation and global continuity of the spectral characteristics are crucial for HSI reconstruction tasks. Therefore, we fully explore these inter-spectral relationships and propose a Correlation and Continuity Network (CCNet) for HSI reconstruction from RGB images. For the correlation of local spectrum, we introduce the Group-wise Spectral Correlation Modeling (GrSCM) module, which efficiently establishes spectral band similarity within a localized range. For the continuity of global spectrum, we design the Neighborhood-wise Spectral Continuity Modeling (NeSCM) module, which employs memory units to recursively model the progressive variation characteristics at the global level. In order to explore the inherent complementarity of these two modules, we design the Patch-wise Adaptive Fusion (PAF) module to efficiently integrate global continuity features into the spectral features in a patch-wise adaptive manner. These innovations enhance the quality of reconstructed HSI. We perform comprehensive comparison and ablation experiments on the mainstream datasets NTIRE2022 and NTIRE2020 for the spectral reconstruction task. Compared to the current advanced spectral reconstruction algorithms, our designed algorithm achieves State-Of-The-Art (SOTA) performance.

高光谱重建光谱建模多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。