arXiv:2608.14994cs.CV2026-08

无需配准和相机响应函数,用像素顺序无关的矩阵匹配实现RGB转高光谱图像

Registration-Free Hyperspectral Reconstruction from RGB via a Permutation-Invariant Gram-Matrix Principle

  • 利用丰度矩阵的排列不变性,不依赖像素对齐和已知相机响应函数
  • 在随机打乱像素后仍能准确重建,相比现有方法更鲁棒
  • 适用于室内、自然场景和遥感,特别适合传感器差异大的情况

从低分辨率高光谱图像(LR-HSI)和高分辨率彩色图像(HR-RGB)重建空间与光谱分辨率均高的高光谱图像(HR-HSI),通常需要精确配准和已知相机响应函数(CRF)。这些假设在不同传感器间难以满足。本文提出一种排列不变的监督原则:解混后的丰度图的格拉姆矩阵仅取决于物质组成,与像素顺序无关。通过匹配丰度格拉姆矩阵,可在无空间对应关系和无预设CRF的情况下学习RGB到HSI的映射。在对HR-RGB像素进行完全随机排列后,当前最优融合方法性能崩溃,而本方法经逆序重索引后重建结果不变。基于此原理,设计了一个残差光谱超分函数,直接将HR-RGB映射为HR-HSI,无需注册、已知CRF或成对监督。在室内、自然场景和遥感多个基准上,该方法性能接近需假设条件的方法,且在假设被违反时依然稳健。消融实验表明,重建精度对用于匹配格拉姆矩阵的特定偏差形式不敏感,说明性能主要源于排列不变性原则而非损失函数调优。

原文摘要 · Abstract (English)

Reconstructing a spatially and spectrally high-resolution hyperspectral image (HR-HSI) from a low-resolution HSI (LR-HSI) and a high-resolution RGB image (HR-RGB) usually assumes precise registration and a known camera response function (CRF). Both assumptions are difficult to satisfy with different sensors. We remove both through a permutation-invariant supervision principle: the Gram matrix of an unmixed abundance map depends on shared material composition but not on pixel ordering. Matching abundance Gram matrices therefore allows RGB-to-HSI mapping to be learned without spatial correspondence and without a predefined CRF. Under a full random permutation of HR-RGB pixels, a state-of-the-art fusion method collapses, whereas our reconstruction is unchanged after inverse reindexing for evaluation. Building on this principle, a residual spectral super-resolution function maps HR-RGB directly to HR-HSI without registration, known CRF, or paired supervision. Across indoor, natural-scene, and remote-sensing benchmarks, the method achieves accuracy comparable to approaches that require these assumptions while remaining robust when they are violated. Loss ablations further show that reconstruction accuracy is largely insensitive to the specific discrepancy used to match the Gram matrices, indicating that performance arises primarily from the permutation-invariant principle rather than loss tuning.

高光谱重建图像配准排列不变性跨模态生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。