用混合码本和灰度感知注意力，提升从RGB重建高光谱图的精度与速度
MCGA: Mixture of Codebooks Hyperspectral Reconstruction via Grayscale-Aware Attention
- 设计混合码本学习通用光谱先验，结合灰度感知注意力对齐特征
- 在多个真实数据集上达到领先性能，推理速度提升4-5倍
- 适合需要高效高精度光谱重建的应用，如遥感、医学成像
从RGB输入重建高光谱图像（HSI）是替代高成本高光谱相机的一种经济方案，但由三个通道还原高维光谱本质为病态问题。现有方法通常使用大型注意力网络直接回归RGB到HSI映射，计算开销大，且仅隐式处理病态性。本文提出MCGA框架，通过异构高光谱数据集学习可迁移的光谱先验（混合码本，MoC），再利用灰度感知光度注意力（GANet）将RGB特征与先验对齐，显式缓解病态性。通过Top-K注意力设计与测试时自适应（TTA）进一步提升效率与鲁棒性。在多个真实世界基准上验证，MCGA实现最先进准确率、强跨数据集泛化能力，且推理速度提升4-5倍。代码将在录用后公开于https://github.com/Fibonaccirabbit/MCGA。
原文摘要 · Abstract (English)
Reconstructing hyperspectral images (HSIs) from RGB inputs provides a cost-effective alternative to hyperspectral cameras, but reconstructing high-dimensional spectra from three channels is inherently ill-posed. Existing methods typically directly regress RGB-to-HSI mappings using large attention networks, which are computationally expensive and handle ill-posedness only implicitly. We propose MCGA, a Mixture-of-Codebooks with Grayscale-aware Attention framework that explicitly addresses these challenges using spectral priors and photometric consistency. MCGA first learns transferable spectral priors via a mixture-of-codebooks (MoC) from heterogeneous HSI datasets, then aligns RGB features with these priors through grayscale-aware photometric attention (GANet). Efficiency and robustness are further improved via top-K attention design and test-time adaptation (TTA). Experiments on multiple real-world benchmarks demonstrate the state-of-the-art accuracy, strong cross-dataset generalization, and 4-5x faster inference. Codes will be available once acceptance at https://github.com/Fibonaccirabbit/MCGA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。