用混合知识蒸馏与谱重建提升高通道遥感图像还原效果
HYDRA: HYbrid knowledge Distillation and spectral Reconstruction Algorithm for high channel hyperspectral camera applications
- 结合教师模型编码高光谱数据与学生模型映射自然图像
- 在多通道场景下实现18%准确率提升，推理速度更快
- 适合遥感、医学成像等需要高精度光谱还原的应用
高光谱图像（HSI）为计算机视觉带来新应用前景。近期研究探索了通用光谱重建（SR）的可行性，即从自然三通道彩色图像中恢复高光谱图像。然而，现有基于多尺度注意力（MSA）的方法仅在极稀疏光谱下表现良好，而现代高光谱传感器包含数百个波段。本文提出新型光谱重建方法：HYDRA（HYbrid knowledge Distillation and spectral Reconstruction Architecture）。通过一个封装潜在高光谱数据的教师模型，以及学习从自然图像到教师编码空间映射的学生模型，并结合新颖训练策略，实现高质量光谱重建。该方法克服了以往模型的关键局限，在所有指标上均达到当前最优性能，包括准确率提升18%，且在不同通道深度下推理速度优于现有SOTA模型。
原文摘要 · Abstract (English)
Hyperspectral images (HSI) promise to support a range of new applications in computer vision. Recent research has explored the feasibility of generalizable Spectral Reconstruction (SR), the problem of recovering a HSI from a natural three-channel color image in unseen scenarios. However, previous Multi-Scale Attention (MSA) works have only demonstrated sufficient generalizable results for very sparse spectra, while modern HSI sensors contain hundreds of channels. This paper introduces a novel approach to spectral reconstruction via our HYbrid knowledge Distillation and spectral Reconstruction Architecture (HYDRA). Using a Teacher model that encapsulates latent hyperspectral image data and a Student model that learns mappings from natural images to the Teacher's encoded domain, alongside a novel training method, we achieve high-quality spectral reconstruction. This addresses key limitations of prior SR models, providing SOTA performance across all metrics, including an 18\% boost in accuracy, and faster inference times than current SOTA models at various channel depths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。