arXiv:2503.09131cs.CV2025-03ICCV被引 13

用多提示框架统一修复各类高光谱图像退化问题。

MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration

  • 引入视觉、文本与光谱提示协同建模退化特征。
  • 在9个任务中均优于现有通用与专用方法。
  • 适合处理真实场景下复杂未知退化的高光谱图像。

高光谱图像(HSI)在成像过程中常遭受多种未知退化,导致严重的光谱与空间失真。现有恢复方法通常依赖特定退化假设,限制了其在复杂场景中的效果。本文提出MP-HSIR,一种新型多提示框架,通过融合光谱、文本与视觉提示,实现跨多种退化类型与强度的通用高光谱图像恢复。具体地,设计了提示引导的空间-光谱变换器,结合空间自注意力与提示引导的双分支光谱自注意力。由于退化对光谱特征影响各异,局部光谱分支引入光谱提示,提供通用低秩光谱模式作为先验以增强重建。同时,文本-视觉协同提示融合高层语义与细粒度视觉特征,编码退化信息以指导恢复过程。在9个高光谱图像恢复任务上的实验表明,包括全任务整合、泛化测试与真实场景案例,MP-HSIR不仅持续优于现有全任务方法,还在多个任务上超越最先进的专用方法。代码与模型已开源于https://github.com/ZhehuiWu/MP-HSIR。

原文摘要 · Abstract (English)

Hyperspectral images (HSIs) often suffer from diverse and unknown degradations during imaging, leading to severe spectral and spatial distortions. Existing HSI restoration methods typically rely on specific degradation assumptions, limiting their effectiveness in complex scenarios. In this paper, we propose \textbf{MP-HSIR}, a novel multi-prompt framework that effectively integrates spectral, textual, and visual prompts to achieve universal HSI restoration across diverse degradation types and intensities. Specifically, we develop a prompt-guided spatial-spectral transformer, which incorporates spatial self-attention and a prompt-guided dual-branch spectral self-attention. Since degradations affect spectral features differently, we introduce spectral prompts in the local spectral branch to provide universal low-rank spectral patterns as prior knowledge for enhancing spectral reconstruction. Furthermore, the text-visual synergistic prompt fuses high-level semantic representations with fine-grained visual features to encode degradation information, thereby guiding the restoration process. Extensive experiments on 9 HSI restoration tasks, including all-in-one scenarios, generalization tests, and real-world cases, demonstrate that MP-HSIR not only consistently outperforms existing all-in-one methods but also surpasses state-of-the-art task-specific approaches across multiple tasks. The code and models are available at https://github.com/ZhehuiWu/MP-HSIR.

高光谱图像多提示图像修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。