arXiv:2501.16222cs.CV2025-01被引 25

用CLIP零样本实现高光谱图像分类,无需人工标注

SPECIAL: Zero-shot Hyperspectral Image Classification With CLIP

  • 用CLIP对高光谱数据生成带置信度的伪标签
  • 多尺度融合提升伪标签质量,准确率在三个数据集上领先
  • 适合无标注数据场景,如遥感监测与农业应用

高光谱图像分类旨在将每个像素归类为特定地物类别,对遥感、环境监测和农业至关重要。尽管深度学习方法已取得显著进展,但现有方法仍依赖人工标注数据,耗时费力。为此,我们提出基于CLIP的零样本高光谱图像分类框架SPECIAL,旨在消除人工标注需求。SPECIAL包含两个阶段:(1) 基于CLIP的伪标签生成,将高光谱数据进行光谱插值生成RGB波段,通过CLIP分类得到带置信度的噪声伪标签;提出多空间尺度融合策略提升标签质量;(2) 结合光谱信息与标签精炼技术,缓解标签噪声,进一步提高分类精度。在三个基准数据集上的实验表明,SPECIAL在零样本高光谱分类中优于现有方法,展现出更强实用性。代码已开源。

原文摘要 · Abstract (English)

Hyperspectral image (HSI) classification aims to categorize each pixel in an HSI into a specific land cover class, which is crucial for applications such as remote sensing, environmental monitoring, and agriculture. Although deep learning-based HSI classification methods have achieved significant advancements, existing methods still rely on manually labeled data for training, which is both time-consuming and labor-intensive. To address this limitation, we introduce a novel zero-shot hyperspectral image classification framework based on CLIP (SPECIAL), aiming to eliminate the need for manual annotations. The SPECIAL framework consists of two main stages: (1) CLIP-based pseudo-label generation, and (2) noisy label learning. In the first stage, HSI is spectrally interpolated to produce RGB bands. These bands are subsequently classified using CLIP, resulting in noisy pseudo-labels that are accompanied by confidence scores. To improve the quality of these labels, we propose a scaling strategy that fuses predictions from multiple spatial scales. In the second stage, spectral information and a label refinement technique are incorporated to mitigate label noise and further enhance classification accuracy. Experimental results on three benchmark datasets demonstrate that our SPECIAL outperforms existing methods in zero-shot HSI classification, showing its potential for more practical applications. The code is available at https://github.com/LiPang/SPECIAL.

高光谱分类零样本学习CLIP遥感

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。