用图像块关联增强遥感零样本分类，不需标注且开销小
Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification
- 基于文本提示和图像编码器的块间关联，实现无监督的传导推理
- 在10个遥感数据集上显著提升零样本分类准确率
- 适合遥感图像分析与少样本场景应用的研究者
遥感视觉-语言模型凭借大规模预训练展现出广泛应用前景。然而，其传统的零样本场景分类方法仍采用将大图像分割为块并独立预测的方式（归纳推理），忽略了重要的上下文信息，限制了性能表现。本文提出一种新方法：利用文本提示生成的初始预测与图像编码器提取的块间亲和关系，通过无监督的传导推理增强零样本能力，计算开销极低。在10个遥感数据集上，使用当前最先进的视觉-语言模型进行实验，均显著优于传统归纳推理方法。代码已开源：https://github.com/elkhouryk/RS-TransCLIP
原文摘要 · Abstract (English)
Vision-Language Models for remote sensing have shown promising uses thanks to their extensive pretraining. However, their conventional usage in zero-shot scene classification methods still involves dividing large images into patches and making independent predictions, i.e., inductive inference, thereby limiting their effectiveness by ignoring valuable contextual information. Our approach tackles this issue by utilizing initial predictions based on text prompting and patch affinity relationships from the image encoder to enhance zero-shot capabilities through transductive inference, all without the need for supervision and at a minor computational cost. Experiments on 10 remote sensing datasets with state-of-the-art Vision-Language Models demonstrate significant accuracy improvements over inductive zero-shot classification. Our source code is publicly available on Github: https://github.com/elkhouryk/RS-TransCLIP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。