arXiv:2602.17799cs.CV2026-02被引 2

不训练模型,仅用现成大模型实现遥感图像文本分割

Enabling Training-Free Text-Based Remote Sensing Segmentation

  • 用CLIP选掩码+SAM生成,零样本完成遥感语义分割
  • 在19个遥感数据集上达到当前最佳性能,支持开放词汇和指代分割
  • 无需训练或仅微调少量参数,适合快速部署到新场景

视觉语言模型(VLM)和视觉基础模型(VFM)的进展为遥感影像的零样本文本引导分割提供了新可能。然而,多数方法仍依赖额外可训练组件,限制了泛化能力和实际应用。本文研究是否可在不添加训练的情况下,仅依靠现有基础模型实现文本驱动的遥感分割。提出一种简单有效的方法,将对比式与生成式VLM结合Segment Anything Model(SAM),构建完全无训练或轻量级LoRA微调的流程。对比方法使用CLIP作为掩码选择器,对SAM的网格提议进行筛选,在完全零样本设置下实现最先进的开放词汇语义分割(OVSS)。生成方法则通过GPT-5生成点击提示供SAM使用,或在零样本下微调Qwen-VL模型,后者表现最佳。在19个遥感基准测试中,涵盖开放词汇、指代和推理任务,充分验证了该方法的强大能力。代码将开源于https://github.com/josesosajs/trainfree-rs-segmentation。

原文摘要 · Abstract (English)

Recent advances in Vision Language Models (VLMs) and Vision Foundation Models (VFMs) have opened new opportunities for zero-shot text-guided segmentation of remote sensing imagery. However, most existing approaches still rely on additional trainable components, limiting their generalisation and practical applicability. In this work, we investigate to what extent text-based remote sensing segmentation can be achieved without additional training, by relying solely on existing foundation models. We propose a simple yet effective approach that integrates contrastive and generative VLMs with the Segment Anything Model (SAM), enabling a fully training-free or lightweight LoRA-tuned pipeline. Our contrastive approach employs CLIP as mask selector for SAM's grid-based proposals, achieving state-of-the-art open-vocabulary semantic segmentation (OVSS) in a completely zero-shot setting. In parallel, our generative approach enables reasoning and referring segmentation by generating click prompts for SAM using GPT-5 in a zero-shot setting and a LoRA-tuned Qwen-VL model, with the latter yielding the best results. Extensive experiments across 19 remote sensing benchmarks, including open-vocabulary, referring, and reasoning-based tasks, demonstrate the strong capabilities of our approach. Code will be released at https://github.com/josesosajs/trainfree-rs-segmentation.

遥感分割零样本文本引导SAM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。