用视觉语言模型零样本分类卫星图像，无需标注数据即可准确识别建筑与地物。
GeoVision Labeler: Zero-Shot Geospatial Classification with Vision and Language Models
- 先用视觉大模型生成图像描述，再由语言模型映射到用户自定义类别。
- 在SpaceNet v7上达到93.2%的零样本准确率，多类任务通过递归聚类提升性能。
- 模块化设计可解释性强，适合灾害响应、土地监测等缺乏标注数据的场景。
地理空间图像分类在灾害响应和土地利用监测中仍是瓶颈，尤其在缺乏标注数据的地区。现有工具（如RS-CLIP）虽宣称具备零样本分类能力，但仍需针对特定任务进行预训练和微调才能达到良好性能。本文提出GeoVision Labeler（GVL），一种严格意义上的零样本分类框架：使用视觉大语言模型（vLLM）生成丰富的人类可读图像描述，再由传统大语言模型（LLM）将其映射至用户定义的类别。该模块化、可解释的流程适用于广泛应用场景。我们在三个基准测试中评估了GVL：SpaceNet v7、UC Merced 和 RESISC45。在SpaceNet v7的二分类任务（建筑物 vs. 非建筑物）中，最高达到93.2%的零样本准确率。对于复杂多分类任务（UC Merced、RESISC45），我们采用递归式LLM驱动聚类生成多层语义元类别，并通过分层分类策略——先区分粗粒度组别，再细化差异——实现具有竞争力的零样本性能。GVL已开源，地址为https://github.com/microsoft/geo-vision-labeler，旨在推动其在真实地理空间工作流中的应用。
原文摘要 · Abstract (English)
Classifying geospatial imagery remains a major bottleneck for applications such as disaster response and land-use monitoring-particularly in regions where annotated data is scarce or unavailable. Existing tools (e.g., RS-CLIP) that claim zero-shot classification capabilities for satellite imagery nonetheless rely on task-specific pretraining and adaptation to reach competitive performance. We introduce GeoVision Labeler (GVL), a strictly zero-shot classification framework: a vision Large Language Model (vLLM) generates rich, human-readable image descriptions, which are then mapped to user-defined classes by a conventional Large Language Model (LLM). This modular, and interpretable pipeline enables flexible image classification for a large range of use cases. We evaluated GVL across three benchmarks-SpaceNet v7, UC Merced, and RESISC45. It achieves up to 93.2% zero-shot accuracy on the binary Buildings vs. No Buildings task on SpaceNet v7. For complex multi-class classification tasks (UC Merced, RESISC45), we implemented a recursive LLM-driven clustering to form meta-classes at successive depths, followed by hierarchical classification-first resolving coarse groups, then finer distinctions-to deliver competitive zero-shot performance. GVL is open-sourced at https://github.com/microsoft/geo-vision-labeler to catalyze adoption in real-world geospatial workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。