用CLIP自动标注景观图地理信息，提升数据稀疏区空间理解能力。
CLIP the Landscape: Automated Tagging of Crowdsourced Landscape Images
- 融合图像、标题与位置嵌入，提升多标签分类精度
- 在49个标签上实现精确匹配准确率，优于纯图像模型
- 轻量级流程可在普通笔记本训练，适合资源受限场景
我们提出一种基于CLIP的多模态多标签分类器，用于预测来自英国群岛范围内的众包景观图像（Geograph数据集）的地理上下文标签。该方法针对Kaggle竞赛任务，基于Geograph中800万张图片的子集，评估标准严格：需在49个可能标签中实现完全匹配准确率。实验表明，将位置和标题嵌入与图像特征结合，可显著提升分类准确率。我们发布了一个轻量级训练管道，可在普通笔记本上运行，使用预训练的CLIP图像与文本嵌入，搭配简单分类头。生成的标签可支持下游任务，如为GeoAI应用构建地点嵌入，增强数据稀疏区域的空间理解能力。
原文摘要 · Abstract (English)
We present a CLIP-based, multi-modal, multi-label classifier for predicting geographical context tags from landscape photos in the Geograph dataset--a crowdsourced image archive spanning the British Isles, including remote regions lacking POIs and street-level imagery. Our approach addresses a Kaggle competition\footnote{https://www.kaggle.com/competitions/predict-geographic-context-from-landscape-photos} task based on a subset of Geograph's 8M images, with strict evaluation: exact match accuracy is required across 49 possible tags. We show that combining location and title embeddings with image features improves accuracy over using image embeddings alone. We release a lightweight pipeline\footnote{https://github.com/SpaceTimeLab/ClipTheLandscape} that trains on a modest laptop, using pre-trained CLIP image and text embeddings and a simple classification head. Predicted tags can support downstream tasks such as building location embedders for GeoAI applications, enriching spatial understanding in data-sparse regions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。