用物种观测和维基文本弱监督，让遥感图像理解生态特征
EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia
- 用物种栖息地描述对齐遥感图像，实现生态表征学习
- 在EUNIS生态分类任务上达成零样本识别效果，验证方法有效性
- 适合生态遥感、多模态学习研究者参考
物种的存在为理解地点的生态属性（如土地覆盖、气候条件或土壤特性）提供了关键线索。我们提出一种直接从遥感（RS）图像预测生态属性的方法，通过将图像与物种栖息地描述对齐。构建了EcoWikiRS数据集，包含高分辨率航空影像、对应的地理定位物种观测记录，以及每种物种在维基百科中的栖息地文本描述。该数据集为遥感视觉语言模型（RS-VLMs）在生态学领域的训练提供了一种可扩展的弱监督方式。此设定下存在弱且嘈杂的监督信号，例如部分文本仅描述物种生态位的局部特征或与特定图像无关。为此，我们提出WINCEL——一种加权版InfoNCE损失函数。在遵循欧洲自然信息系统（EUNIS）栖息地定义的生态系统零样本分类任务上评估模型，结果表明该方法能更生态有意义地理解遥感图像。代码与数据集已开源。
原文摘要 · Abstract (English)
The presence of species provides key insights into the ecological properties of a location such as land cover, climatic conditions or even soil properties. We propose a method to predict such ecological properties directly from remote sensing (RS) images by aligning them with species habitat descriptions. We introduce the EcoWikiRS dataset, consisting of high-resolution aerial images, the corresponding geolocated species observations, and, for each species, the textual descriptions of their habitat from Wikipedia. EcoWikiRS offers a scalable way of supervision for RS vision language models (RS-VLMs) for ecology. This is a setting with weak and noisy supervision, where, for instance, some text may describe properties that are specific only to part of the species' niche or is irrelevant to a specific image. We tackle this by proposing WINCEL, a weighted version of the InfoNCE loss. We evaluate our model on the task of ecosystem zero-shot classification by following the habitat definitions from the European Nature Information System (EUNIS). Our results show that our approach helps in understanding RS images in a more ecologically meaningful manner. The code and the dataset are available at https://github.com/eceo-epfl/EcoWikiRS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。