用植物生态知识增强遥感图像表示,低成本提升生物多样性建模效果
BotaCLIP: Contrastive Learning for Botany-Aware Representation of Earth Observation Data
- 通过对比学习对齐高分辨率航拍图与植物观测数据,注入领域知识
- 在三种生态任务中均优于原始模型和监督基线,提升显著
- 适合生态学、遥感与小样本学习研究者参考
基础模型在图像、文本、音频等多模态数据上展现出强大的可迁移表征能力,常作为下游任务的主要输入。本文针对如何低成本引入领域知识的问题,提出BotaCLIP——一种轻量级多模态对比学习框架,通过将高分辨率航空影像与植物实地观测数据(botanical relevés)对齐,适配预训练的地球观测基础模型DOFA。不同于通用嵌入,BotaCLIP利用正则化策略缓解灾难性遗忘,内化生态结构信息。训练完成后,生成的嵌入可作为下游预测器的可迁移表征。基于真实生物多样性建模需求,我们在植物存在预测、蝴蝶出现建模及土壤营养级群丰度估计三个任务中评估了该方法,结果一致优于DOFA及监督基线。本工作展示了领域感知的模型适配如何在数据稀缺场景下注入专家知识,实现高效表征学习。
原文摘要 · Abstract (English)
Foundation models have demonstrated a remarkable ability to learn rich, transferable representations across diverse modalities such as images, text, and audio. In modern machine learning pipelines, these representations often replace raw data as the primary input for downstream tasks. In this paper, we address the challenge of adapting a pre-trained foundation model to inject domain-specific knowledge, without retraining from scratch or incurring significant computational costs. To this end, we introduce BotaCLIP, a lightweight multimodal contrastive framework that adapts a pre-trained Earth Observation foundation model (DOFA) by aligning high-resolution aerial imagery with botanical relevés. Unlike generic embeddings, BotaCLIP internalizes ecological structure through contrastive learning with a regularization strategy that mitigates catastrophic forgetting. Once trained, the resulting embeddings serve as transferable representations for downstream predictors. Motivated by real-world applications in biodiversity modeling, we evaluated BotaCLIP representations in three ecological tasks: plant presence prediction, butterfly occurrence modeling, and soil trophic group abundance estimation. The results showed consistent improvements over those derived from DOFA and supervised baselines. More broadly, this work illustrates how domain-aware adaptation of foundation models can inject expert knowledge into data-scarce settings, enabling frugal representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。