让野生动物识别模型跨地域通用,提升野外监测效率。
WildIng: A Wildlife Image Invariant Representation Model for Geographical Domain Shift
- 融合图像与文本描述,构建抗地理差异的特征表示
- 在跨区域测试中使基础模型准确率提升30%
- 适合需要跨地域部署的野生动物监测项目
野生动物监测对研究生物多样性丧失和气候变化至关重要。相机陷阱图像为非侵入式分析动物种群和识别生态模式提供了有效手段。然而,人工分析耗时且资源密集。深度学习,尤其是基础模型,已用于自动化野生动物识别,在相同地理区域的数据上表现良好。但这些模型在新地理区域泛化能力差,性能显著下降。例如,在非洲数据集上训练的CLIP+适配器模型准确率达84.77%,但在美国数据集上降至16.17%。这主要因现有模型依赖图像特征,对背景、光照和环境条件变化敏感。为此,我们提出WildIng——一种针对地理域偏移的野生动物图像不变表示模型。通过结合文本描述与图像特征,捕捉物种外观等一致语义信息,增强跨区域泛化能力。实验表明,WildIng在地理域偏移条件下使BioCLIP等基础模型准确率提升30%。我们在美洲和非洲两个不同区域的数据集上进行了评估。代码与模型已公开于https://github.com/Julian075/CATALOG/tree/WildIng。
原文摘要 · Abstract (English)
Wildlife monitoring is crucial for studying biodiversity loss and climate change. Camera trap images provide a non-intrusive method for analyzing animal populations and identifying ecological patterns over time. However, manual analysis is time-consuming and resource-intensive. Deep learning, particularly foundation models, has been applied to automate wildlife identification, achieving strong performance when tested on data from the same geographical locations as their training sets. Yet, despite their promise, these models struggle to generalize to new geographical areas, leading to significant performance drops. For example, training an advanced vision-language model, such as CLIP with an adapter, on an African dataset achieves an accuracy of 84.77%. However, this performance drops significantly to 16.17% when the model is tested on an American dataset. This limitation partly arises because existing models rely predominantly on image-based representations, making them sensitive to geographical data distribution shifts, such as variation in background, lighting, and environmental conditions. To address this, we introduce WildIng, a Wildlife image Invariant representation model for geographical domain shift. WildIng integrates text descriptions with image features, creating a more robust representation to geographical domain shifts. By leveraging textual descriptions, our approach captures consistent semantic information, such as detailed descriptions of the appearance of the species, improving generalization across different geographical locations. Experiments show that WildIng enhances the accuracy of foundation models such as BioCLIP by 30% under geographical domain shift conditions. We evaluate WildIng on two datasets collected from different regions, namely America and Africa. The code and models are publicly available at https://github.com/Julian075/CATALOG/tree/WildIng.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。