arXiv:2506.03182cs.CVcs.LG2025-06被引 1

用动态数据集评测AI发现未知昆虫物种的能力,推动生态保育智能化。

TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models

  • 构建混合图像数据集,包含已知与罕见昆虫,模拟真实发现场景。
  • 顶尖模型在目级识别准确率超90%,但种级识别低于2%,凸显细粒度挑战。
  • 支持定期更新,适合研究开放世界识别与生态AI的学者使用。

全球昆虫多样性快速丧失,构成紧迫的生态危机。当前昆虫物种发现依赖人工,速度慢且受限于分类学专家资源,阻碍及时保护行动。我们提出TerraIncognita,一个动态基准,用于评估前沿多模态模型在从图像中识别未知、可能未描述的昆虫物种这一难题上的表现。该基准数据集结合了专家标注的已知昆虫图像与来自未充分探索生物热点地区的稀有、少有公开图像的物种图像,真实模拟生态学家面临的开放世界发现场景。评估内容涵盖层级分类能力、对分布外(OOD)样本(代表新物种)的检测与拒识能力,以及生成符合专家分类知识的解释能力。值得注意的是,顶级模型在目级分类上F1超过90%,但在种级分类上降至2%以下,凸显从粗到细分类任务的陡峭难度梯度(目→科→属→种)。TerraIncognita将定期更新,通过每季度扩展已知与新物种数据,提供长期演进的前沿AI方法评测平台。所有TerraIncognita数据、结果及未来更新均公开可获取。

原文摘要 · Abstract (English)

The rapid global loss of biodiversity, particularly among insects, represents an urgent ecological crisis. Current methods for insect species discovery are manual, slow, and severely constrained by taxonomic expertise, hindering timely conservation actions. We introduce TerraIncognita, a dynamic benchmark designed to evaluate state-of-the-art multimodal models for the challenging problem of identifying unknown, potentially undescribed insect species from image data. Our benchmark dataset combines a mix of expertly annotated images of insect species likely known to frontier AI models, and images of rare and poorly known species, for which few/no publicly available images exist. These images were collected from underexplored biodiversity hotspots, realistically mimicking open-world discovery scenarios faced by ecologists. The benchmark assesses models' proficiency in hierarchical taxonomic classification, their capability to detect and abstain from out-of-distribution (OOD) samples representing novel species, and their ability to generate explanations aligned with expert taxonomic knowledge. Notably, top-performing models achieve over 90\% F1 at the Order level on known species, but drop below 2\% at the Species level, highlighting the sharp difficulty gradient from coarse to fine taxonomic prediction (Order $\rightarrow$ Family $\rightarrow$ Genus $\rightarrow$ Species). TerraIncognita will be updated regularly, and by committing to quarterly dataset expansions (of both known and novel species), will provide an evolving platform for longitudinal benchmarking of frontier AI methods. All TerraIncognita data, results, and future updates are available \href{https://baskargroup.github.io/TerraIncognita/}{here}.

物种发现生态AI多模态开放世界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。