arXiv:2604.24029cs.CVcs.CL2026-04KDD

统一物种识别与发现,通过检索证据进行可解释推理。

DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery

论文配图:DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery
图 1 · 摘自论文原文
  • 基于检索的多模态框架,用链式思考对比候选物种图像。
  • 在多个数据集上同时提升识别与发现准确率,零样本迁移能力强。
  • 适合生物多样性研究者,尤其关注开放世界新物种探测的人。

在数万种外观相似的生物类群中进行物种识别,并在开放世界环境中发现未知物种,仍是生物多样性研究中的核心挑战。现有方法将识别与发现视为独立问题:分类模型假设封闭集,而发现依赖阈值判断。本文提出 DeepTaxon,一种基于检索的多模态统一框架,通过可解释的视觉证据推理实现识别与发现一体化。给定查询图像,DeepTaxon 从检索索引中获取前 $k$ 个候选物种,每个包含 $n$ 张实例图像,并执行链式思维比较推理。关键在于,将发现重新定义为显式的、基于检索的决策问题,而非隐式的参数记忆问题。仅当检索索引缺乏足够识别证据时,样本才被视为新物种,从而无需人工标注即可自动获得分类或发现标签,实现双任务的自监督训练。通过合成检索增强数据进行监督微调,再对困难样本使用强化学习,将高召回检索转化为高精度决策,支持大规模分类词表扩展。在大规模分布内基准及六个分布外数据集上的实验显示,识别与发现性能均持续提升。消融实验表明,测试时候选数 $k$ 与实例数 $n$ 可有效扩展,具备强零样本跨域迁移能力,且对不同检索编码器表现稳定,为生物多样性研究提供可解释的解决方案。

原文摘要 · Abstract (English)

Identifying species in biology among tens of thousands of visually similar taxa while discovering unknown species in open-world environments remains a fundamental challenge in biodiversity research. Current methods treat identification and discovery as separate problems, with classification models assuming closed sets and discovery relying on threshold-based rejection. Here we present DeepTaxon, a retrieval-augmented multimodal framework that unifies species identification and discovery through interpretable reasoning over retrieved visual evidence. Given a query image, DeepTaxon retrieves the top-$k$ candidate species with $n$ exemplar images each from a retrieval index and performs chain-of-thought comparative reasoning. Critically, we redefine discovery as an explicit, retrieval-based decision problem rather than an implicit parametric memory problem. A sample is novel if and only if the retrieval index lacks sufficient evidence for identification, so each retrieval naturally yields a classification or discovery label without manual annotation, thereby providing automatic supervision for both tasks. We train the framework via supervised fine-tuning on synthetic retrieval-augmented data, followed by reinforcement learning on hard samples, converting high-recall retrieval into high-precision decisions that scale to massive taxonomic vocabularies. Extensive experiments on a large-scale in-distribution benchmark and six out-of-distribution datasets demonstrate consistent improvements in both identification and discovery. Ablation studies further reveal effective test-time scaling with candidate count $k$ and exemplar count $n$, strong zero-shot transfer to unseen domains, and consistent performance across retrieval encoders, establishing an interpretable solution for biodiversity research.

物种识别多模态可解释性开放世界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。