大模型代理可高效完成生物表型注释,突破人工标注瓶颈。
Frontier LLM-based agents can overcome the ontology curation bottleneck for natural phenotypes

- 用大模型作为智能标注代理,自动处理表型文本
- 五款前沿大模型表现接近人类专家,优于旧版NLP工具
- 适合生物信息学与形态学数据整合研究者使用
将自由文本的表型描述映射到本体术语(即表型注释),是实现跨研究比较形态学数据整合的关键。该过程长期依赖高技能人工专家,难以扩展,成为主要瓶颈。Dahdul等人(2018)建立了涵盖七个系统发育研究的实体-质量(EQ)注释金标准(GS),并用基于本体语义相似性的度量评估了三名人工注释员和语义CharaParser NLP工具;结果显示机器-人类一致性显著低于人与人之间的一致性。本文重新评估该基准,采用来自Anthropic和OpenAI的五款前沿托管大模型,每模型作为一个“代理注释员”,在独立工作空间中获取原始文献PDF、与原人类注释员相同的注释指南、四个项目本体(UBERON、PATO、BSPO、GO)及验证脚本。在相同金标准下评估,所有代理均落在原研究中三名训练有素人类注释员的一致性范围内;表现最好的代理接近但未超越最优人类注释员。在所有四项指标上,代理均显著优于Semantic CharaParser。
原文摘要 · Abstract (English)
Linking free-text phenotype descriptions to ontology terms, typically referred to as phenotype annotation, is essential for the cross-study integration of comparative morphological data. This labor intensive process has heavily relied on highly trained human experts, which makes it challenging to scale and thus a key bottleneck. Dahdul et al. (2018) established a Gold Standard (GS) of Entity-Quality (EQ) annotations across seven phylogenetic studies and used it to evaluate three human curators and the Semantic CharaParser NLP tool with ontology-based semantic similarity metrics; they reported that machine-human consistency was significantly lower than inter-curator (human-human) consistency. Here we revisit that benchmark with five frontier hosted LLMs from Anthropic and OpenAI, each operating as an "agentic curator" within a self-contained workspace that supplies the source publication PDF, the same annotation guide used by the original human curators, the four project ontologies (UBERON, PATO, BSPO, GO), and a validation script. Evaluated against the same Gold Standard, every agent fell within the range of inter-curator variability of the three trained human biocurators of the original study; the best performing agents approached but did not reach the best performing human curator. Agents substantially outperformed Semantic CharaParser on all four metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。