通过跨团队数据采集,揭示农业视觉模型在真实场景下的泛化瓶颈。
AgrI Challenge: A Data-Centric AI Competition for Cross-Team Validation in Agricultural Vision
- 各团队独立采集数据,构建多样化的田间图像基准集。
- 单源训练下准确率高但跨队测试时差距达16.20%,多源协同训练可将差距降至1.78%。
- 适合关注数据质量与模型泛化能力的研究者,尤其重视实地验证的农业AI项目。
农业视觉中的机器学习模型常在精心筛选的数据集上表现优异,但在真实田间环境中因训练与部署环境分布差异而失效。多数竞赛聚焦模型设计,将数据视为固定资源,忽视数据采集方式对泛化的影响。我们提出AgrI Challenge,一个以数据为中心的竞赛框架:多个团队独立采集田间数据,形成反映实际采集条件差异的异构多源基准集。为系统评估跨域泛化能力,提出跨团队验证(CTV)机制,将每支团队的数据视为独立领域。CTV包含两种互补协议:仅使用单队数据训练(TOTO),衡量单源泛化;留出一队数据进行训练(LOTO),评估多源协作效果。实验显示,单源训练下模型验证准确率接近完美,但在其他团队数据上测试时,准确率下降高达16.20%(DenseNet121)和11.37%(Swin Transformer)。而多源协同训练显著提升鲁棒性,使差距缩小至2.82%和1.78%。挑战还生成了一个包含50,673张图像、涵盖六种树种的公开数据集,由十二支独立团队完成采集,为研究农业视觉中的领域偏移与数据驱动学习提供多样化基准。
原文摘要 · Abstract (English)
Machine learning models in agricultural vision often achieve high accuracy on curated datasets but fail to generalize under real field conditions due to distribution shifts between training and deployment environments. Moreover, most machine learning competitions focus primarily on model design while treating datasets as fixed resources, leaving the role of data collection practices in model generalization largely unexplored. We introduce the AgrI Challenge, a data-centric competition framework in which multiple teams independently collect field datasets, producing a heterogeneous multi-source benchmark that reflects realistic variability in acquisition conditions. To systematically evaluate cross-domain generalization across independently collected datasets, we propose Cross-Team Validation (CTV), an evaluation paradigm that treats each team's dataset as a distinct domain. CTV includes two complementary protocols: Train-on-One-Team-Only (TOTO), which measures single-source generalization, and Leave-One-Team-Out (LOTO), which evaluates collaborative multi-source training. Experiments reveal substantial generalization gaps under single-source training: models achieve near-perfect validation accuracy yet exhibit validation-test gaps of up to 16.20% (DenseNet121) and 11.37% (Swin Transformer) when evaluated on datasets collected by other teams. In contrast, collaborative multi-source training dramatically improves robustness, reducing the gap to 2.82% and 1.78%, respectively. The challenge also produced a publicly available dataset of 50,673 field images of six tree species collected by twelve independent teams, providing a diverse benchmark for studying domain shift and data-centric learning in agricultural vision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。