提出生态视觉基准BioBench，替代ImageNet评估科学图像模型性能
BioBench: A Blueprint to Move Beyond ImageNet for Scientific ML Benchmarks
- 构建涵盖9个应用任务的生态视觉基准，覆盖4大生物界与6种成像方式
- 在310万张图像上测试，发现ImageNet预测力不足34%，且排名错误率达30%
- 提供一键下载与评估接口，适合生态、计算机视觉及跨学科研究者使用
ImageNet-1K线性探测迁移精度仍是视觉表征质量的默认指标，但已无法预测科学图像任务表现。在46个现代视觉模型检查点中，ImageNet Top-1准确率仅解释了生态任务34%的方差，且对75%以上准确率的模型有30%出现排名错误。本文提出BioBench，一个开放的生态视觉基准，捕捉ImageNet所忽略的关键信息。该基准整合9个公开发布的应用驱动任务、4个分类阶元和6种采集模态（无人机RGB、网络视频、显微图像、现场与标本照片、相机陷阱帧），共包含310万张图像。通过单一Python API可完成数据下载、冻结主干网络的轻量级分类器训练，并报告类别平衡的宏F1分数（鱼类识别与真菌分类任务额外提供领域指标）。在单块A6000 GPU上，ViT-L模型可在6小时内完成全部评估。BioBench为生态领域计算机视觉提供新信号，并为其他科学领域的可靠基准建设提供模板。代码与结果详见https://github.com/samuelstevens/biobench 和 https://samuelstevens.me/biobench。
原文摘要 · Abstract (English)
ImageNet-1K linear-probe transfer accuracy remains the default proxy for visual representation quality, yet it no longer predicts performance on scientific imagery. Across 46 modern vision model checkpoints, ImageNet top-1 accuracy explains only 34% of variance on ecology tasks and mis-ranks 30% of models above 75% accuracy. We present BioBench, an open ecology vision benchmark that captures what ImageNet misses. BioBench unifies 9 publicly released, application-driven tasks, 4 taxonomic kingdoms, and 6 acquisition modalities (drone RGB, web video, micrographs, in-situ and specimen photos, camera-trap frames), totaling 3.1M images. A single Python API downloads data, fits lightweight classifiers to frozen backbones, and reports class-balanced macro-F1 (plus domain metrics for FishNet and FungiCLEF); ViT-L models evaluate in 6 hours on an A6000 GPU. BioBench provides new signal for computer vision in ecology and a template recipe for building reliable AI-for-science benchmarks in any domain. Code and predictions are available at https://github.com/samuelstevens/biobench and results at https://samuelstevens.me/biobench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。