用多头视觉变换器提升植被图像中多物种识别准确率
Multi-Label Plant Species Prediction with Metadata-Enhanced Multi-Head Vision Transformers
- 基于DINOv2骨干网络，设计物种/属/科多头分类架构
- 在140万张图像上训练，7806个物种，私榜排名第三
- 结合分块、动态阈值和集成策略，有效应对复杂场景
我们提出一种多头视觉变换器方法，用于植被样方图像中的多标签植物物种预测，以应对PlantCLEF 2025挑战。该任务需在单物种植物图像上训练模型，却在包含多个物种的样方图像上测试，造成显著领域差异。方法采用预训练的DINOv2 ViT-B/14骨干网络，配备多个分类头，分别进行物种、属、科预测，并利用分类学层级结构。关键创新包括多尺度分块以捕捉不同尺度的植物、基于平均预测长度的动态阈值优化，以及通过袋装法和Hydra模型架构实现的集成策略。还引入图像裁剪去除非植物干扰物，使用top-n过滤约束预测结果，及对数几率阈值策略。实验基于约140万张训练图像，涵盖7,806种植物。结果表现优异，使我们的提交在私榜中位列第三。代码已公开于https://github.com/geranium12/plant-clef-2025/tree/v1.0.0。
原文摘要 · Abstract (English)
We present a multi-head vision transformer approach for multi-label plant species prediction in vegetation plot images, addressing the PlantCLEF 2025 challenge. The task involves training models on single-species plant images while testing on multi-species quadrat images, creating a drastic domain shift. Our methodology leverages a pre-trained DINOv2 Vision Transformer Base (ViT-B/14) backbone with multiple classification heads for species, genus, and family prediction, utilizing taxonomic hierarchies. Key contributions include multi-scale tiling to capture plants at different scales, dynamic threshold optimization based on mean prediction length, and ensemble strategies through bagging and Hydra model architectures. The approach incorporates various inference techniques including image cropping to remove non-plant artifacts, top-n filtering for prediction constraints, and logit thresholding strategies. Experiments were conducted on approximately 1.4 million training images covering 7,806 plant species. Results demonstrate strong performance, making our submission 3rd best on the private leaderboard. Our code is available at https://github.com/geranium12/plant-clef-2025/tree/v1.0.0.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。