用类别原型指导分割模型,实现高分辨率植物图像的多标签识别。
Zero-Shot Segmentation through Prototype-Guidance for Multi-Label Plant Species Identification
- 通过聚类训练集图像生成类别原型,作为测试图像分割的引导信号。
- 在PlantCLEF 2025挑战中取得F1分数0.33331,排名第五。
- 适用于高分辨率植被图中的细粒度多物种识别任务。
本文针对PlantClef 2025挑战提出一种方法,解决高分辨率图像下的细粒度多标签植物物种识别问题。该方法利用训练集中的类别原型作为指导,训练一个基于视觉变换器(ViT)的分割模型,使其在测试图像上重建这些原型。具体地,通过K-Means聚类(K等于类别数)从训练图像提取特征并构建原型。分割模型采用定制化的窄版ViT,其嵌入层由在训练集上预训练的DinoV2冻结模块替代。模型在测试集上学习从图像重构训练集原型,进而通过注意力得分定位感兴趣区域,辅助分类。该方法实现了从单物种分类到多标签分类的域自适应。在私有排行榜上获得第五名,F1得分为0.33331,较第一名低0.03,表明具有竞争力。代码已开源。
原文摘要 · Abstract (English)
This paper presents an approach developed to address the PlantClef 2025 challenge, which consists of a fine-grained multi-label species identification, over high-resolution images. Our solution focused on employing class prototypes obtained from the training dataset as a proxy guidance for training a segmentation Vision Transformer (ViT) on the test set images. To obtain these representations, the proposed method extracts features from training dataset images and create clusters, by applying K-Means, with $K$ equals to the number of classes in the dataset. The segmentation model is a customized narrow ViT, built by replacing the patch embedding layer with a frozen DinoV2, pre-trained on the training dataset for individual species classification. This model is trained to reconstruct the class prototypes of the training dataset from the test dataset images. We then use this model to obtain attention scores that enable to identify and localize areas of interest and consequently guide the classification process. The proposed approach enabled a domain-adaptation from multi-class identification with individual species, into multi-label classification from high-resolution vegetation plots. Our method achieved fifth place in the PlantCLEF 2025 challenge on the private leaderboard, with an F1 score of 0.33331. Besides that, in absolute terms our method scored 0.03 lower than the top-performing submission, suggesting that it may achieved competitive performance in the benchmark task. Our code is available at \href{https://github.com/ADAM-UEFS/PlantCLEF2025}{https://github.com/ADAM-UEFS/PlantCLEF2025}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。