无需训练即可分割多叶植物，利用图文注意力实现零样本层级分割。
Zero-shot Hierarchical Plant Segmentation via Foundation Segmentation Models and Text-to-image Attention
- 结合基础分割模型与视觉语言模型，通过图文注意力推理植物结构。
- 在多物种、多生长阶段数据上超越现有零样本方法，跨域性能优于监督方法。
- 适合无标注数据的植物个体分割任务，尤其适用于新物种快速分析。
基础分割模型可在无需训练的情况下从俯视作物图像中提取叶片实例（即零样本）。然而，对由多个重叠叶片组成的完整植物个体进行分割仍具挑战性，这被称为层级分割任务，通常需标注训练数据，且数据常为特定物种，耗时费力。为此，我们提出ZeroPlantSeg，一种针对莲座型植物个体的零样本分割方法，仅依赖俯视图像。该方法整合基础分割模型以提取叶片实例，并引入视觉语言模型，通过结构推理识别植物个体，全程无需额外训练。在涵盖多种植物物种、生长阶段和拍摄环境的数据集上评估表明，本方法超越现有零样本方法，并实现优于监督方法的跨域性能。代码已开源：https://github.com/JunhaoXing/ZeroPlantSeg。
原文摘要 · Abstract (English)
Foundation segmentation models achieve reasonable leaf instance extraction from top-view crop images without training (i.e., zero-shot). However, segmenting entire plant individuals with each consisting of multiple overlapping leaves remains challenging. This problem is referred to as a hierarchical segmentation task, typically requiring annotated training datasets, which are often species-specific and require notable human labor. To address this, we introduce ZeroPlantSeg, a zero-shot segmentation for rosette-shaped plant individuals from top-view images. We integrate a foundation segmentation model, extracting leaf instances, and a vision-language model, reasoning about plants' structures to extract plant individuals without additional training. Evaluations on datasets with multiple plant species, growth stages, and shooting environments demonstrate that our method surpasses existing zero-shot methods and achieves better cross-domain performance than supervised methods. Implementations are available at https://github.com/JunhaoXing/ZeroPlantSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。