arXiv:2411.16196cs.CVcs.LG2024-11被引 10

无需人工标注,用大模型训练出高效水果检测小模型。

Learn from Foundation Model: Fruit Detection Model without Manual Annotation

  • 用SAM2和OpenCLIP实现零样本分割与分类,跳过标注环节。
  • 通过知识蒸馏使模型更小更快,性能接近有标注训练的模型。
  • 适合农业领域数据少、部署要求高的场景,开源数据集超2.5万张。

大型基础模型的突破使得在数据稀缺领域迁移预训练知识成为可能。农业领域普遍存在数据不足问题。本研究提出一种无需人工标注的框架,从基础模型训练出高效的专用小型检测模型。方法分两步:首先利用SAM2进行图像分割,OpenCLIP实现零样本开放词汇分类,形成分割-描述-匹配(SDM)阶段;其次采用新型知识蒸馏机制,将SDM模型压缩为轻量级、可边缘部署的小模型(称作SDM-D),显著提升推理速度与感知精度。该方法在多种水果检测任务(包括目标检测、语义分割、实例分割)中表现优异,性能几乎媲美有大量标注数据训练的模型。值得注意的是,SDM-D在所有测试水果数据集上均优于Grounding SAM和YOLO-World等开放集检测方法。此外,我们构建了包含超过25,000张图像的MegaFruits水果分割数据集,所有代码与数据已公开于https://github.com/AgRoboticsResearch/SDM-D.git。

原文摘要 · Abstract (English)

Recent breakthroughs in large foundation models have enabled the possibility of transferring knowledge pre-trained on vast datasets to domains with limited data availability. Agriculture is one of the domains that lacks sufficient data. This study proposes a framework to train effective, domain-specific, small models from foundation models without manual annotation. Our approach begins with SDM (Segmentation-Description-Matching), a stage that leverages two foundation models: SAM2 (Segment Anything in Images and Videos) for segmentation and OpenCLIP (Open Contrastive Language-Image Pretraining) for zero-shot open-vocabulary classification. In the second stage, a novel knowledge distillation mechanism is utilized to distill compact, edge-deployable models from SDM, enhancing both inference speed and perception accuracy. The complete method, termed SDM-D (Segmentation-Description-Matching-Distilling), demonstrates strong performance across various fruit detection tasks object detection, semantic segmentation, and instance segmentation) without manual annotation. It nearly matches the performance of models trained with abundant labels. Notably, SDM-D outperforms open-set detection methods such as Grounding SAM and YOLO-World on all tested fruit detection datasets. Additionally, we introduce MegaFruits, a comprehensive fruit segmentation dataset encompassing over 25,000 images, and all code and datasets are made publicly available at https://github.com/AgRoboticsResearch/SDM-D.git.

水果检测零样本知识蒸馏农业AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。