用少量标注数据训练烘焙品检测模型,实测效果超全监督版本。
Learning to Detect Baked Goods with Limited Supervision
- 结合开放词汇检测与图像级标签,实现弱监督训练
- 仅用图像级标注即达mAP 0.91,伪标签微调提升19.3%
- 适合标注稀缺的工业视觉场景,如烘焙业智能监控
监测剩余产品可为优化生产提供关键洞察,尤其对保质期极短的德国烘焙业至关重要。自动化可降低人力成本、提升准确率并简化流程。本文提出使用目标检测模型从图像中识别烘焙品,但德国烘焙品种类繁多,全监督训练成本过高且难以扩展。尽管开放词汇检测器(如OWLv2、Grounding DINO)具灵活性,但实验表明其在该任务中表现不足。为此,我们构建了包含19类烘焙品、不同标注水平的数据集。提出两种弱监督训练流程:第一,结合OWLv2与Grounding DINO定位结果和图像级标签进行弱监督训练;第二,利用Segment Anything 2作为伪标签传播模型,在视频帧上微调以增强视角鲁棒性。基于YOLOv11的模型在仅依赖图像级标注下达到mAP 0.91,经伪标签微调后在非理想部署条件下性能提升19.3%。结合两种流程的模型在非理想条件下超越全监督基线,证明了弱监督策略的有效性。
原文摘要 · Abstract (English)
Monitoring leftover products provides valuable insights that can be used to optimize future production. This is especially important for German bakeries because freshly baked goods have a very short shelf life. Automating this process can reduce labor costs, improve accuracy, and streamline operations. We propose automating this process using an object detection model to identify baked goods from images. However, the large diversity of German baked goods makes fully supervised training prohibitively expensive and limits scalability. Although open-vocabulary detectors (e.g., OWLv2, Grounding DINO) offer lexibility, we demonstrate that they are insufficient for our task. While motivated by bakeries, our work addresses the broader challenges of deploying computer vision in industries, where tasks are specialized and annotated datasets are scarce. We compile dataset splits with varying supervision levels, covering 19 classes of baked goods. We propose two training workflows to train an object detection model with limited supervision. First, we combine OWLv2 and Grounding DINO localization with image-level supervision to train the model in a weakly supervised manner. Second, we improve viewpoint robustness by fine-tuning on video frames annotated using Segment Anything 2 as a pseudo-label propagation model. Using these workflows, we train YOLOv11 for our detection task due to its favorable speed accuracy tradeoff. Relying solely on image-level supervision, the model achieves a mean Average Precision (mAP) of 0.91. Finetuning with pseudo-labels raises model performance by 19.3% under non-ideal deployment conditions. Combining these workflows trains a model that surpasses our fully-supervised baseline model under non-ideal deployment conditions, despite relying only on image-level supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。