提出新数据集与方法,解决工业缺陷少样本分割难题
Exploring Few-Shot Defect Segmentation in General Industrial Scenarios with Metric Learning and Vision Foundation Models
- 基于度量学习与视觉基础模型构建新方法
- 发现视觉基础模型在多种缺陷场景中表现更优
- 适合工业质检、少样本学习研究者参考
工业缺陷分割对制造质量控制至关重要。由于缺陷样本稀缺,少样本语义分割(FSS)在此领域具有重要价值。然而,现有研究多聚焦于简单纹理上的缺陷,未覆盖更复杂的工业场景。本文旨在填补这一空白,探索多样化工业产品中的少样本缺陷分割(FDS)。为此,我们构建了一个真实世界的新数据集,并重组部分现有数据集,形成更全面的FDS基准。在该基准上,我们系统评估了基于度量学习的FSS方法,包括元学习与视觉基础模型(VFMs)两类。结果表明,现有元学习方法普遍不适用,而VFMs潜力显著。我们进一步研究了不同VFMs在特征匹配与使用Segment Anything(SAM)模型两种范式下的表现,提出一种基于特征匹配的高效新方法。同时发现,通过视频追踪模式,SAM2在解决FDS任务中尤为有效。所贡献的数据集与代码将开源:https://github.com/liutongkun/GFDS。
原文摘要 · Abstract (English)
Industrial defect segmentation is critical for manufacturing quality control. Due to the scarcity of training defect samples, few-shot semantic segmentation (FSS) holds significant value in this field. However, existing studies mostly apply FSS to tackle defects on simple textures, without considering more diverse scenarios. This paper aims to address this gap by exploring FSS in broader industrial products with various defect types. To this end, we contribute a new real-world dataset and reorganize some existing datasets to build a more comprehensive few-shot defect segmentation (FDS) benchmark. On this benchmark, we thoroughly investigate metric learning-based FSS methods, including those based on meta-learning and those based on Vision Foundation Models (VFMs). We observe that existing meta-learning-based methods are generally not well-suited for this task, while VFMs hold great potential. We further systematically study the applicability of various VFMs in this task, involving two paradigms: feature matching and the use of Segment Anything (SAM) models. We propose a novel efficient FDS method based on feature matching. Meanwhile, we find that SAM2 is particularly effective for addressing FDS through its video track mode. The contributed dataset and code will be available at: https://github.com/liutongkun/GFDS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。