用合成数据训练工业分割模型,无需真实图像或人工标注。
Industrial Synthetic Segment Pre-training
- 基于公式驱动的监督学习生成合成数据,聚焦复杂遮挡场景
- 在5个工业领域平均mAP达46.0,媲美ImageNet-21k预训练
- 仅需10万张图像,适合资源受限场景
视觉基础模型(VFMs)在真实工业场景的分割任务中表现日益突出。然而,基于真实图像数据集预训练的VFMs仍面临三大挑战:(1)因与自然图像差异大,工业数据上表现不佳;(2)商业使用受法律伦理限制,可扩展性差;(3)在计算与数据资源有限条件下构建训练框架仍具挑战。这引出核心问题:能否不依赖真实图像或人工标注构建工业分割模型?为此,我们提出实例核心分割数据集(InsCore),一个基于公式驱动监督学习(FDSL)的合成数据生成框架及其预训练数据集。InsCore不依赖真实图像的视觉外观或领域,而是假设:在预训练阶段学会处理复杂遮挡是工业领域强性能的关键。在医疗、生物医学、遥感、制造和物流五个领域上的实验表明,基于InsCore预训练的模型,使用ViTDet骨干网络时平均mAP为45.2,使用Swin Transformer骨干网络时为46.0,与ImageNet-21k监督预训练(45.0)相当,且完全未使用真实图像。作为对比,提示版SAM结合真值边界框在相同基准上达到45.4。此外,InsCore仅包含10万张图像和320万掩码,约为SA-1B数据集规模的1/110和1/312。
原文摘要 · Abstract (English)
Vision Foundation Models (VFMs) have made remarkable progress and are increasingly being applied to segmentation tasks in real-world industrial settings. However, VFMs pre-trained on real-image datasets still face several challenges: (1) they do not always perform well on industrial datasets due to significant differences from natural imagery, (2) legal and ethical restrictions, such as limitations on commercial use, constrain extensibility, and (3) building training frameworks under limited computational and data resources remains a critical issue. These challenges raise a fundamental question: can we construct industrial segmentation models without relying on real images or manual annotations? To address this question, we propose the Instance Core Segment Dataset (InsCore), a synthetic data generation framework and the resulting pre-training dataset based on Formula-Driven Supervised Learning (FDSL). InsCore is designed not around the visual appearance or domain of real images, but around the hypothesis that learning to handle complex occlusions during pre-training is a key factor for strong performance in industrial domains. Through experiments across five domains (medical, biomedical, remote sensing, manufacturing, and logistics) we demonstrate that InsCore pre-trained models achieve average mAP scores of 45.2 with the ViTDet backbone and 46.0 with the Swin Transformer backbone, on par with ImageNet-21k supervised pre-training (45.0) while using no real images at all. As a reference point under different input assumptions, prompted SAM with ground-truth bounding boxes attains 45.4 on the same benchmarks. Finally, InsCore consists of only 100k images and 3.2M masks, roughly 1/110 and 1/312 the scale of the SA-1B dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。