针对数据稀缺场景,生成能弥补模型弱点的高质量合成图像与标注。
Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios
- 用统一特征提取器和反馈驱动的挖掘模块,自动发现并生成难样本。
- 在MVTec上达到83.63%像素级AP,Polyp数据集上达81.64% mIoU。
- 无需为不同物体单独训练,适合医疗、工业等数据少领域使用。
医疗、工业和自动驾驶等领域常面临数据稀缺问题,导致模型过拟合与数据集不平衡,影响检测与分割性能。现有生成方法合成的样本重复或简单,无法提供针对性的‘关键信息’以弥补下游模型缺陷,且通常需为不同对象分别训练,计算效率低。为此,我们提出Crucial-Diff,一种无领域依赖的框架,用于生成关键训练样本。该方法包含两个核心模块:场景无关特征提取器(SAFE)通过统一特征提取器捕获目标信息;弱点感知样本挖掘器(WASM)基于下游模型检测结果的反馈,生成难检样本,并与SAFE输出融合。所提Crucial-Diff框架生成多样且高质量的数据,在MVTec上实现83.63%像素级AP和78.12% F1-MAX;在肠镜息肉数据集上达到81.64% mIoU和87.69% mDice。代码已公开于https://github.com/JJessicaYao/Crucial-diff。
原文摘要 · Abstract (English)
The scarcity of data in various scenarios, such as medical, industry and autonomous driving, leads to model overfitting and dataset imbalance, thus hindering effective detection and segmentation performance. Existing studies employ the generative models to synthesize more training samples to mitigate data scarcity. However, these synthetic samples are repetitive or simplistic and fail to provide "crucial information" that targets the downstream model's weaknesses. Additionally, these methods typically require separate training for different objects, leading to computational inefficiencies. To address these issues, we propose Crucial-Diff, a domain-agnostic framework designed to synthesize crucial samples. Our method integrates two key modules. The Scene Agnostic Feature Extractor (SAFE) utilizes a unified feature extractor to capture target information. The Weakness Aware Sample Miner (WASM) generates hard-to-detect samples using feedback from the detection results of downstream model, which is then fused with the output of SAFE module. Together, our Crucial-Diff framework generates diverse, high-quality training data, achieving a pixel-level AP of 83.63% and an F1-MAX of 78.12% on MVTec. On polyp dataset, Crucial-Diff reaches an mIoU of 81.64% and an mDice of 87.69%. Code is publicly available at https://github.com/JJessicaYao/Crucial-diff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。