用智能代理生成高质量图像数据，解决小样本类别标注难题。
Gen-n-Val: Agentic Image Data Generation and Validation
- 用大模型优化提示词，驱动扩散模型生成单物体高清图像与分割图。
- 合成数据无效率从50%降至7%，稀有类别检测性能提升7.6%以上。
- 适合需要海量高质量合成数据的视觉模型训练者使用。
数据稀缺、标签噪声和长尾类别不平衡仍是目标检测与实例分割等计算机视觉任务中的关键挑战，尤其在包含大量词汇的大规模基准如LVIS上，多数类别仅出现在少数图像中。现有合成数据生成方法仍存在单掩码含多个对象、分割不准确、类别标签错误等问题，制约其有效性。为此，我们提出Gen-n-Val，一种基于智能体的新型数据生成框架，融合层扩散（LD）、大语言模型（LLM）与视觉语言大模型（VLLM），用于生成高质量且多样化的实例掩码与图像。该框架包含两个智能体：(1) LD提示智能体（基于LLM），优化提示以引导LD生成高质量单物体前景图像及对应分割掩码；(2) 数据验证智能体（基于VLLM），过滤低质量合成实例图像。两智能体提示通过TextGrad进行优化。相比当前最优方法MosaicFusion，本方法将无效合成数据比例从50%降至7%，在Mask R-CNN的LVIS实例分割任务中稀有类别性能提升7.6%，在YOLOv9c与YOLO11m的COCO实例分割任务中稀有类别mAP提升3.6%。此外，在开放词汇目标检测基准上，相较YOLO-Worldv2-M，Gen-n-Val在YOLO11m上实现7.1% mAP提升。该框架具备模型容量与数据规模可扩展性。代码已开源：https://github.com/aiiu-lab/Gen-n-Val。
原文摘要 · Abstract (English)
The data scarcity, label noise, and long-tailed category imbalance remain important and unresolved challenges in many computer vision tasks, such as object detection and instance segmentation, especially on large-vocabulary benchmarks like LVIS, where most categories appear in only a few images. Current synthetic data generation methods still suffer from multiple objects per mask, inaccurate segmentation, incorrect category labels, and other issues, limiting their effectiveness. To address these issues, we introduce Gen-n-Val, a novel agentic data generation framework that leverages Layer Diffusion (LD), a Large Language Model (LLM), and a Vision Large Language Model (VLLM) to produce high-quality and diverse instance masks and images for object detection and instance segmentation. Gen-n-Val consists of two agents: (1) the LD prompt agent, an LLM, optimizes rompts to encourage LD to generate high-quality foreground single-object images and corresponding segmentation masks; and (2) the data validation agent, a VLLM, filters out low-quality synthetic instance images. The system prompts for both agents are optimized by TextGrad. Compared to state-of-the-art synthetic data approaches like MosaicFusion, our approach reduces invalid synthetic data from 50% to 7% and improves performance by 7.6% on rare classes in LVIS instance segmentation with Mask R-CNN, and by 3.6% mAP on rare classes in COCO instance segmentation with YOLOv9c and YOLO11m. Furthermore, Gen-n-Val shows significant improvements (7.1% mAP) over YOLO-Worldv2-M in open-vocabulary object detection benchmarks with YOLO11m. Moreover, Gen-n-Val has scalability in model capacity and dataset size. The code is available at https://github.com/aiiu-lab/Gen-n-Val.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。