arXiv:2512.15310cs.CV2025-12被引 3

用大模型自动生成无真实图像的分割训练数据,实现零样本弱监督分割。

SynthSeg-Agents: Multi-Agent Synthetic Data Generation for Zero-Shot Weakly Supervised Semantic Segmentation

  • 通过多智能体框架自动生成语义丰富的合成图像提示并生成图像。
  • 在PASCAL VOC 2012和COCO 2014上实现媲美真实数据的分割性能。
  • 无需真实图像,适合低成本、可扩展的语义分割应用。

弱监督语义分割(WSSS)旨在仅使用图像级标签生成像素级预测,无需密集标注。现有方法虽利用生成模型扩充数据,但仍依赖真实训练样本。本文提出零样本弱监督分割(ZSWSSS),并设计由大语言模型驱动的多智能体框架SynthSeg Agents,完全不依赖真实图像生成合成训练数据。该框架包含两个核心模块:自迭代优化提示的提示生成智能体与基于视觉语言模型生成图像的图像生成智能体。前者通过循环优化、记忆机制和提示空间探索,生成多样化且语义丰富的图像提示;后者利用VLM生成候选图像,再通过冻结的CLIP评分模型筛选高质量样本,并训练一个ViT分类器对整个合成数据集进行语义精标。实验表明,在PASCAL VOC 2012和COCO 2014上,该框架无需真实图像即达到具有竞争力的性能,展现出大模型驱动智能体在高效、可扩展语义分割中的潜力。

原文摘要 · Abstract (English)

Weakly Supervised Semantic Segmentation (WSSS) with image level labels aims to produce pixel level predictions without requiring dense annotations. While recent approaches have leveraged generative models to augment existing data, they remain dependent on real world training samples. In this paper, we introduce a novel direction, Zero Shot Weakly Supervised Semantic Segmentation (ZSWSSS), and propose SynthSeg Agents, a multi agent framework driven by Large Language Models (LLMs) to generate synthetic training data entirely without real images. SynthSeg Agents comprises two key modules, a Self Refine Prompt Agent and an Image Generation Agent. The Self Refine Prompt Agent autonomously crafts diverse and semantically rich image prompts via iterative refinement, memory mechanisms, and prompt space exploration, guided by CLIP based similarity and nearest neighbor diversity filtering. These prompts are then passed to the Image Generation Agent, which leverages Vision Language Models (VLMs) to synthesize candidate images. A frozen CLIP scoring model is employed to select high quality samples, and a ViT based classifier is further trained to relabel the entire synthetic dataset with improved semantic precision. Our framework produces high quality training data without any real image supervision. Experiments on PASCAL VOC 2012 and COCO 2014 show that SynthSeg Agents achieves competitive performance without using real training images. This highlights the potential of LLM driven agents in enabling cost efficient and scalable semantic segmentation.

弱监督生成数据多智能体零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。