通过自进化多智能体框架,让被拒绝的数据反哺训练,提升图文生成质量。
DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation

- 用四个智能体协作:检索、验证、批评、生成,形成反馈闭环。
- 在0.75M数据量下,OCR-F1比最强基线提升85.3%(TextScenesHQ)和35.3%(LongTextBench)。
- 适合需要高质量图文生成的场景,如广告设计、文档合成。
文本丰富的图像生成是极具挑战的任务,模型需同时生成视觉真实、文字清晰、语义一致且布局合理的图像。现有数据构建流程采用静态采集-筛选-冻结范式,一旦样本被拒即丢弃,但这些失败样本常包含可利用的错误信号,如OCR错误和语义错位,导致后续阶段重复犯错。为此,本文提出DataEvolver——一种自进化多智能体数据构建框架。该框架将数据构建视为反馈驱动的策略演化过程:检索器收集候选样本,验证器赋予质量评分并标注拒因,批评者将每轮反馈总结为语义级反馈,生成器则针对未覆盖区域进行定向合成。更新后的反馈记忆引导下一阶段构建。在多个文本丰富图像生成基准上的实验表明,在相同数据预算下,DataEvolver生成的数据优于固定数据集基线。在PixArt-alpha上以0.75M规模运行时,其在TextScenesHQ上的OCR-F1较最强基线提升85.3%,在LongTextBench上提升35.3%。结果在两个评测集上均一致,并可迁移至Show-o2,说明其优势不依赖特定生成器。这表明被拒样本能提供可操作的反馈,用于持续优化图文数据构建。
原文摘要 · Abstract (English)
Text-rich image generation is one of the most challenging settings in image generation, since models must simultaneously produce visually realistic images and render legible, semantically aligned, and layout-consistent text. Existing data pipelines usually follow a static crawl-filter-freeze paradigm. They collect candidate samples, filter them once, and freeze the accepted data for training. However, rejected samples are usually discarded, although they often contain useful failure signals such as OCR errors and semantic mismatches. As a result, later construction rounds may repeat the same failure modes. To address these limitations, we propose DataEvolver, a self-evolving multi-agent framework for text-rich image data construction. DataEvolver treats data construction as feedback-driven construction policy evolution. A Retriever collects candidate samples, a Verifier assigns quality scores and rejection causes, a Critic summarizes round-level feedback into semantic feedback, and a Generator completes under-covered regions through targeted synthesis. The updated feedback memory then guides the next construction round. Experiments on text-rich image generation benchmarks show that DataEvolver produces more useful training data than fixed-dataset baselines under matched data budgets. At the 0.75M scale on PixArt-alpha, DataEvolver improves OCR-F1 over the strongest baseline by 85.3 percent on TextScenesHQ and 35.3 percent on LongTextBench. The improvements are consistent across both evaluated benchmarks and also transfer to Show-o2, indicating that the benefit of DataEvolver is not tied to a single downstream generator. These results suggest that rejected samples can provide actionable feedback for improving text-rich image data construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。