arXiv:2602.02051cs.AI2026-02

无需训练的智能图像生成代理,自动优化提示并修正缺陷。

SIDiffAgent: Self-Improving Diffusion Agent

  • 用多模型协同自主优化提示和生成过程
  • 在GenAIBench上达成0.884的VQA得分,超越多数开源与闭源模型
  • 通过记忆库实现迭代自我改进,适合需要稳定输出的场景

文本到图像扩散模型已推动生成式AI发展,实现高质量、逼真的图像合成。但其实际应用仍受多重限制:对提示词表述敏感、语义模糊(如“mouse”指动物或鼠标)、出现解剖畸变等瑕疵,且需精心设计提示。现有方法通常需额外训练,可控性有限,难以适应真实场景。我们提出无需训练的自提升扩散代理SIDiffAgent,利用Qwen系列模型(Qwen-VL、Qwen-Image、Qwen-Edit、Qwen-Embedding)实现自主提示工程、识别并修正低质生成结果,以及细粒度瑕疵去除,显著提升输出可靠性与一致性。该框架通过数据库存储过往经验,实现迭代自我优化,将历史经验转化为各阶段的提示引导。在GenAIBench测试中,平均VQA得分为0.884,显著优于开源、闭源模型及现有代理方法。代码将在论文接受后公开。

原文摘要 · Abstract (English)

Text-to-image diffusion models have revolutionized generative AI, enabling high-quality and photorealistic image synthesis. However, their practical deployment remains hindered by several limitations: sensitivity to prompt phrasing, ambiguity in semantic interpretation (e.g., ``mouse" as animal vs. a computer peripheral), artifacts such as distorted anatomy, and the need for carefully engineered input prompts. Existing methods often require additional training and offer limited controllability, restricting their adaptability in real-world applications. We introduce Self-Improving Diffusion Agent (SIDiffAgent), a training-free agentic framework that leverages the Qwen family of models (Qwen-VL, Qwen-Image, Qwen-Edit, Qwen-Embedding) to address these challenges. SIDiffAgent autonomously manages prompt engineering, detects and corrects poor generations, and performs fine-grained artifact removal, yielding more reliable and consistent outputs. It further incorporates iterative self-improvement by storing a memory of previous experiences in a database. This database of past experiences is then used to inject prompt-based guidance at each stage of the agentic pipeline. \modelour achieved an average VQA score of 0.884 on GenAIBench, significantly outperforming open-source, proprietary models and agentic methods. We will publicly release our code upon acceptance.

扩散模型图像生成智能代理自提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。