arXiv:2411.17176cs.CVcs.AI2024-11CVPR被引 10

让用户自由聊天描述需求,自动完成图像生成全流程。

ChatGen: Automatic Text-to-Image Generation From FreeStyle Chatting

  • 通过多阶段进化策略,自动解析用户自由对话中的生成需求。
  • 在自建基准上,生成准确率和图像质量显著优于基线方法。
  • 适合希望免调参、快速生成图像的研究者与普通用户。

尽管文本到图像生成模型取得了显著进展,用户在实际应用中仍面临试错难题,主要源于撰写合适提示词、选择模型及配置参数等繁琐步骤。本文提出自动文本到图像生成框架,允许用户以自由对话方式描述需求,自动完成全部生成流程。为此,我们构建了首个专门针对自动文本到图像生成的基准测试集ChatGenBench,包含高质量配对数据与多样化自由输入,支持对自动化流程各环节的全面评估。考虑到该任务具有多步推理复杂性,我们设计了多阶段进化策略ChatGen-Evo,逐步赋予模型自动化能力。在分步准确率与图像质量上的广泛评测表明,ChatGen-Evo显著超越多种基线模型。研究还揭示了推动自动文本到图像生成发展的关键洞见。所有数据、代码与模型将公开于https://chengyou-jia.github.io/ChatGen-Home。

原文摘要 · Abstract (English)

Despite the significant advancements in text-to-image (T2I) generative models, users often face a trial-and-error challenge in practical scenarios. This challenge arises from the complexity and uncertainty of tedious steps such as crafting suitable prompts, selecting appropriate models, and configuring specific arguments, making users resort to labor-intensive attempts for desired images. This paper proposes Automatic T2I generation, which aims to automate these tedious steps, allowing users to simply describe their needs in a freestyle chatting way. To systematically study this problem, we first introduce ChatGenBench, a novel benchmark designed for Automatic T2I. It features high-quality paired data with diverse freestyle inputs, enabling comprehensive evaluation of automatic T2I models across all steps. Additionally, recognizing Automatic T2I as a complex multi-step reasoning task, we propose ChatGen-Evo, a multi-stage evolution strategy that progressively equips models with essential automation skills. Through extensive evaluation across step-wise accuracy and image quality, ChatGen-Evo significantly enhances performance over various baselines. Our evaluation also uncovers valuable insights for advancing automatic T2I. All our data, code, and models will be available in \url{https://chengyou-jia.github.io/ChatGen-Home}

文本生成图像自动提示对话生成多阶段推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。